Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipelining Followups #141

Open
tzaffi opened this issue Aug 18, 2023 · 0 comments
Open

Pipelining Followups #141

tzaffi opened this issue Aug 18, 2023 · 0 comments

Comments

@tzaffi
Copy link
Contributor

tzaffi commented Aug 18, 2023

Problem

Some issues remaining at the end of #128. However, the long validation process made it safer to push fixes as followups

Remove the *pImpl parameter from Retries() in conduit/pipeline/common.go

Per @winder

  • breaking out the context.Context out into a parameter, I was specifically looking for it to make sure the wait is properly terminated.
  • also breaking out RetryCount and logger. It's already a pure function in how it's used, but the signature doesn't guarantee that it's pure.

Alternatively, just make Retries private

Trim the special end of round message

In the final EC2 test, the size of the log at catchup grew from 14.1 GB to 16.2 GB. This is attributable to the change of the end of round log message. Currently it looks like

UPDATED Pipeline NextRound=31446285. FINISHED Pipeline round r=31446284 (6 txn) exported in 3.721356ms

In particular the UPDATED... prefix was added.

While this message requires a special format to comply with our testing (internal tools logstats.go and the E2E test), we can certainly trim it down and the UPDATED NextRound info is not crucial (it can be carved out as a DEBUG level message if not already present).

Urgency

Medium

@tzaffi tzaffi mentioned this issue Aug 18, 2023
10 tasks
This was referenced Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants