Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flamenco, repair: cache good repair peers to file #3878

Merged
merged 1 commit into from
Jan 8, 2025

Conversation

topointon-jump
Copy link
Contributor

We periodically cache the known good repair peers to a file, so that we can request from these when we restart. This reduces startup times.

@topointon-jump topointon-jump changed the title flamenco, repair: cache good repair peers to file [wip] flamenco, repair: cache good repair peers to file Jan 7, 2025
@mmcgee-jump
Copy link
Contributor

This has sandbox issues and seems like a bit of a workaround. Why is it slow to get a fresh list of repair peers on boot?

@topointon-jump topointon-jump force-pushed the topointon/repair-cache-peers branch 4 times, most recently from fac99c2 to f8191f7 Compare January 8, 2025 18:59
src/flamenco/repair/fd_repair.c Show resolved Hide resolved
src/app/fdctl/run/tiles/repair.seccomppolicy Show resolved Hide resolved
src/flamenco/repair/fd_repair.c Show resolved Hide resolved
src/flamenco/repair/fd_repair.c Show resolved Hide resolved
src/flamenco/repair/fd_repair.c Show resolved Hide resolved
@@ -559,17 +568,133 @@ fd_repair_decay_stats( fd_repair_t * glob ) {
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general alignment + spacing nits

@topointon-jump topointon-jump force-pushed the topointon/repair-cache-peers branch from ff4620d to 2151c9e Compare January 8, 2025 19:39
@topointon-jump
Copy link
Contributor Author

topointon-jump commented Jan 8, 2025

This has sandbox issues

Corrected the approach to set up the sandbox properly, thanks for the spot 😄

Why is it slow to get a fresh list of repair peers on boot?

It is slow because we don't have the stake weights until the full snapshot is loaded, and we don't have the IP addresses of the nodes until these are fetched through gossip. We are working on making both of these faster but in the meantime this cache also helps.

@topointon-jump topointon-jump changed the title [wip] flamenco, repair: cache good repair peers to file flamenco, repair: cache good repair peers to file Jan 8, 2025
@topointon-jump topointon-jump added this pull request to the merge queue Jan 8, 2025
Merged via the queue into main with commit 82ecf83 Jan 8, 2025
10 checks passed
@topointon-jump topointon-jump deleted the topointon/repair-cache-peers branch January 8, 2025 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants