Add random offset capabilities to desynchronize parallel workers #615
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, this is a feature request plus code to implement it--I'd be happy to adjust the DSL or implementation if you had different ideas for how to do this. The idea is to implement the pattern described here where when several nodes are running the same code, we avoid having their crons all fire at the same time by starting each job with a random sleep.
My thought was to make the sleep distribution configurable on a per-list level, so that you can have some jobs be fuzzier than others (have a daily job have an hourly offset, an hourly job have a 2 minute offset). So the schedule.rb could look like
Jobs in the first list would fire anywhere in the first four minutes of the hour, while jobs in the second list will fire anywhere between 12:00 and 2:00 PM, with any second being equally likely.