Performance considerations for async task pool sizes?

[postmodern]

I remember reading somewhere that it's recommended to not allocate potentially infinite number of async tasks, but to set a fixed pool size. I'm designing a rule-based async scanner that tests input values against a potentially large number of rules which contain the test logic. I'm debating whether I should have a fixed pool size of async tasks with a queue containing the rule and value pairs to evaluate, or whether I should allocate an async task per-every rule (and maybe allow configuring the concurrency for individual rules).

[ioquatix]

There is no easy answer to this as it’s impossible to predict future workloads but one way to try and balance the number of tasks with the available hardware resources is https://github.com/socketry/async/blob/main/lib/async/idler.rb

[postmodern]

Do you have an example of how to use Async::Idler?

[trevorturk]

Perhaps socketry/async-job@2e5745e

[postmodern]

One of the scenarios I'm also worried about is whether I get a large amount of values which take a long time for certain rules to process (maybe it takes a long time to connect to the remote port or the HTTP server is heavily rate limited) and having the async task pool get "clogged up". This would require the user to Ctrl^C the command and restart it with a larger pool size until they found the ideal pool size to "unclog" the queue.

[ioquatix]

Yes, using a fixed sized pool is a bad idea, and using work stealing is a good idea, e.g. use async-job which handles these concerns.

[postmodern]

@ioquatix does async-job allow passing values back out of the jobs? I also need an output queue of sorts to yield results back to a callback.

[ioquatix]

This is not directly supported at this time. If you consider your work flow (usually a form of map-reduce or fanout), you have a client producing work, and waiting for one or more of those jobs to finish.

There are some challenges associated with this: maybe the background job queue is not running, etc.

You need to consider how to manage this state, for example you might use a counter, or a queue in Redis, or a database table, to capture the "values back out of the jobs". It might be nice to provide a default synchronization mechanism for this in async-job, or maybe it is backend specific.

For reference I believe parallel_workforce is also relevant from a design point of view. https://github.com/EnjoyTech/parallel_workforce so maybe we can take some inspiration from it.