Migrating to Solid Queue

[claudiocebpaz]

What do you think about moving to the new rails 8 solid queue? Moving away from sidekiq?

[nickjj]

Hi,

I'm reluctant to initially switch to it but I think maybe long term it could be ok.

Concerns

Performance

Basecamp and Hey have tested Solid Queue but they have dedicated hardware with some of the best NVME SSD drives you can get.

Most general purpose VPS (DigitalOcean, etc.) have no where near that disk performance.

I'd like to how see Solid Queue performs on a real app with a low end VPS where Postgres is doing everything (SQL, cache, queue and action cable).

Simplicity

Given we're using Docker Compose, we already have Redis up and running with zero hassle.

There's no real complexity decompression for dropping Redis with this project.

Stability

Sidekiq has been run through the ringer for over a decade. Solid Queue is basically brand new. I'm sure there's rough edges and that's not to put down the team working on it. It's just the reality of any software project. Sidekiq has been used on tons of projects.

Long term wins of switching to Solid Queue

• Once Solid Queue has feature parity with Sidekiq (+ the free / open source Sidekiq plugins) then it becomes more compelling
• Sidekiq is great but it is commercial software if you want to have some of those useful features, if your app is brand new you might not be ready to pay for Sidekiq pro or enterprise but still want those features
  • A stop gap for this is to use third party free plugins that sort of do the same thing but aren't "officially" supported

Overall I want to switch but I'm not comfortable with it yet, especially if you plan to stick with Postgres. It seems like Good Job is more performant because it uses Postgres specific DB features that don't exist in MySQL or SQLite but Solid Queue wants to remain compatible with all 3 DBs so unless a drastic change is made with Solid Queue it may always be slower than Good Job.

The question there is, is it "fast enough"? I'm not sure yet, it's possible.

I'd like to see an extensive set of real world benchmarks be done on a low end VPS between all 3 solutions, or at least Solid Queue vs Sidekiq.