Starting Multiple RQ Workers

[jvanderaa]

If you need to have multiple RQ workers (only workers on Nautobot 1.0.x) using a local install, you will need to make systemd units for the multiple workers. This is then helpful if you want to have multiple processes running while using the Django RQ system. This may not be necessary for those that have Celery workers, as those have a different process for expanding how many jobs are run.

https://python-rq.org/patterns/systemd/
https://www.stevenrombauts.be/2019/01/run-multiple-instances-of-the-same-systemd-unit/

Steps

1. Rename RQ Worker systemd file

sudo mv /etc/systemd/system/nautobot-rq-worker.service /etc/systemd/system/nautobot-rq-worker@.service

2. Execute Daemon Reload

sudo systemctl daemon-reload

3. Start services as necessary (this creates 4 workers)

$ sudo systemctl start nautobot-rq-worker@1.service
$ sudo systemctl start nautobot-rq-worker@2.service
$ sudo systemctl start nautobot-rq-worker@3.service
$ sudo systemctl start nautobot-rq-worker@4.service

[nniehoff]

What if these are celery workers?

[jtdub]

Celery starts four workers by default.

https://docs.celeryproject.org/en/2.0-archived/configuration.html