Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When database is down, worker will down #303

Closed
vitalyzhakov opened this issue Jan 17, 2019 · 12 comments
Closed

When database is down, worker will down #303

vitalyzhakov opened this issue Jan 17, 2019 · 12 comments
Labels
type:docs Documentation

Comments

@vitalyzhakov
Copy link

vitalyzhakov commented Jan 17, 2019

What steps will reproduce the problem?

  1. Install yii2-queue, using DB Mysql driver
  2. Start process via supervisord https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/worker.md
  3. Kill database or drop table queue
  4. In php error.log see errors, processes will died.
  5. When achieving max number of retries, worker will down forever.
  6. Up database. Worker died.

What's expected?

Worker live when database is live.

What do you get instead?

2019-01-16 12:19:23,808 INFO gave up: yii-queue-worker_01 entered FATAL state, too many start retries too quickly
2019-01-16 12:19:23,814 INFO exited: yii-queue-worker_00 (exit status 1; not expected)
2019-01-16 12:19:24,815 INFO gave up: yii-queue-worker_00 entered FATAL state, too many start retries too quickly

Workers is down.

Additional info

Q A
Yii version 2.0.15.1
PHP version 7.1.14
Operating system Debian 8
@samdark
Copy link
Member

samdark commented Jan 17, 2019

That should be solved with supervisor.

@samdark samdark closed this as completed Jan 17, 2019
@samdark
Copy link
Member

samdark commented Jan 17, 2019

@vitalyzhakov
Copy link
Author

@samdark ,
I did not find the solution about this issue on docs page
https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/worker.md

Can you tell solution?

@samdark
Copy link
Member

samdark commented Jan 17, 2019

Do you really want it trying to connect to dead database forever writing to logs endlessly?

@samdark
Copy link
Member

samdark commented Jan 17, 2019

autorestart=true this will ensure supervisord will try to start worker again if it terminated abnormally.

@vitalyzhakov
Copy link
Author

Do you really want it trying to connect to dead database forever writing to logs endlessly?

@samdark , I think, automatic disaster recovery is the best way.

But if it is difficult now to develop, description about this library behavior and example solution may be situate in docs.

If I understand, supervisord config file must include startretries=99999. Is it right?

@samdark samdark reopened this Jan 18, 2019
@samdark
Copy link
Member

samdark commented Jan 18, 2019

Yes. The time between retries will increment exponentially though.

@samdark samdark added the type:docs Documentation label Jan 18, 2019
@vitalyzhakov
Copy link
Author

@samdark , hello!
The time between retries will increment exponentially though.
Supervisord does not have this option now Supervisor/supervisor#659

@samdark
Copy link
Member

samdark commented Jan 30, 2019

It is exponential by default.

@YanAlex
Copy link

YanAlex commented Oct 28, 2021

There is a problem with Windows, there are no supervisot support

@samdark
Copy link
Member

samdark commented Oct 28, 2021

@samdark
Copy link
Member

samdark commented Oct 28, 2021

Closing since actually it's mentioned in current docs.

@samdark samdark closed this as completed Oct 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:docs Documentation
Projects
None yet
Development

No branches or pull requests

3 participants