Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please allow to disable hubble node auto-upgrade cronjob #2381

Open
metatxn opened this issue Oct 25, 2024 · 0 comments
Open

Please allow to disable hubble node auto-upgrade cronjob #2381

metatxn opened this issue Oct 25, 2024 · 0 comments
Labels
s-triage Needs to be reviewed, designed and prioritized

Comments

@metatxn
Copy link

metatxn commented Oct 25, 2024

What is the bug?

There are multiple issues that i'm facing when hubble auto-upgrade cronjob is set to true:

  1. when autoupgrader cron job updates files and restarts the Hubble containers, the Hubble container becomes unhealthy and stops functioning. This occurs consistently during the auto-update process. Below are the logs from hubble container.
  2. Please allow grafana service to read the environment vars from grafana.env file. The reason is I'm using Caddy as a reverse proxy to manage SSL certificates so that Hubble and Grafana can run over HTTPS. To enable Grafana to work with Caddy over HTTPS, I need to add the environment variables like GF_SERVER_ROOT_URL, GF_SERVER_SERVE_FROM_SUB_PATH to the Grafana service. However, each time the cron job updates the docker-compose.yml file, these environment variables are lost, causing Grafana to stop working over HTTPS since Caddy can no longer route requests to it correctly.

Hubble container logs

{"message":"\n Hubble Startup Checks\n","timestamp":"2024-10-24T06:00:35.999Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"------------------------\n","timestamp":"2024-10-24T06:00:35.999Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"✅ | Farcaster: 2024.10.16 Hubble: 1.16.0\n","timestamp":"2024-10-24T06:00:35.999Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"✅ | Detected 15GB of RAM\n","timestamp":"2024-10-24T06:00:35.999Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"✅ | Directory /home/node/app/apps/hubble/.rocks is writable\n","timestamp":"2024-10-24T06:00:35.999Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"✅ | Directory /home/node/app/apps/hubble/.hub is writable\n","timestamp":"2024-10-24T06:00:36.000Z","type":"out","process_id":0,"app_name":"hubble"}
{"message":"✅ | Found PeerId 12D3KooWDY2gx6CnJ7Gg8XNQpfe5EpafWstBUjeHC4FhFP2EjYR5\n","timestamp":"2024-10-24T06:00:36.004Z","type":"out","process_id":0,"app_name":"hubble"}
thread '<unnamed>' panicked at src/statsd.rs:63:14:
called `Result::unwrap()` on an `Err` value: MetricError { repr: IoError(Custom { kind: Uncategorized, error: "failed to lookup address information: Temporary failure in name resolution" }) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: internal error in Neon module: called `Result::unwrap()` on an `Err` value: MetricError { repr: IoError(Custom { kind: Uncategorized, error: "failed to lookup address information: Temporary failure in name resolution" }) }
    at rsCreateStatsdClient (file:///home/node/app/apps/hubble/src/rustfunctions.ts:90:7)
    at initializeStatsd (file:///home/node/app/apps/hubble/src/utils/statsd.ts:41:3)
    at Command.<anonymous> (file:///home/node/app/apps/hubble/src/cli.ts:401:9)
You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: internal error in Neon module: called `Result::unwrap()` on an `Err` value: MetricError { repr: IoError(Custom { kind: Uncategorized, error: "failed to lookup address information: Temporary failure in name resolution" }) }
    at rsCreateStatsdClient (file:///home/node/app/apps/hubble/src/rustfunctions.ts:90:7)
    at initializeStatsd (file:///home/node/app/apps/hubble/src/utils/statsd.ts:41:3)
    at Command.<anonymous> (file:///home/node/app/apps/hubble/src/cli.ts:401:9)
@github-actions github-actions bot added the s-triage Needs to be reviewed, designed and prioritized label Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
s-triage Needs to be reviewed, designed and prioritized
Projects
None yet
Development

No branches or pull requests

1 participant