Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch to hosted influxdb for stats #164

Open
13 of 20 tasks
alsuren opened this issue Mar 4, 2023 · 2 comments
Open
13 of 20 tasks

switch to hosted influxdb for stats #164

alsuren opened this issue Mar 4, 2023 · 2 comments

Comments

@alsuren
Copy link
Collaborator

alsuren commented Mar 4, 2023

I think that our redis instance is maybe full or something. I've archived some of the stats onto #106 but I didn't delete them from redis. Maybe I should do that now.

Crazy plan:

  • delete stats from redis to get us up and running again
  • stop using redis for stats
    • pick influxdb IOx is now the default and the free tier pricing is generous - https://www.influxdata.com/influxdb-cloud-pricing/ (5MB/5 minutes, with 30day retention)
      • set up an org
      • add @NobodyXu (currently sharing the org with my backup mijia-homie monitoring, but I will probably bin that soon)
      • add other maintainers (email addresses please)
    • (optional) make the existing stats server proxy http requests onto another stats server that we can all mess about with (or fan out and give everyone their own "staging" server?)
    • write stats to influxdb using the http api
    • make github actions cronjob read from influxdb directly
      • select sum of all counts from the previous day, for the selected arch, grouped by package name
    • (optional) make a cronjob that archives stats onto a git branch.
    • (optional) make a thing to roll-up the data into one point per hour per (crate, version, arch, client) (summing up the counts for each tag) and set the milisecond-precision database retention policy to 1 week or something

Outstanding questions:

  • Should we switch away from typescript while we are do this?
  • Should we use this as an opportunity to fold the stats server code into this monorepo?
@NobodyXu
Copy link
Member

NobodyXu commented Mar 4, 2023

@cargo-bins/binstall can you guys all provide your email address please?

@passcod
Copy link
Member

passcod commented Mar 5, 2023

make a thing to roll-up the data into one point per hour per (crate, version, arch, client) (summing up the counts for each tag) and set the milisecond-precision database retention policy to 1 week or something

influx can do that for you btw, it's a core feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants