Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate API history more efficiently than querying the entire history at once by having a slow and fast update system #22

Open
AlexLipp opened this issue May 7, 2024 · 0 comments

Comments

@AlexLipp
Copy link
Owner

AlexLipp commented May 7, 2024

Currently the history bar-code is generated, using POOPy, by querying the entire history for every monitor going back to the start of time (May 2022). Whilst comprehensive, this system is very intensive and takes a long time. Additionally, as time progresses, more and more data will be stored in the API making it even more intensive to query. Additionally, it increases the chances of a request being bounced which will terminate the entire process increasing downtime (also undesirable).

A better approach would be to only occasionally (e.g., weekly) refresh very old data (which does not update very often) and instead only update the last week (say). This would reduce the number of overall requests made, increase the speed of the more frequent updates, and reduce the chance of downtime from bad requests.

@AlexLipp AlexLipp changed the title Generate API history more efficiently than querying the entire history at once Generate API history more efficiently than querying the entire history at once by having a slow and fast update system May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant