Rate limits

[gregziegan]

Hello! I'm working on a simple static site backed by a single Google Sheet. There's around 600 rows to this sheet. I fetch a single row for each page of the application, and the full sheet for a few filter/search pages. I've had great success with this approach, but it's now breaking down due to rate limits after transitioning to one row per detail page.

The Sheets API allows a maximum of 300 requests per minute, and since I have no way to tune elm-pages build, I have no way of slowing this build down to deal with rate limits. I could split the sheet between two accounts or do some kind of exponential back-off retry... but I'm not sure if elm-pages build will pick back up where it left off or retry the already successful 300 pages.

Any suggestions here? I'm open to new approaches in design as well.

[dillonkearns]

Hello! 👋

It's a great topic, and one that has been on my mind as well.

First off, if you do elm-pages build --keep-cache, it will retain your HTTP cache. You can also run elm-pages dev --keep-cache to run your dev server without clearing out the HTTP cache. So this may be enough to give you what you need for now.

However, the reason that isn't the default is because when you're running a build you probably want to avoid stale data. And if you're restarting the dev server, you may want to make sure there is fresh HTTP data.

There is a DataSource.Port module that lets you consume JSON from custom JavaScript functions, so that's a good place to experiment with ideas, including using an HTTP cache. That said, I'm also trying to work out the right caching behavior here. Right now DataSource.Port responses will also use an aggressive cache for responses that have the exact same request body, and like the HTTP cache it is cleared out if when you restart the dev server or run a build without the --keep-cache flag. This is for two reasons: 1) performance (in case a DataSource.Port is computationally intensive), and 2) consistent declarative data. The DataSource abstraction in elm-pages is designed to be a declarative way to get data, so it shouldn't be giving back different responses and shouldn't need to be called multiple times, at least in the context of building a single page. The semantics here are especially subtle because this can actually change in the context of server-rendered pages in elm-pages 3.0 because you may want to do effectful things with the outside world in that context so you might want to ensure that each call results in being executed again (i.e. it should never be cached).

So that's a whole lot of different semantics to keep in mind. I'm still exploring the best abstractions and design choices to equip users with the right tools to navigate these things intuitively and effectively.

Conditional HTTP Requests

Web browsers use the concept of Conditional HTTP requests to cache requests based on certain HTTP headers. For example, if the browser receives an etag header with a request, it can pass that back in an HTTP request header if it performs that request again and use an HTTP 304 response with an empty body to tell it to use its cached HTTP response.

Some APIs, like GitHub's for example, will not count HTTP 304 responses against your rate limit, which is a win-win (better performance for both parties).

DataSource.Http is already resolved using NodeJS (not Elm's elm/http), so we have a lot of control over how to perform HTTP requests already. I've been considering using something like this make-fetch-happen NPM package (or similar) since it has built-in conditional HTTP request caching.

Challenges

I have a few design questions that I need clarity on before implementing conditional HTTP requests in DataSource.Http.

1. Should I assume that it's always okay to make conditional HTTP requests, or not? For example, if an API counts 304 responses against the rate limit count, maybe the user doesn't want to always perform these but instead wants to configure when to go hit the server again
2. Should there be a way to fall back to cached responses on error?
3. Where would configuration like this be specified, and what would the API look like? Would it be chained on like DataSource.Http.get "https://api.github.com/repos/dillonkearns/elm-pages" decoder |> DataSource.Http.withCacheOptions { ... }, or something else?
4. Should there be a global way to set default options for DataSource.Http caching?
5. What about when a server doesn't support Conditional HTTP Requests? Should there still be a way to define the caching strategy to use for that?
6. In a context where it makes sense to have different data for the same request (like in a server-rendered pages if you do an effectful HTTP request or DataSource.Port), how should that be specified? Where should that be allowed (since it can be a footgun and in many contexts is not what you want)?

Those are some of the questions I'm thinking through right now. Any thoughts, or related use cases, are welcome! I know it's a lot of questions, but hopefully that gets the ball rolling a bit to start turning these ideas into something actionable.