[Bug]: Possible regression with service worker / Windows cache fix causing large bandwidth use

[timothyhoward]

Describe the bug

A possible regression introduced between Evidence 39.1.1 and Evidence 39.1.4 that causes multiple reads on Parquet files, resulting in cases where 51.5MB of data is transferred for 53.1MB of resources. In 39.1.1, this same page transfers 6.5MB of data for 26.8MB of resources. 26.8MB x 2 = 53.6MB, making me think that the browser and the service worker are doing a double request for the same set of data. It's affecting Chrome and Edge on Windows and seems to be associated with the service worker and/or recent cache fix for the TProtocolException issue.

Steps to Reproduce

1. Open an Evidence instance in Microsoft Edge or Google Chrome on Windows.
2. Open Developer Tools and the 'Network' tab.
3. Record bandwidth usage and notice multiple requests made by client browser then the service worker.

Logs

No response

System Info

System:
    OS: Windows 10 10.0.20348
    CPU: (8) x64 Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz
    Memory: 4.34 GB / 32.00 GB
  Binaries:
    Node: 20.17.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.6.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (119.0.2151.97)
  npmPackages:
    @evidence-dev/core-components: ^4.7.5 => 4.7.5
    @evidence-dev/csv: ^1.0.12 => 1.0.12
    @evidence-dev/duckdb: ^1.0.11 => 1.0.11
    @evidence-dev/evidence: 39.1.4 => 39.1.4
    @evidence-dev/mssql: ^1.0.9 => 1.0.9
    @evidence-dev/sqlite: ^2.0.6 => 2.0.6
    @timhoward/evidence-connector-mssql: 1.0.1 => 1.0.1

Severity

serious, but I can work around it

Additional Information, or Workarounds

I've downgraded back to 39.1.1 and implemented the manual cache header fix for the TProtocolException in the IIS configuration.

[zachstence]

I think it just appears like more data is being transferred because we are intercepting and rewriting the request in the service worker. The external fetch only happens once, and then the data is passed between our service worker and the DuckDB service worker (which is the one that initiated the request) locally on your machine, so there shouldn't be any additional external network traffic.

Even though dev tools is reporting this as more data transferred, I don't think we are making a duplicate external request for the parquet file, its just how it appears due to our rewriting.

Did you see an increase in network traffic anywhere other than dev tools?

[timothyhoward]

I think it just appears like more data is being transferred because we are intercepting and rewriting the request in the service worker. The external fetch only happens once, and then the data is passed between our service worker and the DuckDB service worker (which is the one that initiated the request) locally on your machine, so there shouldn't be any additional external network traffic.

Even though dev tools is reporting this as more data transferred, I don't think we are making a duplicate external request for the parquet file, its just how it appears due to our rewriting.

Did you see an increase in network traffic anywhere other than dev tools?

Unfortunately that's not the behaviour I am seeing on my end.

I believe that the intended behaviour is attempted initially, but the page essentially hangs whilst the service worker downloads everything (12 seconds), and then reloads and seems to download it everything from the service worker. Clicking onto another page, this process repeats, essentially tanking the responsiveness for the user.

As the screenshots show, I'm getting 50MB+ of download with the new version, whereas on the version I've backtracked to I get 12MB. Going off the time of the request, I don't think the second download is coming from the client.

As it stands, I will need to remain on the 39.1.1 before the changes were made. The new version is essentially unusable for us as it stands.

Unfortunately I can't do a lot of deeper networking analysis as we're operating this instance in a secure environment, although I'll see what I can do.