set sqlite PRAGMAs to improve performance

[matt-codecov]

quick changes to SQLite pragmas to improve bulk insert performance:

• journal_mode=OFF: no transaction journal, we can't roll back (in SQLite). we can still "rollback" by just not persisting/exporting the SQLite database after failed processing
• cache_size=-200000: set the SQLite page cache size to ~200MB
• synchronous=OFF: trust the filesystem to perform reads reliably

journal_mode and cache_size together improve bulk insert speed for my 110mb test report by about a second. cache_size increases the RAM usage, but it's capped. i chose 200MB arbitrarily but we can turn this knob later when running on real traffic

with a 200MB page cache, i found RAM usage topped off at ~150MB for my 110mb test report. profile data shows SQLite doesn't do any disk writes until the end when the transaction gets committed, and not a lot of time is spent waiting on disk IO overall. most of the time remaining is spent on index updates and such

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.21%. Comparing base (c0a4b65) to head (e1ceeb4).

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #13   +/-   ##
=======================================
  Coverage   99.21%   99.21%           
=======================================
  Files          17       17           
  Lines        7249     7252    +3     
=======================================
+ Hits         7192     7195    +3     
  Misses         57       57           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Swatinem]

we can't roll back (in SQLite). we can still "rollback" by just not persisting/exporting the SQLite database after failed processing

In that case, does it make sense to use transactions at all?

[matt-codecov]

In that case, does it make sense to use transactions at all?

y'know, great question. before merging this, i'll measure that. i think transactions still affect locking behavior and maybe more, but maybe they won't matter