Skip to content

Comment on SQLite performance tuning: concurrent reads, multiple GBs and 100k SELECTs/sparent

Comments

You’re eliminating the “D” from “ACID” with these settings, and risk data loss on power failure and/or entire database corruption.

I can make a car go really fast if I eliminate the weight of having safety equipment on it like a bumper, seatbelt and airbag.

Which, for this use case - bulk generating a read-only data file from a source dataset in a batch job - seems like a pretty good performance tradeoff, no? In the case of a failure of some kind - no big deal, just restart the process anew.

This is one of those use cases where SQLite isn’t replacing <database> - it’s replacing fopen.

Exactly, this use case is write-once then read-only after that. Random updates to the sqlite file after that would need more “normal” settings, but they wouldn’t also need as much throughput.

It would be nice to have a compendium of SQLite use-cases and their respective configurations.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.