Comment on SQLite performance tuning: concurrent reads, multiple GBs and 100k SELECTs/sComments−avinassh3yI am experimenting with SQLite, where I try inserting 1B rows in under a minute. The current best is inserting 100M rows at 23s. I cut many corners to get performance, but the tweaks might suit your workload.I have explained my rationale and approach here - https://avi.im/blag/2021/fast-sqlite-inserts/the repo link - https://github.com/avinassh/fast-sqlite3-inserts−iveqy3yYou probably already seen it but here's a very good write up on how to improve insert performance: https://stackoverflow.com/questions/1711631/improve-insert-p...
Comments
I am experimenting with SQLite, where I try inserting 1B rows in under a minute. The current best is inserting 100M rows at 23s. I cut many corners to get performance, but the tweaks might suit your workload.
I have explained my rationale and approach here - https://avi.im/blag/2021/fast-sqlite-inserts/
the repo link - https://github.com/avinassh/fast-sqlite3-inserts
You probably already seen it but here's a very good write up on how to improve insert performance: https://stackoverflow.com/questions/1711631/improve-insert-p...