Just convert it to a binary data structure suitable for this purpose and use the old-fashioned bsearch. It's so simple that you wouldn't even need SQLite.
It seems that you could just compile the data into the WASM blob. Then use a more optimized query engine than sqlite. This should be very fast to compile (most of the WASM is just a byte buffer, the code is just a few binary searches and some result encoding). The downside is that you need to recompile to update the repo, but I don't think that should be particularly expensive.
That's an interesting idea as well.
Daniel wrote https://github.com/danielfullmer/nkv which also was a pretty fun experiment on how to improve data access within Nix. I love these cursed ideas.
Comments
Just convert it to a binary data structure suitable for this purpose and use the old-fashioned bsearch. It's so simple that you wouldn't even need SQLite.
It seems that you could just compile the data into the WASM blob. Then use a more optimized query engine than sqlite. This should be very fast to compile (most of the WASM is just a byte buffer, the code is just a few binary searches and some result encoding). The downside is that you need to recompile to update the repo, but I don't think that should be particularly expensive.
That's an interesting idea as well. Daniel wrote https://github.com/danielfullmer/nkv which also was a pretty fun experiment on how to improve data access within Nix. I love these cursed ideas.
(author) If you have questions, happy to try and answer some.
If this much JSON could fit in a microcontroller’s memory (7.5MB in text), and it’s a performance issue, maybe it’s worth upgrading the JSON parser?
The WASM extension me like. Would be great to get that into mainline.