I see the appeal and I'd totally consider using SQLite as a vector store with the proper extensions/support (I'd imagine this exists; does it?), but the code shown there really isn't an apples to apples comparison, is it? Every query fetches all vectors, deserializes each from JSON, allocates memory, and instantiates them as numpy arrays, and then proceeds to do an O(n) search for cosine similarity on embeddings which in the example aren't normalized. At this point, network latency for a (presumably loopback) grpc call isn't what I'd be concerned with. There's really no reason to use SQLite at all in this case, just keep everything in memory and save state to disk if that's what you care about.
This was a 10 minute proof of concept! There are so many optimizations I'll do on the next iteration, but the idea is that people are reaching for databases that are overkill and add network latency that can be avoided
Comments
I see the appeal and I'd totally consider using SQLite as a vector store with the proper extensions/support (I'd imagine this exists; does it?), but the code shown there really isn't an apples to apples comparison, is it? Every query fetches all vectors, deserializes each from JSON, allocates memory, and instantiates them as numpy arrays, and then proceeds to do an O(n) search for cosine similarity on embeddings which in the example aren't normalized. At this point, network latency for a (presumably loopback) grpc call isn't what I'd be concerned with. There's really no reason to use SQLite at all in this case, just keep everything in memory and save state to disk if that's what you care about.
This was a 10 minute proof of concept! There are so many optimizations I'll do on the next iteration, but the idea is that people are reaching for databases that are overkill and add network latency that can be avoided