Vector DBs don't create embeddings; they store them. As the article points out, the LLM's slowness to respond diminishes the performance that vector DB's can potentially add.
You can easily create embeddings locally though, with small (L)LMs. Three lines of code using hugging face. I don't understand the point of this article.
Look it this way: you have a web app that can handle 10 req/s. It does not matter if you add a database behind it that can handle 10,000 req/s. 10 req/s still limit you. You're not gaining any performance benefits. I always wondered why you would need a dedicated vector DB.
Comments
Vector DBs don't create embeddings; they store them. As the article points out, the LLM's slowness to respond diminishes the performance that vector DB's can potentially add.
You can easily create embeddings locally though, with small (L)LMs. Three lines of code using hugging face. I don't understand the point of this article.
Look it this way: you have a web app that can handle 10 req/s. It does not matter if you add a database behind it that can handle 10,000 req/s. 10 req/s still limit you. You're not gaining any performance benefits. I always wondered why you would need a dedicated vector DB.
vector DBs have much higher 'semantic' recall than classical search engines if you want to ask questions about your documents or previous discussions.
Actually, some Vector DB's can generate embeddings as well as storing them, Chroma in particular uses SentenceTransformers models.