Comment on Here comes the Muybridge camera moment but for textparentComments−1024core2yAny tools to replicate @oca.computer's work?Once we have the 1000-dim vector embeddings I can make the rest work. Not sure how to go from 20-word span to a 1000-dim vector embedding.−10c82yGenerating embeddings is relatively simple with a model and Python code. There's plenty of them on HuggingFace, along with code examples.all-MiniLM-L6-v2 is a really (if not the most) popular one (albeit not SotA), with 384 dimensions: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v...Edit: A more modern and robust suite of models comes from Nomic, and can generate embeddings with 64 to 768 dimensions (https://huggingface.co/nomic-ai/nomic-embed-text-v1.5).When the author talks about thousands of dimensions, they're probably talking about the OpenAI embedding models.
Comments
Any tools to replicate @oca.computer's work?
Once we have the 1000-dim vector embeddings I can make the rest work. Not sure how to go from 20-word span to a 1000-dim vector embedding.
Generating embeddings is relatively simple with a model and Python code. There's plenty of them on HuggingFace, along with code examples.
all-MiniLM-L6-v2 is a really (if not the most) popular one (albeit not SotA), with 384 dimensions: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v...
Edit: A more modern and robust suite of models comes from Nomic, and can generate embeddings with 64 to 768 dimensions (https://huggingface.co/nomic-ai/nomic-embed-text-v1.5).
When the author talks about thousands of dimensions, they're probably talking about the OpenAI embedding models.