Hi folks - Yangqing from Lepton here. The idea came from a coffee chat with a colleague on the question: how much of the RAG quality comes from the old good search engine, vs LLMs? And we figured out that the best way is to build a quick experiment and try it out. What we learned is that search engine results matter a lot, and probably more important than LLMs. We decided to put it up as a site and also open source the full code.
You can try plug in different search engines or even your own elastic interface, write different LLM prompts, pick different LLM models - a lot of ablation studies that could be tried out.
Comments
Hi folks - Yangqing from Lepton here. The idea came from a coffee chat with a colleague on the question: how much of the RAG quality comes from the old good search engine, vs LLMs? And we figured out that the best way is to build a quick experiment and try it out. What we learned is that search engine results matter a lot, and probably more important than LLMs. We decided to put it up as a site and also open source the full code.
You can try plug in different search engines or even your own elastic interface, write different LLM prompts, pick different LLM models - a lot of ablation studies that could be tried out.
We appreciate your interest and happy Friday!