For example, for us, we are building an LLM chatbot that pulls in the data of a technical book publisher. They have 20 years of technical books, and 20 years of videotaped conference talks.
Hard:
- We're using LangChain, which isn't always great
- The data pipeline was trickier than I had initially thought
- Indexing embeddings (in PostGres) is just hard (requires tons of ram)
But the hardest thing has been working on conversation quality. We've started to use LangSmith, which was a godsend for tracing and observability, and came out fairly recently. But it's not perfect and I wish there were better tools out there.
I have been using it since the week it was in private beta, albeit a lot less recently, and thought it was good, though with some confusing UX and a handful of bugs.
Comments
For example, for us, we are building an LLM chatbot that pulls in the data of a technical book publisher. They have 20 years of technical books, and 20 years of videotaped conference talks.
Hard:
- We're using LangChain, which isn't always great
- The data pipeline was trickier than I had initially thought
- Indexing embeddings (in PostGres) is just hard (requires tons of ram)
But the hardest thing has been working on conversation quality. We've started to use LangSmith, which was a godsend for tracing and observability, and came out fairly recently. But it's not perfect and I wish there were better tools out there.
What do you find lacking in LangSmith?
I have been using it since the week it was in private beta, albeit a lot less recently, and thought it was good, though with some confusing UX and a handful of bugs.
Just generally a lot of abstractions that seem sometimes overwrought, and seem to hide details.
I would say that's langchain's ethos as a whole! haha