Show HN: Litellm – Simple library to standardize OpenAI, Cohere, Azure LLM I/O
github.com/BerriAII built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. litellm has two functions - completion(), embedding()
Comments
Great start! Are you planning to add to following:
Retries w exponential backoff, Caching, Streaming output, Function-calling support
You can use tenacity for retries and wouldn't you want to cache the request / response around the endpoint instead of the gpt call -> that's what we ended up doing.
Streaming output and function-calling support is interesting
Could you elaborate on this — “wouldn’t want to cache around the endpoint instead of GPT call”. Just want to see if I’m missing an important consideration here
You're probably calling openai from a backend server. If you're making the call because a user asked a question -> your endpoint received it -> did some processing -> called openai -> returned the response,
why would you cache the openai call instead of the endpoint that's receiving the user call?
good points, probably going to add streaming output, function calling support. As for retries tenacity does a great job already
Take a look at llm-client a similar library that also support chat, async and more llm providers https://github.com/uripeled2/llm-client-sdk
thanks for sharing, while your library looks really powerful my goal with Litellm is simplicity
This is amazing. Really needed something like this to standardize all my different AI APIs!
On a side note - I love how quickly your team is shipping! Do keep it going!
Thanks!
Thanks!
This is much needed. For someone looking to quickly implement, having a simple interface goes a long way.
Thanks
Very cool Ishaan!
Thank you !
Why like this?
azure uses the openai python sdk, just remaps certain components. The models are also user-named. This makes it hard to detect if a model passed in is an azure model
azure models have custom names - eg I call mine 'chat-gpt-test1', I require some flag to know if it's an azure model