Yes. There're a couple of problems I encountered during my trial and ended up choose to build my own:
1. the version I was using requires the configuration of a 'reranking model', which AFAIK has to be from cohere. Funny enough, I can later opt to not use this reranking model, but I cannot get any functionality working if I don't preconfigure the cohere API key for this model. The problem with this is that, I need to request my company's infosec to allow API call onto cohere, for a functionality that I didn't intend to use.
And I think this results in my observation of the startup llm app world vs the heavily-regulated finance industry world: that the startup llm world is taking a default opt-in, default saas mode; however the regulated clients cannot benefit from these practices due to arcane regulation requirements. In this case, dify requires the setup of an API key for a service that I don't intend to use, and I have to go through the hurdles just to get the hello world going.
My analysis is that, there are 2 ways to make the heavily-regulated clients happy: 1. major cloud provider integrating everything into the myriad of its service catalogs and offer a full suite of services with the sales and compliance completed as a whole. 2. startup go down a curated, opinionated, battery-included oss approach with the least of sass fuss.
2. for some reason when I test using the simplest QA template, it seems dify cannot retrieve the document of any kind that I uploaded. I don't know why it happened but debugging the simplest offering of an app that boasts ease of use is not my thing.
I didn't go further from there. I wish my experience is just an extremely rare case since dify definitely seems to be the most polished all-in-one provider out there. If my experience turned out to be more positive I'll definitely stick with it, now I just feel picking a all-in-one is not the right choice.
Comments
Did you try dify? I found it was a good beginning for me.
https://dify.ai/
Yes. There're a couple of problems I encountered during my trial and ended up choose to build my own: 1. the version I was using requires the configuration of a 'reranking model', which AFAIK has to be from cohere. Funny enough, I can later opt to not use this reranking model, but I cannot get any functionality working if I don't preconfigure the cohere API key for this model. The problem with this is that, I need to request my company's infosec to allow API call onto cohere, for a functionality that I didn't intend to use.
And I think this results in my observation of the startup llm app world vs the heavily-regulated finance industry world: that the startup llm world is taking a default opt-in, default saas mode; however the regulated clients cannot benefit from these practices due to arcane regulation requirements. In this case, dify requires the setup of an API key for a service that I don't intend to use, and I have to go through the hurdles just to get the hello world going.
My analysis is that, there are 2 ways to make the heavily-regulated clients happy: 1. major cloud provider integrating everything into the myriad of its service catalogs and offer a full suite of services with the sales and compliance completed as a whole. 2. startup go down a curated, opinionated, battery-included oss approach with the least of sass fuss.
2. for some reason when I test using the simplest QA template, it seems dify cannot retrieve the document of any kind that I uploaded. I don't know why it happened but debugging the simplest offering of an app that boasts ease of use is not my thing.
I didn't go further from there. I wish my experience is just an extremely rare case since dify definitely seems to be the most polished all-in-one provider out there. If my experience turned out to be more positive I'll definitely stick with it, now I just feel picking a all-in-one is not the right choice.
you own it dude? Damn crazy good UI