Everyone I know just uses the hosted ones, because of the sheer performance gap.
For now, you can do all the custom/manual training you want, but gpt4 will almost always outperform it with the right context.
Hopefully that will change in the future. Even then, I don't expect people to want to self-host as in on their own machines. More like custom training, then host either on SAAS or PAAS, or their own on-prem if they have it. Spending the performance of a personal laptop isn't worth the reduction of performance on other tasks. Again, maybe that will change.
It doesn't need to be an exclusive choice. Hosted and local models can complement each other.
On one of the projects I used Chat GPT-4 to write instructions/tutorials that were then executed by local model on a large chunk of data (cleaning up product catalogues).
GPT-4 then reviewed some results and fine-tuned the instructions.
One example - a local model (Mistral 7B OpenChat 3.5, which ranks high in my benchmarks) was generating additional search keywords for products at online marketplace. A second run of the model was cleaning up bad keywords.
The fun part here - ChatGPT-4 reviewed some user searches, product details and comments of marketing department. Then it generated condensed tutorials on writing good keywords for the products in this system (keywords have to cover unexpected search terms that people would use when searching for a product).
The tutorial was supposed to be for the "junior marketing assistant", but in reality it was fed to Mistral 7B.
The second pass was done similarly. "Hey, ChatGPT, these are some sample keywords that are produced by junior marketing assistant according to your tutorial. Review them and write a short guide on correcting most common problems".
It works nicely.
Other cases of local models that work good - custom embeddings (multi-lingual, mapped to the same vector space), custom TTS, custom STT. These are mostly used for specialised personal assistants.
Comments
Everyone I know just uses the hosted ones, because of the sheer performance gap.
For now, you can do all the custom/manual training you want, but gpt4 will almost always outperform it with the right context.
Hopefully that will change in the future. Even then, I don't expect people to want to self-host as in on their own machines. More like custom training, then host either on SAAS or PAAS, or their own on-prem if they have it. Spending the performance of a personal laptop isn't worth the reduction of performance on other tasks. Again, maybe that will change.
It doesn't need to be an exclusive choice. Hosted and local models can complement each other.
On one of the projects I used Chat GPT-4 to write instructions/tutorials that were then executed by local model on a large chunk of data (cleaning up product catalogues).
GPT-4 then reviewed some results and fine-tuned the instructions.
interesting, what does the local model do?
One example - a local model (Mistral 7B OpenChat 3.5, which ranks high in my benchmarks) was generating additional search keywords for products at online marketplace. A second run of the model was cleaning up bad keywords.
The fun part here - ChatGPT-4 reviewed some user searches, product details and comments of marketing department. Then it generated condensed tutorials on writing good keywords for the products in this system (keywords have to cover unexpected search terms that people would use when searching for a product).
The tutorial was supposed to be for the "junior marketing assistant", but in reality it was fed to Mistral 7B.
The second pass was done similarly. "Hey, ChatGPT, these are some sample keywords that are produced by junior marketing assistant according to your tutorial. Review them and write a short guide on correcting most common problems".
It works nicely.
Other cases of local models that work good - custom embeddings (multi-lingual, mapped to the same vector space), custom TTS, custom STT. These are mostly used for specialised personal assistants.