Just saw this in the release notes, super excited to try it out.
Compose supporting an AI agent stack out of the box could be a big deal. I just saw a Reddit thread around using compose in production and I'm thinking I can put these two together. The idea of spinning up a whole multi-container setup for an LLM app (like a vector DB, orchestrator, backend, etc.) with a single compose up could work to actually get stuff to prod. I’ve been hacking around and struggling with this. Curious to see how far I can push this, might finally be the clean setup I can share.
Excited to see models get first-class support! Will be interesting to see what opportunities this opens with this being formalized (other tooling/integrations, etc.).
Note that Compose files using the previous model declarations (using providers) will still work.
Comments
Just saw this in the release notes, super excited to try it out.
Compose supporting an AI agent stack out of the box could be a big deal. I just saw a Reddit thread around using compose in production and I'm thinking I can put these two together. The idea of spinning up a whole multi-container setup for an LLM app (like a vector DB, orchestrator, backend, etc.) with a single compose up could work to actually get stuff to prod. I’ve been hacking around and struggling with this. Curious to see how far I can push this, might finally be the clean setup I can share.
Excited to see models get first-class support! Will be interesting to see what opportunities this opens with this being formalized (other tooling/integrations, etc.).
Note that Compose files using the previous model declarations (using providers) will still work.
The big question is still - how will the models reach production?
You can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/
Well, more wondering how I go from having local models into a production app with the model available - I guess swap it with a cloud offering?
You can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)
Following this.
That sounds interesting
super pumped to see models get this support through docker
This is brilliant.