Ask HN: Those running agents 24/7, what's your workflow and what are they doing?
In either a local AI or a hosted AI arrangements, I'm seeing lots of posts of people saying they have a need for agents 24/7, assuming they're in the software space. What are these agents actually doing? How close to the frontier are these models, and what does it cost?
I run a SaaS business, but I feel like agents aren't really working for me as once the last large task finishes that I assign for the day. I have a huge backlog that I would like to automate but I'm struggling to comprehend how people get quality outputs from tasks and where the user is supposed to be in the pipeline and how things go from idea to task to automated build to what I assume is the final human intervention which is the review process.
Top line, I'm assuming they're using some sort of issue tracking system like Linear or GitHub issues into either a cloud agent or a work tree where it's built out and tested? What about for things that aren't bug or issue based, but are actually features? How much context do you provide? How do you stop the AI just running with things instead of asking for clarity when required, etc.
Is there actually a net positive benefit to this setup, or do you end up just having to patch all of the work that was done autonomously overnight most of the time?
Comments
It's useful for search type problems where the result is relatively compact and verifiable in isolation. I haven't found a way to build production quality software that way in general but for certain kinds of reverse engineering or system optimization problems it's helpful.
Are you talking problems along the lines of Karpathy's autoresearch? Reverse engineering does seem pretty straightforward actually in that use case, but for practical business building work as opposed to hobby projects that's where I'm hitting the wall at the moment.
I haven't looked closely enough at his autoresearch stuff to know how similar it is. From what I have seen the quality of output is directly tied to the quality of the request, a human can only read and understand some finite amount of material in a day, so maintaining enough understanding to ask quality questions becomes the limiting factor. What are the priorities you want to move faster on?
For 24/7 agent ops with review in the loop I recommend this: https://aicommander.dev