I'm a hands on engineering leader for a team of about 20 engineers and I've been spending the past 6 months trying to get my team to understand just this.
On Friday we had a coffee hour to share how we've been working recently and they all seemed perplexed at the workflows I've been adopting. It seems natural to me as someone who's been a manager for some time now, but very alien to all those who've never gone down that path.
That's not to say my workflows are superior, but they're extremely different to some of my teams now. In reality it's just leaning heavily on things like prds, limiting communication between agents, etc.
Out of curiosity, what was the path that the engineers have been taking? The limiting communication between agents to my mind is obvious to anyone who hasn't swallowed orthodoxy Agile development completely whole. And in my experience the people who have, are rarely the devs. Context switching, between agents, people or whoever, require ramp up time to relearn context. It's always slower and more expensive, other potential upsides about long term training or developers being more replaceable notwithstanding. But no agent gets long term training, not in a 1M context window.
And our in house "Agile Practitioners" (their job title) pushed this to a state where the next four tickets in a task get broken down to four hour long pieces of work go to four different people, regardless of their context or skill set rather than what two years ago was what one person just owned and did in a day. Work in progress is minimized by the metrics, but it doubled the overall man hours. And the Ops Director gets promoted on that WIP metric while half the team quit citing the changes due to the added communication burden and lack of ownership.
I'm thinking some people are approaching agentic development in the same manner, splitting work up across agents too aggressively, giving each agent new context/upfront plans, when the one session could have done the whole problem in the one context window.
Right, the ultimate context switch is constant pinging and meetings. There’s very little reason to have engineers be in 5+ meetings a day. But at a lot of orgs, they are, and that’s an extreme context switch cost 5x a day. It’s different if it’s a purely technical meeting about the problem you’re solving right now, but they’re typically not.
In the few instances that I’ve been able to see a total stranger prompt a model, I learn a lot about them.
The model output is fairly uniform because the models are fairly uniform, but the input is how I understand the prompter’s theory of mind for the LLM.
The higher fidelity the theory of mind, the more productive the resulting conversation is. Basic things, like knowing what the model is even aware of.
It’s okay at subjective product judgements with limited context, and so the best engineers make the most important decisions themselves, constraining the model’s solution space to something looking like success.
Maintaining consistent progress towards a common goal with a bunch of different perspectives is the job.
Comments
I'm a hands on engineering leader for a team of about 20 engineers and I've been spending the past 6 months trying to get my team to understand just this.
On Friday we had a coffee hour to share how we've been working recently and they all seemed perplexed at the workflows I've been adopting. It seems natural to me as someone who's been a manager for some time now, but very alien to all those who've never gone down that path.
That's not to say my workflows are superior, but they're extremely different to some of my teams now. In reality it's just leaning heavily on things like prds, limiting communication between agents, etc.
Out of curiosity, what was the path that the engineers have been taking? The limiting communication between agents to my mind is obvious to anyone who hasn't swallowed orthodoxy Agile development completely whole. And in my experience the people who have, are rarely the devs. Context switching, between agents, people or whoever, require ramp up time to relearn context. It's always slower and more expensive, other potential upsides about long term training or developers being more replaceable notwithstanding. But no agent gets long term training, not in a 1M context window.
What? One of the first principles in doing Lean and Agile correctly is limiting Work in Progress and avoiding context switching whenever possible.
And our in house "Agile Practitioners" (their job title) pushed this to a state where the next four tickets in a task get broken down to four hour long pieces of work go to four different people, regardless of their context or skill set rather than what two years ago was what one person just owned and did in a day. Work in progress is minimized by the metrics, but it doubled the overall man hours. And the Ops Director gets promoted on that WIP metric while half the team quit citing the changes due to the added communication burden and lack of ownership.
I'm thinking some people are approaching agentic development in the same manner, splitting work up across agents too aggressively, giving each agent new context/upfront plans, when the one session could have done the whole problem in the one context window.
Most organizations are not doing Agile correctly.
Everywhere I've worked claims to be doing Agile, but none of them have ever done anything to avoid context switching or limiting work in progress
if its like anything i've experienced in multiple places, they are just implementing scrum and calling it a day.
meanwhile the business runs on quarterly projects that have to be done by a certain marketing date, so its usually scrum + waterfall over and over
Right, the ultimate context switch is constant pinging and meetings. There’s very little reason to have engineers be in 5+ meetings a day. But at a lot of orgs, they are, and that’s an extreme context switch cost 5x a day. It’s different if it’s a purely technical meeting about the problem you’re solving right now, but they’re typically not.
In the few instances that I’ve been able to see a total stranger prompt a model, I learn a lot about them.
The model output is fairly uniform because the models are fairly uniform, but the input is how I understand the prompter’s theory of mind for the LLM.
The higher fidelity the theory of mind, the more productive the resulting conversation is. Basic things, like knowing what the model is even aware of.
It’s okay at subjective product judgements with limited context, and so the best engineers make the most important decisions themselves, constraining the model’s solution space to something looking like success.
Maintaining consistent progress towards a common goal with a bunch of different perspectives is the job.
Can you share some details of the workflows that work for you?
I would also love to hear more details.
Is anyone on your team not using multi agent?