Does it matter whether you ask the agent to write the tests first or last? Is one way TDD and the other not?
Yes. TDD = Test _Driven_ Development, meaning you write the test first, then the implementation.
If you write the test after, that's just called "development" afaik :)
I've only done TDD in very limited cases, like when you're implementing against a known spec. For "regular" work I'm usually still figuring out the shape and code interface for what I'm building, so writing tests first is impractical and wasteful.
Readme-driven-development is more fun: write the readme (ie: the intention) first.
This says - if coding with an LLM - to write the tests before you commit.
Is there a name for AI looping until there's test coverage? Did this study require >98% test coverage? Line or branch coverage?
What will coverage-guided fuzzing fuzz if there's 100% test coverage?
Perhaps dev codebase comprehension is a significant unmeasured external benefit of writing tests first TDD, but LLMs don't train online so all you win is the context cost?
Comments
Yes. TDD = Test _Driven_ Development, meaning you write the test first, then the implementation.
If you write the test after, that's just called "development" afaik :)
I've only done TDD in very limited cases, like when you're implementing against a known spec. For "regular" work I'm usually still figuring out the shape and code interface for what I'm building, so writing tests first is impractical and wasteful.
Readme-driven-development is more fun: write the readme (ie: the intention) first.
But then what do you call coding without tests?
This says - if coding with an LLM - to write the tests before you commit.
Is there a name for AI looping until there's test coverage? Did this study require >98% test coverage? Line or branch coverage?
What will coverage-guided fuzzing fuzz if there's 100% test coverage?
Perhaps dev codebase comprehension is a significant unmeasured external benefit of writing tests first TDD, but LLMs don't train online so all you win is the context cost?