LLM generated tests are the ultimate response to “Our CTO wants high test coverage but we all know it’s bullshit and provides little to no value”. They exist to tick a checkbox.
Maybe there’s a little value in using these as regression tests. Beyonce rule and all that. Kinda like double entry accounting. “Oh the code makes the same mistake in 2 places, that must mean it’s on purpose”
Yes. The problem with automatically deriving tests from your implementation is that you’re no longer testing your intent, just your implementation. Your tests and your code will have the same bugs dutifully recorded in 2 places.
Your Beyonce rule no longer means “I meant to do that” it now means “I wrote what I wrote”
Comments
LLM generated tests are the ultimate response to “Our CTO wants high test coverage but we all know it’s bullshit and provides little to no value”. They exist to tick a checkbox.
Maybe there’s a little value in using these as regression tests. Beyonce rule and all that. Kinda like double entry accounting. “Oh the code makes the same mistake in 2 places, that must mean it’s on purpose”
If you fixed it then you should have put a test on it?
Yes. The problem with automatically deriving tests from your implementation is that you’re no longer testing your intent, just your implementation. Your tests and your code will have the same bugs dutifully recorded in 2 places.
Your Beyonce rule no longer means “I meant to do that” it now means “I wrote what I wrote”