Proper refactoring with an llm requires full testing coverage
Proper refactoring WITHOUT an LLM requires full test coverage! But most definitely WITH an LLM.
In cases where there is no test coverage, the first thing I do is have the LLM write one test at a time. The problem there is that if you truly wanted valid tests, you'd have to actually break the code, watch the test fail to prove it was valid (basically, the inverse of TDD) and then re-fix the code and start on the next test, but in practice, it is difficult to get an LLM to stick to this loop. I wish someone would train or refine some coding LLM to use either TDD or this form of "inverse TDD" where you're applying tests after the fact and also want to check their validity. (Or tell me how to do it.) Because mere prompting doesn't seem to stick- it always regresses to the mean eventually.
(I'm currently seeking work, btw, and would probably be happy to help refactor old code, advise people on code, etc. Sorry for the self-promo.)
Comments
Proper refactoring WITHOUT an LLM requires full test coverage! But most definitely WITH an LLM.
In cases where there is no test coverage, the first thing I do is have the LLM write one test at a time. The problem there is that if you truly wanted valid tests, you'd have to actually break the code, watch the test fail to prove it was valid (basically, the inverse of TDD) and then re-fix the code and start on the next test, but in practice, it is difficult to get an LLM to stick to this loop. I wish someone would train or refine some coding LLM to use either TDD or this form of "inverse TDD" where you're applying tests after the fact and also want to check their validity. (Or tell me how to do it.) Because mere prompting doesn't seem to stick- it always regresses to the mean eventually.
(I'm currently seeking work, btw, and would probably be happy to help refactor old code, advise people on code, etc. Sorry for the self-promo.)