Which then opens you up to exactly that class of bugs that also caused this issue. Having test specific code and feature flags and then testing a tweaked version isn't really covering all the cases then.
Just like in this case where a hardcoded password was set to maybe log in through a test based on the naming "test_default".
Comments
But only if there's no rate limiting or increasing timeouts for wrong passwords which in most cases exists.
Right, but you can bypass that in a testing environment.
Which then opens you up to exactly that class of bugs that also caused this issue. Having test specific code and feature flags and then testing a tweaked version isn't really covering all the cases then.
Just like in this case where a hardcoded password was set to maybe log in through a test based on the naming "test_default".