This isn't really true. At smartphone OEMs we certainly do have boxes to put the devices in that perform physical tests like on the touch screen, microphones, speakers, antennas, etc.. And in mobile development we have UI automation tests that confirm buttons are certain color, have certain text or state, the right screens popup when pressed, etc. - heck we have a program called the monkey that presses everything that can be in addition to the UI automation scripts. I know the web side of things has Selenium and similar robots. I think he just hasn't ever worked somewhere where everything is tested which is reasonable. In many cases you have nothing to do with the OS your software is running on, for example, so there isn't as much point in testing beyond what your app outputs to it.
I think what Uncle Bob is referring to are things like layout, color, and so on. Of course you can automate browser interactions with Selenium, but you can't easily catch layout changes, broken UI elements, or regressions. The only method I know of that can come close is automated screen capture comparison. But that wouldn't work perfectly and still requires human intervention to check out false positives.
"The only method I know of that can come close is automated screen capture comparison. But that wouldn't work perfectly and still requires human intervention to check out false positives."
It would require human intervention in the case of a failure, to be sure, but those cases where it can guarantee I don't need to bother looking because my change didn't change anything visible in those screenshots is potentially a significant boon.
I'd argue that the test has value if it's central to what you're doing and will save you time in the long run. Not every test falls into that category, so it's a cost-benefit analysis.
He talks about fiddling with UI elements which is sometimes a one-time thing after you get it setup. Writing tests for that is sometimes a waste of time.
Now, if you have code that's going to do some form of complex screen manipulation and it's a big piece of what you're doing, it makes more sense to automate some tests.
Comments
This isn't really true. At smartphone OEMs we certainly do have boxes to put the devices in that perform physical tests like on the touch screen, microphones, speakers, antennas, etc.. And in mobile development we have UI automation tests that confirm buttons are certain color, have certain text or state, the right screens popup when pressed, etc. - heck we have a program called the monkey that presses everything that can be in addition to the UI automation scripts. I know the web side of things has Selenium and similar robots. I think he just hasn't ever worked somewhere where everything is tested which is reasonable. In many cases you have nothing to do with the OS your software is running on, for example, so there isn't as much point in testing beyond what your app outputs to it.
I think what Uncle Bob is referring to are things like layout, color, and so on. Of course you can automate browser interactions with Selenium, but you can't easily catch layout changes, broken UI elements, or regressions. The only method I know of that can come close is automated screen capture comparison. But that wouldn't work perfectly and still requires human intervention to check out false positives.
"The only method I know of that can come close is automated screen capture comparison. But that wouldn't work perfectly and still requires human intervention to check out false positives."
It would require human intervention in the case of a failure, to be sure, but those cases where it can guarantee I don't need to bother looking because my change didn't change anything visible in those screenshots is potentially a significant boon.
I'd argue that the test has value if it's central to what you're doing and will save you time in the long run. Not every test falls into that category, so it's a cost-benefit analysis.
He talks about fiddling with UI elements which is sometimes a one-time thing after you get it setup. Writing tests for that is sometimes a waste of time.
Now, if you have code that's going to do some form of complex screen manipulation and it's a big piece of what you're doing, it makes more sense to automate some tests.
And do you write all those touch screen, antenna, microphone and speaker tests before you start writing any of the code?
True, we've spent a lot of time building machines with actuators, sensors, etc. to physically test UIs. Also, screen scraping to verify GUIs.
I get Uncle Bob's point, though, and I welcome what looks like a very reasonable peace offering to the zealots on the other side.
These are integration tests really, not unit tests
Unless you do a small change then make the robot do everything.
(Also, this may also be used for device tests in the production line)