"But if I want to be sure that the bell rings when the proper signals are sent to the driver, I either have to set up that microphone or just listen to the bell.
How can I test that the right stuff is drawn on the screen? Either I set up a camera and write code that can interpret what the camera sees, or I look at the screen while running manual tests."
Or you programmatically capture the screen output and compare it to a known-good capture. Not the best solution, but it's a solution that I've seen used. 99% of the time, the screen output is the same, and occasionally it'll be different and require human intervention to determine whether the change is right or wrong.
Getting notified when things aren't what you'd expect them to be is pretty valuable.
We used this in a videogame engine test framework to verify specific game states against gold-master images. It was particularly useful for verifying when our physics engine had changed in subtle ways; one of our tests involved dropping a couple of boxes on top of each other and verifying where they landed.
All kinds of stuff would disrupt that test---which made it great for knowing when we'd changed something subtle that would have real impact on our game engine's users.
Comments
"But if I want to be sure that the bell rings when the proper signals are sent to the driver, I either have to set up that microphone or just listen to the bell.
How can I test that the right stuff is drawn on the screen? Either I set up a camera and write code that can interpret what the camera sees, or I look at the screen while running manual tests."
I have nothing else to add
Or you programmatically capture the screen output and compare it to a known-good capture. Not the best solution, but it's a solution that I've seen used. 99% of the time, the screen output is the same, and occasionally it'll be different and require human intervention to determine whether the change is right or wrong.
Getting notified when things aren't what you'd expect them to be is pretty valuable.
This can work well for form-like things, but how would you do this for a 3d game, e.g.
Hmm. Computer vision project #346.
http://www.sikuli.org/
We used this in a videogame engine test framework to verify specific game states against gold-master images. It was particularly useful for verifying when our physics engine had changed in subtle ways; one of our tests involved dropping a couple of boxes on top of each other and verifying where they landed.
All kinds of stuff would disrupt that test---which made it great for knowing when we'd changed something subtle that would have real impact on our game engine's users.
Yeah, this is complicated, it may have changed but it may still be correct
like a value of 15.000002 instead of 15.000013 but errors may accumulate in the end.
Yeah, I love Sikuli. I've only used it to automate clicking through game GUI, not for anything in the game scene itself.