The visual plotting component I was working on would suddenly stop producing output: just a black screen but everything else was normal - no leaks, race conditions, crashes, etc - just no video. The bug was so rare that only customers ever saw it, but they were pissed (serious manufacturing folks, they ran our software 24/7 on production floors)
Eventually I wrote a debugging code around it to take screen shots of the main window and compare it with a reference image about 20 times a second (after each blit) and left it running for months on an old PC.
One night I got an SMS sent by my debugging script with attached call stack, etc, but I knew what had happened without even reading it: I got it 1 second after daytime savings time kicked in - the plotter didn't handle it properly and was trying to plot data from the future, which, of course hadn't happened yet.
Comments
The visual plotting component I was working on would suddenly stop producing output: just a black screen but everything else was normal - no leaks, race conditions, crashes, etc - just no video. The bug was so rare that only customers ever saw it, but they were pissed (serious manufacturing folks, they ran our software 24/7 on production floors)
Eventually I wrote a debugging code around it to take screen shots of the main window and compare it with a reference image about 20 times a second (after each blit) and left it running for months on an old PC.
One night I got an SMS sent by my debugging script with attached call stack, etc, but I knew what had happened without even reading it: I got it 1 second after daytime savings time kicked in - the plotter didn't handle it properly and was trying to plot data from the future, which, of course hadn't happened yet.