" has anyone seen a failed CPU piece which has failed during normal operation?"
Several. But I'm lucky in that I worked for NetApp for 5 years which have several million NetApp filers in the field that were all calling home when they had issues, and Google which has a very large number of CPUs all around the planet doing their bidding. With visibility into a population like that you see faults that are once in a billion happen about once a month :-).
Two general kinds of failures though, the more common one is a system machine check (the internal logic detected the fault condition and put the CPU into the machine check state) which happens when 3 or more bits go sideways in the various RAM inside the mesh of execution units. Nominally ECC protected it can detect but not correct multi-bit errors. Power it off, power it on, and restart from a known condition and it's good as new.
The more rare occurrence is that something in the CPU fails which results in the CPU not coming out of RESET even, or immediately going into a machine check state. When you find those Intel often wants you to send it back to them so they can do failure analysis on it. The most common root cause analysis for those is some moderate electrostatic damage which took a while to finally finish the process of failing.
Some of the more interesting papers at the ISSCC are sometimes on lifetime expectancy of small geometry transistors. They are a lot more susceptible to damage and disruption due to cosmic rays and other environmental agents.
In the best case your app crashes or misbehaves randomly not unlike bad RAM (yes it can happen even with ECC). In the worst case there is a subtle numeric error that could potentially percolate to users. Usually it's about floating point issues, since usually you don't use floats to access arrays or do pointer arithm, issues in that unit may not cause an early failure and hence go undetected.
Simple case is that your machine just halts. If you have a way of looking at them, it might puts a bios code on the LEDs. On servers the baseboard management chip (BMC) will usually record a machine check event as well.
Comments
" has anyone seen a failed CPU piece which has failed during normal operation?"
Several. But I'm lucky in that I worked for NetApp for 5 years which have several million NetApp filers in the field that were all calling home when they had issues, and Google which has a very large number of CPUs all around the planet doing their bidding. With visibility into a population like that you see faults that are once in a billion happen about once a month :-).
Two general kinds of failures though, the more common one is a system machine check (the internal logic detected the fault condition and put the CPU into the machine check state) which happens when 3 or more bits go sideways in the various RAM inside the mesh of execution units. Nominally ECC protected it can detect but not correct multi-bit errors. Power it off, power it on, and restart from a known condition and it's good as new.
The more rare occurrence is that something in the CPU fails which results in the CPU not coming out of RESET even, or immediately going into a machine check state. When you find those Intel often wants you to send it back to them so they can do failure analysis on it. The most common root cause analysis for those is some moderate electrostatic damage which took a while to finally finish the process of failing.
Some of the more interesting papers at the ISSCC are sometimes on lifetime expectancy of small geometry transistors. They are a lot more susceptible to damage and disruption due to cosmic rays and other environmental agents.
Interesting. How would these failures manifest themselves for a user?
In the best case your app crashes or misbehaves randomly not unlike bad RAM (yes it can happen even with ECC). In the worst case there is a subtle numeric error that could potentially percolate to users. Usually it's about floating point issues, since usually you don't use floats to access arrays or do pointer arithm, issues in that unit may not cause an early failure and hence go undetected.
Simple case is that your machine just halts. If you have a way of looking at them, it might puts a bios code on the LEDs. On servers the baseboard management chip (BMC) will usually record a machine check event as well.