HPE was notified by a Solid State Drive (SSD) manufacturer [...]
That's a curious bit of context. It seems to imply they're shifting some of the blame onto their manufacturer? I makes me wonder if this firmware is 100% HPE specific, or if there a 2^16 hours bug about to bite a bunch of other pipelines.
Of course HPE doesn't write their own firmware from scratch. It's likely just whitelabeled by the drive manufacturer. HPE is just a reseller of existing OEM drives, like all other enterprise server manufacturers are.
There might have been some small changes, but the bulk of the code is likely copy pasted from a common codebase that is shared across models or even families. There's no way they're rewriting the entire controller code just for one customer, it doesn't make sense technically or from a business perspective.
This is a 2^15 hours bug, not a 2^16 hours bug. Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra".
Using signed integers for values that are always positive isn't necessarily a mistake.
Most notably because for signed integers (in C and C++) overflow is undefined behavior. This allows for more aggressive optimizations by the compiler.
Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior.
Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior.
I read somewhere that the true reason for that advice is that it allows the compiler to silently store an "int" loop counter in a 64-bit register, without having to care about 32-bit overflow. If you use size_t for the loop counter, that's no longer an issue.
It commonly factors in loop analysis around unrolling and vectorization, e.g. the loop will run exactly 16*n times OR an integer will overflow and it'll run some other number of times.
Use of signed precludes the overflow and the exact bound enables efficient vectorization.
Undefined behavior is platform-specific behavior. In this case it means that rollover’s effect on the value depends on how the register stores the integer and on how the particular instruction is documented to behave.
No, undefined behavior is specified by the standard. There is also 'implementation defined' which tends to be both architecture and compiler specific.
The difference is that a program which invokes 'implementation defined' behavior can be well defined, whereas a program that invokes undefined behavior is literally free to do anything.
Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra".
Our industry really sucks. We need languages where this can't happen, and we need testing procedures where these things are caught. I wonder if software is the industry with the lowest quality:importance ratio.
Presumably the bug would still be considered a bug if it occured at 65536 hours? The incorrectly-signed bit only makes it appear sooner, but it's not the bug.
If you purchase a Carepaq, can actually extend the service beyond that?
3,4,and 5-Year 24x7 Carepaqs are available on purchase for the _whole_ chassis and parts... but I wonder if I can keep purchasing 1-year carepaq extensions beyond the 6th year.
I'm always baffled when companies try to pass these issues off onto one of their sub-vendors - especially for critical issues like this.
I didn't choose YOUR sub-vendor. You did. It's your responsibility to ensure that sub-vendor is operating at your standards. Passing blame to a sub-vendor indicates an unwillingness to take accountability.
I probably wouldn't even call it blame. Certainly if HPE isn't doing a full firmware audit (which I don't expect them to do), there's no way to run into this issue until it shows up in life testing. The manufacturer/supplier would be in the best position to encounter these types of issues first.
Comments
That's a curious bit of context. It seems to imply they're shifting some of the blame onto their manufacturer? I makes me wonder if this firmware is 100% HPE specific, or if there a 2^16 hours bug about to bite a bunch of other pipelines.
Of course HPE doesn't write their own firmware from scratch. It's likely just whitelabeled by the drive manufacturer. HPE is just a reseller of existing OEM drives, like all other enterprise server manufacturers are.
It is possible, though, that this firmware was written specifically for HPE by the OEM.
There might have been some small changes, but the bulk of the code is likely copy pasted from a common codebase that is shared across models or even families. There's no way they're rewriting the entire controller code just for one customer, it doesn't make sense technically or from a business perspective.
The HP specific part is just to make sure that the HP disk vendor lockin works: HP controllers only talk to HP branded disks...
This is a 2^15 hours bug, not a 2^16 hours bug. Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra".
Using signed integers for values that are always positive isn't necessarily a mistake. Most notably because for signed integers (in C and C++) overflow is undefined behavior. This allows for more aggressive optimizations by the compiler.
Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior.
I read somewhere that the true reason for that advice is that it allows the compiler to silently store an "int" loop counter in a 64-bit register, without having to care about 32-bit overflow. If you use size_t for the loop counter, that's no longer an issue.
I haven't seen an example where that's the case.
It commonly factors in loop analysis around unrolling and vectorization, e.g. the loop will run exactly 16*n times OR an integer will overflow and it'll run some other number of times.
Use of signed precludes the overflow and the exact bound enables efficient vectorization.
At least on X86 that truncation behavior is free. 32-bit EAX is just first 32-bits of RAX.
Undefined behavior is platform-specific behavior. In this case it means that rollover’s effect on the value depends on how the register stores the integer and on how the particular instruction is documented to behave.
No, undefined behavior is specified by the standard. There is also 'implementation defined' which tends to be both architecture and compiler specific.
The difference is that a program which invokes 'implementation defined' behavior can be well defined, whereas a program that invokes undefined behavior is literally free to do anything.
And also more useful warnings from static analysis, since if the analysis can prove that the value will overflow this is guaranteed to be an error.
Our industry really sucks. We need languages where this can't happen, and we need testing procedures where these things are caught. I wonder if software is the industry with the lowest quality:importance ratio.
Presumably the bug would still be considered a bug if it occured at 65536 hours? The incorrectly-signed bit only makes it appear sooner, but it's not the bug.
Someone says below that rollover may work OK but not negative times, for whatever reasons.
Oh! So in that case owners just need to wait 32,768 hours and they'll be fine. /s
That's 7.4 years, and well outside warranty.
If you purchase a Carepaq, can actually extend the service beyond that?
3,4,and 5-Year 24x7 Carepaqs are available on purchase for the _whole_ chassis and parts... but I wonder if I can keep purchasing 1-year carepaq extensions beyond the 6th year.
Yes, its possible. However, prices go up with age.
2^16 would be still rather too small, 65536 hours is not impossible duration.
I'm always baffled when companies try to pass these issues off onto one of their sub-vendors - especially for critical issues like this.
I didn't choose YOUR sub-vendor. You did. It's your responsibility to ensure that sub-vendor is operating at your standards. Passing blame to a sub-vendor indicates an unwillingness to take accountability.
I mean, that's no shock coming from HP.
I probably wouldn't even call it blame. Certainly if HPE isn't doing a full firmware audit (which I don't expect them to do), there's no way to run into this issue until it shows up in life testing. The manufacturer/supplier would be in the best position to encounter these types of issues first.