The (costly) lesson learned here doesn't revolve around Primedice's RNG architecture but the strict importance of testing.
We all face pressure to deliver but at the end of the day -like brakes on a car- testing is one thing you should never cut corners on.
Although another 2 weeks of testing may not have explicitly exposed the vulnerability, it surely would have offered a better baseline from which to evolve better heuristics for the analysis of the exploit when it did occur.
I'd say the degree of testing depends on the domain. Facebook in its early days could probably afford to drop a some data on the floor. Boeing and its subcontractors can't simply can't screw up flight software (or this happens: https://en.wikipedia.org/wiki/2015_Seville_A400M_crash). In-between, we're seeing a lot of Bitcoin and other sorts of handling money associated firms discover that "move fast and break things" can result rather quickly in a broken firm.
Yeah, this is definitely one of those places where if you didn't test and analyze the system relatively well before deployment, you probably shouldn't be surprised that someone came along and did this. Is it nice/fair, no, not really, but if you're going to be moving lots of money in and out, you really ought to be developing to the same standards as maybe a brokerage or bank (not that they do exceptionally well on this front either). Also doxxing this guy in this context seems rather juvenile, especially given that it's unlikely it will help your cause in any tangible way.
Testing is important, but I don't think that's enough for this sort of problem. What this needs is in-depth code review. Tests are only as good as the tests you write, and if there was no test to account for this particular issue nothing would have been caught.
For concurrency issues someone thinking real hard is as important as testing.
someone thinking real hard is as important as testing
With one caveat.
The "thinking real hard" has to happen before and as the code is written.
If you try to catch fundamental design problems and misconceptions during a code review, you're much too late. At that point there is a "status quo" i.e. the existing code, and a defensive resistance to change, even if you can demonstrate clear problems with said code. The end result is band-aids on top of band-aids, rather than a proper solution to the problems.
Edit: the article itself alludes to band-aids:
our developer had improperly patched the glitch
i.e. they attempted a quick fix, instead of understanding at a fundamental level the overall mistakes in what they were doing.
Comments
The (costly) lesson learned here doesn't revolve around Primedice's RNG architecture but the strict importance of testing.
We all face pressure to deliver but at the end of the day -like brakes on a car- testing is one thing you should never cut corners on.
Although another 2 weeks of testing may not have explicitly exposed the vulnerability, it surely would have offered a better baseline from which to evolve better heuristics for the analysis of the exploit when it did occur.
I'd say the degree of testing depends on the domain. Facebook in its early days could probably afford to drop a some data on the floor. Boeing and its subcontractors can't simply can't screw up flight software (or this happens: https://en.wikipedia.org/wiki/2015_Seville_A400M_crash). In-between, we're seeing a lot of Bitcoin and other sorts of handling money associated firms discover that "move fast and break things" can result rather quickly in a broken firm.
Yeah, this is definitely one of those places where if you didn't test and analyze the system relatively well before deployment, you probably shouldn't be surprised that someone came along and did this. Is it nice/fair, no, not really, but if you're going to be moving lots of money in and out, you really ought to be developing to the same standards as maybe a brokerage or bank (not that they do exceptionally well on this front either). Also doxxing this guy in this context seems rather juvenile, especially given that it's unlikely it will help your cause in any tangible way.
Testing is important, but I don't think that's enough for this sort of problem. What this needs is in-depth code review. Tests are only as good as the tests you write, and if there was no test to account for this particular issue nothing would have been caught.
For concurrency issues someone thinking real hard is as important as testing.
someone thinking real hard is as important as testing
With one caveat.
The "thinking real hard" has to happen before and as the code is written.
If you try to catch fundamental design problems and misconceptions during a code review, you're much too late. At that point there is a "status quo" i.e. the existing code, and a defensive resistance to change, even if you can demonstrate clear problems with said code. The end result is band-aids on top of band-aids, rather than a proper solution to the problems.
Edit: the article itself alludes to band-aids:
i.e. they attempted a quick fix, instead of understanding at a fundamental level the overall mistakes in what they were doing.