I wonder what effect self selection bias has in people who end up writing hand crafted complex parsing code in C for untrusted data in ring 0. You either have to believe that it's doable to get right or that it doesn't matter much if you don't, or "it's not my worry".
There are plenty of tools designed to help you write parsers that compile down to C. Alternatively there is the microkernel approach. Either one (or both) would satisfy GP's implication that hand-written C parsers in ring 0 are bad.
Well, Android's Bluetooth stack is being rewritten in Rust so you probably can go pretty far in memory-safe languages (though of course Rust isn't a managed language, just a safe one).
Yeah, and we're slowly slowly moving towards that model. One barrier is that some safe languages, like Rust, support far fewer target platforms than the Linux kernel does. There are C compilers for everything, but only a few Rust backends.
Something similar to Wuffs[0] (posted on HN very recently), which compiles down to C, might be a good compromise between portability and safe languages. (There may be some contorted way to have Rust emit C, too.)
Rust uses LLVM, which likely should have easier time being ported than an entire Rust compiler. Rust handles cross-compilation pretty well so you don't have to compile on an actual embedded device, or something.
I wonder if an LLVM backend that issues a very simple and predictable subset of C would be a viable way to support exotic old architectures which only have a C compiler. LLVM-cbe is a thing: https://github.com/JuliaComputing/llvm-cbe
Indeed! Hoare wrote about his self-reflection on the failures of his own software group.
There was also failure in prediction, in estimation of program size and speed, of effort required, in planning the coordination and interaction of programs, in providing an early warning that things were going wrong. There were faults in our control of program changes, documentation, liaison with other departments, with our management, and with our customers. We failed in giving clear and stable definitions of the responsibilities of individual programmers and project leaders,--Oh, need I go on?
What was amazing was that a large team of highly intelligent programmers could labor so hard and so long on such an unpromising project. You know, you shouldn't trust us intelligent programmers. We can think up such good arguments for convincing ourselves and each other of the utterly absurd. Especially don't believe us when we promise to repeat an earlier success, only bigger and better next time.
Apologies. Yeah, there's room for a lot of whataboutism as long as browsers and other user facing apps are also using unsafe languages for parsing untrusted code. But still that's not an excuse to do it IMO.
And indeed "the system is to blame" in that it's hard to get drivers using safe techniques into the Linux kernel, and Linus is famously anti-security. But I think for the individual programmer, they still end up choosing one of the above 3 mental models.
So I still think it would be interesting to know how people think about security when writing parsers in C in ring 0 for Linux drivers that exposed by default in billions in devices.
It's not like NASA sent a rover, written almost fully in C, to Mars. It's not like billions of cars and even more billions of their ECUs are written in C. It's not like the firmware of the keyboard you're writing your comment on, or even the OS/browser you're using is written in C. C bad, Rust good.
This isn't about Rust, the same problems were well known (and long suffered & complaned about) before Rust came around and there are well known much older engineering techniques for parsing untrusted data. It's nice that the Rust phenomenon has brought with it some new spirit of vigor and momentum to break out of the apathy, though.
Re Rover code and ECUs - this is the difference between safety critical code and security critical code at the attack surface.
The first kind deals primarily with "don't keel over or go crazy when natural phenomena throws unexpected circumstances at you", the second deals with inputs crafted by intelligent adversaries who can see your code and test & iterate attacks to exploit any flaws they uncovered through analysis or experimentation against your implementation. (Of course if we nitpick, an intelligent attacker is a natural phenomenon.)
NASA rovers and car ECUs have minimal people looking to exploit them, so I'm not overly convinced they're exploit free either.
I'm not a Rust evangelist or even a user, but the current paradigm of "THIS TIME, we'll write safe, complex, performant C/C++ code properly" isn't the solution, nor is manually squashing bugs one by one.
The solution seems to be a combination of improving the tooling around existing C\C++, and starting new projects in safer languages when possible.
Comments
I wonder what effect self selection bias has in people who end up writing hand crafted complex parsing code in C for untrusted data in ring 0. You either have to believe that it's doable to get right or that it doesn't matter much if you don't, or "it's not my worry".
// quick hack, fix later
{ .... }
Not a valid C comment.
Double slash single-line comments are valid C99 and later.
They also work in Borland C 3.1 for DOS
[flagged]
There are plenty of tools designed to help you write parsers that compile down to C. Alternatively there is the microkernel approach. Either one (or both) would satisfy GP's implication that hand-written C parsers in ring 0 are bad.
Well, Android's Bluetooth stack is being rewritten in Rust so you probably can go pretty far in memory-safe languages (though of course Rust isn't a managed language, just a safe one).
Dumb question (not a kernel or C developer): can't you call into code compiled from a memory safe language, like in a shared object file?
Yeah, and we're slowly slowly moving towards that model. One barrier is that some safe languages, like Rust, support far fewer target platforms than the Linux kernel does. There are C compilers for everything, but only a few Rust backends.
Something similar to Wuffs[0] (posted on HN very recently), which compiles down to C, might be a good compromise between portability and safe languages. (There may be some contorted way to have Rust emit C, too.)
[0]: https://github.com/google/wuffs
Rust uses LLVM, which likely should have easier time being ported than an entire Rust compiler. Rust handles cross-compilation pretty well so you don't have to compile on an actual embedded device, or something.
I wonder if an LLVM backend that issues a very simple and predictable subset of C would be a viable way to support exotic old architectures which only have a C compiler. LLVM-cbe is a thing: https://github.com/JuliaComputing/llvm-cbe
Definitely, for example check how to make Go .so libraries.
https://medium.com/swlh/build-and-use-go-packages-as-c-libra...
Naturally the runtime also comes along, but that is another matter.
Do you know C.A.Hoare?
I advise you to read his Turing award speech from 1981.
https://www.cs.fsu.edu/~engelen/courses/COP4610/hoare.pdf
Indeed! Hoare wrote about his self-reflection on the failures of his own software group.
There was also failure in prediction, in estimation of program size and speed, of effort required, in planning the coordination and interaction of programs, in providing an early warning that things were going wrong. There were faults in our control of program changes, documentation, liaison with other departments, with our management, and with our customers. We failed in giving clear and stable definitions of the responsibilities of individual programmers and project leaders,--Oh, need I go on?
What was amazing was that a large team of highly intelligent programmers could labor so hard and so long on such an unpromising project. You know, you shouldn't trust us intelligent programmers. We can think up such good arguments for convincing ourselves and each other of the utterly absurd. Especially don't believe us when we promise to repeat an earlier success, only bigger and better next time.
Apologies. Yeah, there's room for a lot of whataboutism as long as browsers and other user facing apps are also using unsafe languages for parsing untrusted code. But still that's not an excuse to do it IMO.
And indeed "the system is to blame" in that it's hard to get drivers using safe techniques into the Linux kernel, and Linus is famously anti-security. But I think for the individual programmer, they still end up choosing one of the above 3 mental models.
So I still think it would be interesting to know how people think about security when writing parsers in C in ring 0 for Linux drivers that exposed by default in billions in devices.
its called a microkernel
Yup. C bad, Rust good.
It's not like NASA sent a rover, written almost fully in C, to Mars. It's not like billions of cars and even more billions of their ECUs are written in C. It's not like the firmware of the keyboard you're writing your comment on, or even the OS/browser you're using is written in C. C bad, Rust good.
This isn't about Rust, the same problems were well known (and long suffered & complaned about) before Rust came around and there are well known much older engineering techniques for parsing untrusted data. It's nice that the Rust phenomenon has brought with it some new spirit of vigor and momentum to break out of the apathy, though.
Re Rover code and ECUs - this is the difference between safety critical code and security critical code at the attack surface.
The first kind deals primarily with "don't keel over or go crazy when natural phenomena throws unexpected circumstances at you", the second deals with inputs crafted by intelligent adversaries who can see your code and test & iterate attacks to exploit any flaws they uncovered through analysis or experimentation against your implementation. (Of course if we nitpick, an intelligent attacker is a natural phenomenon.)
Those have tons of exploitable bugs!!
NASA rovers and car ECUs have minimal people looking to exploit them, so I'm not overly convinced they're exploit free either.
I'm not a Rust evangelist or even a user, but the current paradigm of "THIS TIME, we'll write safe, complex, performant C/C++ code properly" isn't the solution, nor is manually squashing bugs one by one.
The solution seems to be a combination of improving the tooling around existing C\C++, and starting new projects in safer languages when possible.