I'm not advocating for a complete rewrite of Postgres in C, but I'd like to emphasize the fact that security is not the only good reason to justify the transition of a project from C to Rust:
Nowadays most developers comes from memory-managed languages
=> for us, C is a scary land where it's really hard to get things right when you don't have all the prerequisite knowledge (what's the stack ?)
=> the language is really old, you don't have all the fancy and useful tools and syntactic sugar (package manager, proper error handling, generics)
=> then we don't want to invest a lot of time to learn a difficult language that is so unproductive.
=> and even if we do out of curiosity, we'll most likely never feel confident enough to dig into an open-source project and contribute.
Rust on the other hand, is like C but with a personal IA assistant (the compiler) and all the syntactic sugar you're used to (and even more if you're not familiar with the ML family). That's what's attracting so many web developers to Rust !
In my opinion it means that, in a somewhat near future, the amount of potential contributors for projects written in Rust could be higher than for projects written in C. And those contributor are also likely to be more productive, thanks to the language features and tooling.
You're describing qualities that are meaningless to a database user. A Rust implementation of PostgreSQL, with its runtime checks, would almost certainly be slower than the existing implementation. So a Rust port is a non-starter. Database users don't care about your syntactic sugar, that you find manual memory management a "scary land" due to background in managed languages, or how many contributors you have. Speaking of which, the PostgreSQL project is healthy and has many, many contributors. And while we're discussing developer productivity, I can't even imagine the build time of a Rust port.
What's annoying about OP's email is that they clearly did no research on the matter. You can find many discussions on the list about "upgrading" to C++, and there has been much wringing of hands. The last I heard, it sounded like a faction was considering having the project build with a C++ compiler rather than a C compiler for better type checks and as a step on that path.
A Rust implementation of PostgreSQL, with its runtime checks, would almost certainly be slower than the existing implementation. So a Rust port is a non-starter.
Wait, which runtime checks are you talking about ? This is a recurring FUD we often see on HN about Rust and I'd like to clarify this point.
IIRC, there is no mandatory runtime check in idiomatic Rust except UTF8 validation for strings[0]. There is no overflow checks on numbers, and no bounds checking when using iterators[1]. Another known overhead is the default use of siphash in hashmaps for DoS prevention[2], but this is not really a «runtime check».
Do you have anything else in mind ?
Speaking of which, the PostgreSQL project is healthy and has many, many contributors.
Indeed, and as I said, I'm not advocating for a port of PostgreSQL in Rust. But for other projects that suffer from a lack of attention[3], it could be a good move.
And while we're discussing developer productivity, I can't even imagine the build time of a Rust port.
touché, but compile time is one of the biggest focus of the Rust core team atm, so I'm confident things will improve in 2017 :)
[1]: there are bounds checking when you access array elements through indexes, but most of the time you don't do that since iterators are great. If you really need to, and this have a performance impact, you can opt-out the bound checking in unsafe block.
[2]: But this is opt-out by using a different hash function, if you use your hashmap in a non adversarial environment.
[3]: like librsvg which was chronically under-maintained.
[1] is great. Rephrased, "there isn't runtime bounds checking, unless you use the language. But you can opt out in an `unsafe` block." OK? So we're back to the usual Rust argument: At least your unsafe code is greppable. And I don't think any competent C or C++ developer considers that a worthy tradeoff when it comes with a more complex, constantly evolving language, far less mindshare, slow builds without solutions (unlike C++), and so on.
Regarding [2], I think it's safe to consider any software that takes input as operating in a hostile environment.
That is an extreme mis-characterization of stymaar's excellent summary. in short, there are no runtime checks for iterating through an array, and since the general case of random access to an array can't be safe (without dependent typing), that is the one place the runtime checks are automatically added, and even there you can opt out of them. It is easy to get C equivalent speed with far more safety.
You are spreading FUD, indeed.
Which runtime checks are you concerned about, exactly?
Just array indexing? It would be interesting to see your criticism fleshed out.
I am not sure your criticism will amount to anything if array access is it, though. Posrgres is full of custom data structures and macros as it is. StringInfo and stuff like that -- they are not just "using the language", writing nice simple C code. Probably because databases really push you to use advanced techniques in places.
there are bounds checking when you access array elements through indexes
I wish we didn't need bounds checking; the problem could be solved with dependently typed languages. Unfortunately Rust just panics when you access out of bounds, and doesn't warn you at compile time. Rust isn't really all that impressive.
By your own admission, what you are asking for would require dependent typing. I am hugely in favor of rust eventually acquiring such capability, but it would be a huge change, and there are reasons why dependent typing hasn't hit mainstream languages yet.
If that is what I think it is, see https://docs.rs/indexing for something similar, crafted from invariance over higher-ranked lifetimes, instead of types.
I don't think Rust can ever replicate the Haskell implementation identically (if we had HRTB over types), as type parametrism is gone (see: specialization RFC).
However, lifetime parametrism serves a similar role in Rust, and lifetimes are closer to a concept of "instance" than types.
You're describing qualities that are meaningless to a database user.
This is an old strawman. No one is asking the users to write Rust and a change in developer tooling doesn't by itself change things for users. The question is whether tools help developers help users.
Comments
I'm not advocating for a complete rewrite of Postgres in C, but I'd like to emphasize the fact that security is not the only good reason to justify the transition of a project from C to Rust:
Nowadays most developers comes from memory-managed languages => for us, C is a scary land where it's really hard to get things right when you don't have all the prerequisite knowledge (what's the stack ?) => the language is really old, you don't have all the fancy and useful tools and syntactic sugar (package manager, proper error handling, generics) => then we don't want to invest a lot of time to learn a difficult language that is so unproductive. => and even if we do out of curiosity, we'll most likely never feel confident enough to dig into an open-source project and contribute.
Rust on the other hand, is like C but with a personal IA assistant (the compiler) and all the syntactic sugar you're used to (and even more if you're not familiar with the ML family). That's what's attracting so many web developers to Rust !
In my opinion it means that, in a somewhat near future, the amount of potential contributors for projects written in Rust could be higher than for projects written in C. And those contributor are also likely to be more productive, thanks to the language features and tooling.
You're describing qualities that are meaningless to a database user. A Rust implementation of PostgreSQL, with its runtime checks, would almost certainly be slower than the existing implementation. So a Rust port is a non-starter. Database users don't care about your syntactic sugar, that you find manual memory management a "scary land" due to background in managed languages, or how many contributors you have. Speaking of which, the PostgreSQL project is healthy and has many, many contributors. And while we're discussing developer productivity, I can't even imagine the build time of a Rust port.
What's annoying about OP's email is that they clearly did no research on the matter. You can find many discussions on the list about "upgrading" to C++, and there has been much wringing of hands. The last I heard, it sounded like a faction was considering having the project build with a C++ compiler rather than a C compiler for better type checks and as a step on that path.
Wait, which runtime checks are you talking about ? This is a recurring FUD we often see on HN about Rust and I'd like to clarify this point.
IIRC, there is no mandatory runtime check in idiomatic Rust except UTF8 validation for strings[0]. There is no overflow checks on numbers, and no bounds checking when using iterators[1]. Another known overhead is the default use of siphash in hashmaps for DoS prevention[2], but this is not really a «runtime check».
Do you have anything else in mind ?
Indeed, and as I said, I'm not advocating for a port of PostgreSQL in Rust. But for other projects that suffer from a lack of attention[3], it could be a good move.
touché, but compile time is one of the biggest focus of the Rust core team atm, so I'm confident things will improve in 2017 :)
[0]: which can be worked around if need be by using &[u8] instead of &str (see: https://news.ycombinator.com/item?id=13268051 from /u/burntsushi)
[1]: there are bounds checking when you access array elements through indexes, but most of the time you don't do that since iterators are great. If you really need to, and this have a performance impact, you can opt-out the bound checking in unsafe block.
[2]: But this is opt-out by using a different hash function, if you use your hashmap in a non adversarial environment.
[3]: like librsvg which was chronically under-maintained.
[1] is great. Rephrased, "there isn't runtime bounds checking, unless you use the language. But you can opt out in an `unsafe` block." OK? So we're back to the usual Rust argument: At least your unsafe code is greppable. And I don't think any competent C or C++ developer considers that a worthy tradeoff when it comes with a more complex, constantly evolving language, far less mindshare, slow builds without solutions (unlike C++), and so on.
Regarding [2], I think it's safe to consider any software that takes input as operating in a hostile environment.
That is an extreme mis-characterization of stymaar's excellent summary. in short, there are no runtime checks for iterating through an array, and since the general case of random access to an array can't be safe (without dependent typing), that is the one place the runtime checks are automatically added, and even there you can opt out of them. It is easy to get C equivalent speed with far more safety. You are spreading FUD, indeed.
Which runtime checks are you concerned about, exactly?
Just array indexing? It would be interesting to see your criticism fleshed out.
I am not sure your criticism will amount to anything if array access is it, though. Posrgres is full of custom data structures and macros as it is. StringInfo and stuff like that -- they are not just "using the language", writing nice simple C code. Probably because databases really push you to use advanced techniques in places.
I wish we didn't need bounds checking; the problem could be solved with dependently typed languages. Unfortunately Rust just panics when you access out of bounds, and doesn't warn you at compile time. Rust isn't really all that impressive.
By your own admission, what you are asking for would require dependent typing. I am hugely in favor of rust eventually acquiring such capability, but it would be a huge change, and there are reasons why dependent typing hasn't hit mainstream languages yet.
I know nothing about Rust, but you might find Oleg Kiselyov's: "Eliminating Array Bound Checking through Non-Dependent Types" interesting.
http://okmij.org/ftp/Haskell/eliminating-array-bound-check.l...
You may also like:
http://okmij.org/ftp/Haskell/number-parameterized-types.pdf
http://okmij.org/ftp/Computation/resource-aware-prog/tfp.pdf
If that is what I think it is, see https://docs.rs/indexing for something similar, crafted from invariance over higher-ranked lifetimes, instead of types.
I don't think Rust can ever replicate the Haskell implementation identically (if we had HRTB over types), as type parametrism is gone (see: specialization RFC). However, lifetime parametrism serves a similar role in Rust, and lifetimes are closer to a concept of "instance" than types.
By comparison to C, though...
This is an old strawman. No one is asking the users to write Rust and a change in developer tooling doesn't by itself change things for users. The question is whether tools help developers help users.