The reason you are not seeing crashes when allocating with Rust and freeing with C (or vice versa) is that by default Rust also uses the libc allocator.
I’m a bit confused by this writeup. It seems to me that you’re experiencing two issues:
1. An object that’s really a Rust Vec under the hood is awkward when you try to pass ownership to C. ISTM you end up solving it by creating what is, in effect, a handle, and freeing that using a custom free function. Is this really a problem with Rust FFI? This style of API is very common in native C code, and it’s a lot more flexible and future-proof than having a library user call free() directly.
2. You’re consuming the API in Rust and being sad that there’s no native defer statement. But Rust can do RAII — could you build a little wrapper struct that implements Drop?
Valgrind sometimes has problems with encapsulated C libraries slow anonymous memory leaks. For example, the session caching in modern libcurl is obfuscated fairly well from novices.
Valgrind is good to run as a sanity check, but I prefer a month under a regression test hammer to validate something isn't going to slowly frag itself before scheduled maintenance. =3
It's funny. When I first tried Rust in 2018 they were still statically linking jemalloc into every binary rustc compiled by default, and that alone very much put me off of the language for a while.
Apparently they did away with jemalloc in favor of the system allocator that same year but nonetheless when I came back to it years later I was very happy to learn of its removal.
Jemalloc was removed in November 2018 [1] and didn't land in stable where it defaulted to the system allocator until Rust 1.32 released in January 2019 [2]
At the time in 2018, however, I was using Linux but now use Windows. I just now learned that Rust used the system allocator in Windows longer than for Linux [2] where it was still using jemalloc for the majority of 2018.
Jemalloc added over a megabyte to every project for only questionable gains, and it was awkward and unwieldy to remove it. While there are good reasons to use a different allocator depending on the project, Rust defaulting to this type of behavior failed a certain personal litmus test on what it wanted to be as a language in that it felt like it was fighting the system rather than integrating with it.
It also does not give a good first impression at all to newcomers to see their hello world project built in release mode take up almost 2MiB of space. Today it's a much more (subjectively) tolerable 136kiB on Windows (considering that Rust std is statically linked).
Right, but, so what? I can't imagine a practical reason this matters for the vast majority of situations. So either you're doing something pretty niche, or it's a purely aesthetic complaint.
Okay, that's a niche scenario. I agree Rust during the statically linked jemalloc era might not be ideal in that scenario. But just because you can concoct some possible scenario in which it's not the best language to use doesn't mean you should write it off completely.
I would think Embedded Linux would be one of the most desired use cases for rust (certainly it's a place where higher level languages are often not an option due to resource usage).
It's a matter of what use cases are catered to by default, and what use cases are possible. For example, having debug symbols on by default makes it easier to start using the language and debug your applications, at the cost of more disk space and having to configure your project if your needs are different. I think the discussion would have more merit if the defaults weren't configurable. But of course, the choice of defaults is one that will be contentious no matter what. It should still be done following the Hippocratic Oath of "first, do no harm". And some of us need to accept that our usecases are more niche than others, including mine.
Seems like overstepping bounds. I expect a language runtime to make use of system provided facilities wherever feasible. I also expect it to provide hooks for me to link in alternatives.
If nothing else doing otherwise is likely to cause compatibility issues for someone at some point. For example see all the problems Go had with DIY syscalls pretty much everywhere except for Linux.
There's a legitimate question of whether the kernel ABI or the libc API qualifies as the system provided facilities on a linux box. But that uncertainty only furthers my latter point about compatibility.
There are embedded Linux systems with a total image size of ~10MB (not image as in docker "image" but an actual disk image). It would be quite hard to justify the additional dependency.
Comments
The reason you are not seeing crashes when allocating with Rust and freeing with C (or vice versa) is that by default Rust also uses the libc allocator.
https://stdrs.dev/nightly/x86_64-unknown-linux-gnu/src/std/s...
Miri and Valgrind will usually catch this kind of issue. I did lots of work mixing C and Rust and that tripped me as well at the beginning. I wrote about it if someone is interested: https://gaultier.github.io/blog/perhaps_rust_needs_defer.htm...
I’m a bit confused by this writeup. It seems to me that you’re experiencing two issues:
1. An object that’s really a Rust Vec under the hood is awkward when you try to pass ownership to C. ISTM you end up solving it by creating what is, in effect, a handle, and freeing that using a custom free function. Is this really a problem with Rust FFI? This style of API is very common in native C code, and it’s a lot more flexible and future-proof than having a library user call free() directly.
2. You’re consuming the API in Rust and being sad that there’s no native defer statement. But Rust can do RAII — could you build a little wrapper struct that implements Drop?
Valgrind sometimes has problems with encapsulated C libraries slow anonymous memory leaks. For example, the session caching in modern libcurl is obfuscated fairly well from novices.
Valgrind is good to run as a sanity check, but I prefer a month under a regression test hammer to validate something isn't going to slowly frag itself before scheduled maintenance. =3
It's funny. When I first tried Rust in 2018 they were still statically linking jemalloc into every binary rustc compiled by default, and that alone very much put me off of the language for a while.
Apparently they did away with jemalloc in favor of the system allocator that same year but nonetheless when I came back to it years later I was very happy to learn of its removal.
By 2018 it was the system malloc by default, so that does not add up.
Jemalloc was removed in November 2018 [1] and didn't land in stable where it defaulted to the system allocator until Rust 1.32 released in January 2019 [2]
[1]: https://github.com/rust-lang/rust/pull/55238
[2]: https://blog.rust-lang.org/2019/01/17/Rust-1.32.0/
At the time in 2018, however, I was using Linux but now use Windows. I just now learned that Rust used the system allocator in Windows longer than for Linux [2] where it was still using jemalloc for the majority of 2018.
Why?
Jemalloc added over a megabyte to every project for only questionable gains, and it was awkward and unwieldy to remove it. While there are good reasons to use a different allocator depending on the project, Rust defaulting to this type of behavior failed a certain personal litmus test on what it wanted to be as a language in that it felt like it was fighting the system rather than integrating with it.
It also does not give a good first impression at all to newcomers to see their hello world project built in release mode take up almost 2MiB of space. Today it's a much more (subjectively) tolerable 136kiB on Windows (considering that Rust std is statically linked).
Right, but, so what? I can't imagine a practical reason this matters for the vast majority of situations. So either you're doing something pretty niche, or it's a purely aesthetic complaint.
Those are certainly valid. There are entire Linux distros whose mere existence is as the answer to what is only an aesthetic complaint.
So it's wasted space if you don't need it. Being a small amount of space to waste doesn't make it reasonable to do that.
Imagine you had a bunch of rust binaries on your system instead of one. And you're running on a 2 GB emmc.
Okay, that's a niche scenario. I agree Rust during the statically linked jemalloc era might not be ideal in that scenario. But just because you can concoct some possible scenario in which it's not the best language to use doesn't mean you should write it off completely.
I would think Embedded Linux would be one of the most desired use cases for rust (certainly it's a place where higher level languages are often not an option due to resource usage).
It's a matter of what use cases are catered to by default, and what use cases are possible. For example, having debug symbols on by default makes it easier to start using the language and debug your applications, at the cost of more disk space and having to configure your project if your needs are different. I think the discussion would have more merit if the defaults weren't configurable. But of course, the choice of defaults is one that will be contentious no matter what. It should still be done following the Hippocratic Oath of "first, do no harm". And some of us need to accept that our usecases are more niche than others, including mine.
Seems like overstepping bounds. I expect a language runtime to make use of system provided facilities wherever feasible. I also expect it to provide hooks for me to link in alternatives.
If nothing else doing otherwise is likely to cause compatibility issues for someone at some point. For example see all the problems Go had with DIY syscalls pretty much everywhere except for Linux.
There's a legitimate question of whether the kernel ABI or the libc API qualifies as the system provided facilities on a linux box. But that uncertainty only furthers my latter point about compatibility.
There are embedded Linux systems with a total image size of ~10MB (not image as in docker "image" but an actual disk image). It would be quite hard to justify the additional dependency.
Okay, then don't use a language with statically-linked jemalloc for those. Why is that a reason to write off the entire language for all purposes?
Sorry, but why that can be a downside in 2018?