That doesn't mean it's easier to interface to Python than C or C++ though. I'd find it hard to imagine it being easier, and I know it's not simpler.
I think Rust can be easier to learn than C. You get high-level features like modules instead of include guards and preprocessor hacks, and you can skip learning the part about how to debug segfaults.
I'll have to watch the video below, but don't you have to write some sort of Rust wrapper for every definition in Python.h? What about macros like Py_INCREF and Py_DECREF?
We have bindgen to automate most of that for you. For macros, you do need to duplicate them, but people should just do that once (and Google brings up several projects in which people have already written bindings for that).
People always try to cover it up with "nice" abstractions, but they invariably end up being leaky (e.g. with respect to threads, garbage collection, OS portability, etc.)
But I don't see how Rust makes that any worse than in C.
Another huge can of worms: the build system. With a plain C extension, all I need is a C compiler on the system, and I can just do "python setup.py build". The situation with Windows is also quite messy -- I can't imagine Rust making it better.
Yes, you do need to add Rust support to the extension's build system. That is fair, but I think Rust's advantages outweigh that cost :)
A system composed of Python and C will necessarily have fewer layers than one composed of Python and Rust, simply because Python is written in C and its interface is defined in C.
That doesn't make any sense to me. It's all machine code at runtime. The pertinent question is whether Rust adds any measurable abstraction taxes at runtime (it doesn't) and whether Rust uses the same ABI as C (it does).
EDIT: I left out the BIGGEST point -- a dealbreaker. Python does NOT have an ABI. It has an API. AFAIK, that means you have to write a Rust ABI-compatible wrapper for EVERY VERSION of Python.
That is unfortunate, but someone could write a crate that dynamically determines which Python is in use and chooses the right functions based on that, put it on Cargo, and everyone could link against it.
Anyway, people are already writing Python programs that call into Rust via ctypes and whatnot, so this hasn't stopped people yet.
So if you just want to speed up an existing Python program, I would still recommend using a simple C or C++ extension. This solves both single-threaded speed issues and give you parallelism with multiple cores, so you will get huge speedups.
At the cost of memory safety, which is a big deal. And you have to learn C, which for a dynamic language programmer can be a lot harder than learning Rust.
Since you mentioned multicore, I should mention that the threading facilities available "out of the box" in C and C++ are much harder to use than the equivalent ones available in Rust, and if you reach for something like TBB and Boost now you have all the build system issues you described previously.
Sure, we have the same understanding of what's going on in each situation. But I can't see how anyone could call the Rust version as simple as C/C++. There are simply more concepts involved.
You can try to cover them up with magic code generators and version wrappers, but in my experience those are precisely the leaky abstractions that make people scared of FFI. They work 99% of the time, then bite you 1% of the time. The built-in Python FFI is somewhat ugly, but simple and debuggable.
I'm not saying you shouldn't use Rust for Python extensions -- just that there is an inherent awkwardness in doing so, given that Python is written in C, and the Python 2.x has no stable ABI. Of course Rust offers benefits, so if you want to pay the cost for those benefits, it might be worth it.
Comments
I think Rust can be easier to learn than C. You get high-level features like modules instead of include guards and preprocessor hacks, and you can skip learning the part about how to debug segfaults.
We have bindgen to automate most of that for you. For macros, you do need to duplicate them, but people should just do that once (and Google brings up several projects in which people have already written bindings for that).
But I don't see how Rust makes that any worse than in C.
Yes, you do need to add Rust support to the extension's build system. That is fair, but I think Rust's advantages outweigh that cost :)
That doesn't make any sense to me. It's all machine code at runtime. The pertinent question is whether Rust adds any measurable abstraction taxes at runtime (it doesn't) and whether Rust uses the same ABI as C (it does).
That is unfortunate, but someone could write a crate that dynamically determines which Python is in use and chooses the right functions based on that, put it on Cargo, and everyone could link against it.
Anyway, people are already writing Python programs that call into Rust via ctypes and whatnot, so this hasn't stopped people yet.
At the cost of memory safety, which is a big deal. And you have to learn C, which for a dynamic language programmer can be a lot harder than learning Rust.
Since you mentioned multicore, I should mention that the threading facilities available "out of the box" in C and C++ are much harder to use than the equivalent ones available in Rust, and if you reach for something like TBB and Boost now you have all the build system issues you described previously.
Sure, we have the same understanding of what's going on in each situation. But I can't see how anyone could call the Rust version as simple as C/C++. There are simply more concepts involved.
You can try to cover them up with magic code generators and version wrappers, but in my experience those are precisely the leaky abstractions that make people scared of FFI. They work 99% of the time, then bite you 1% of the time. The built-in Python FFI is somewhat ugly, but simple and debuggable.
I'm not saying you shouldn't use Rust for Python extensions -- just that there is an inherent awkwardness in doing so, given that Python is written in C, and the Python 2.x has no stable ABI. Of course Rust offers benefits, so if you want to pay the cost for those benefits, it might be worth it.