Interesting article, but I wouldn't say introducing a 3rd language is the easiest way to call C code from Python, that distinction goes to the CFFI library by Armin Rigo et al. I've used it to wrap several C libraries and it's amazingly powerful. Bonus it works with CPython and PyPy.
First there’s PSL ctypes,[1] which is supposedly inferior to cffi to some extent, but hey it comes with your CPython installation, no third-party dep required.
Or you can write a C/C++ extension module.[2] A bit more heavy-handed and there’s a learning curve, but once you’ve done it once or twice it’s really not hard, and it affords you great flexibility.
The solution in the blog post is an extension module. Written for you, by the compiler. Instead of a learning curve and "it's really not hard", it's "no learning curve" and "I struggle to find how this could be easier".
I would take a tested and true solution sanctioned by CPython core devs any day over a novelty third party library that gets another language toolchain involved.
Also, the solution seems to only cover what’s possible with ctypes already. If don’t need the flexibility there’s little reason to hand roll an extension module. (Not saying ctypes doesn’t have its problems; e.g. when you pass in a wrong number of arguments instead of throwing a TypeError it segfaults.)
ctypes does not parse C headers and doesn't know about enums or function parameter types — only that they have a symbol in the .dynsym section of a .so.
Maybe you're thinking of cffi? Cffi does basically what this blog post covers, except without D. It requires a C compiler at some stage in the process, but you can pregenerate during 'build' phase and ship a header snapshot to avoid requiring the C compiler at runtime.
No, ctypes does not. It can't handle macros, and it requires users to define the structures in Python themselves. It's miles away from what's in the blog post.
I'm curious about what ctypes' supposed inferiorities are. Last time I used it was to implement native printing in Windows (long story short: needed to print to a ZPL printer, and Qt doesn't seem to offer any ability to just send raw data to a printer, so I implemented it myself with the Winspool API), and it was a breeze (comparatively speaking, i.e. as much of a breeze as Windows API programming can be).
If ctypes can handle the Windows APIs, then I'm convinced it can handle just about anything.
ctypes allows you to trivially mistype C objects, send the wrong types and number of arguments to C functions, etc. Cffi just imports C headers and gives you more or less what TFA illustrates (without D).
I mentioned two completely different solutions, it’s not clear what “these” is referring to (both?). Also, both are clearly supported on Windows and used by tons of successful libraries and applications, so without details about your use case it’s not a very useful comment.
After a week, I've finally had time to look at cffi. It's not even remotely as easy. I tried using it just now to call nanomsg and gave up immediately. I can't just give it the headers, I have to pass it a string which is valid C code. I can't read the headers (separately!) and pass that in either because it can't handle preprocessor directives. It's a mess.
I don't understand how the OP could write this article and not talk about FFI? What a glaring oversight. (FWIW: The FFI module works extremely well in NodeJS as well.)
Comments
Interesting article, but I wouldn't say introducing a 3rd language is the easiest way to call C code from Python, that distinction goes to the CFFI library by Armin Rigo et al. I've used it to wrap several C libraries and it's amazingly powerful. Bonus it works with CPython and PyPy.
https://cffi.readthedocs.io/en/latest/
Don’t forget the builtin options.
First there’s PSL ctypes,[1] which is supposedly inferior to cffi to some extent, but hey it comes with your CPython installation, no third-party dep required.
Or you can write a C/C++ extension module.[2] A bit more heavy-handed and there’s a learning curve, but once you’ve done it once or twice it’s really not hard, and it affords you great flexibility.
[1] https://docs.python.org/3/library/ctypes.html
[2] https://docs.python.org/3/extending/extending.html
The solution in the blog post is an extension module. Written for you, by the compiler. Instead of a learning curve and "it's really not hard", it's "no learning curve" and "I struggle to find how this could be easier".
I would take a tested and true solution sanctioned by CPython core devs any day over a novelty third party library that gets another language toolchain involved.
Also, the solution seems to only cover what’s possible with ctypes already. If don’t need the flexibility there’s little reason to hand roll an extension module. (Not saying ctypes doesn’t have its problems; e.g. when you pass in a wrong number of arguments instead of throwing a TypeError it segfaults.)
ctypes does not parse C headers and doesn't know about enums or function parameter types — only that they have a symbol in the .dynsym section of a .so.
Maybe you're thinking of cffi? Cffi does basically what this blog post covers, except without D. It requires a C compiler at some stage in the process, but you can pregenerate during 'build' phase and ship a header snapshot to avoid requiring the C compiler at runtime.
I didn’t say ctypes takes the same approach — I even mentioned ctypes’ shortcomings. I only said ctypes makes what this does possible already.
No, ctypes does not. It can't handle macros, and it requires users to define the structures in Python themselves. It's miles away from what's in the blog post.
I'm curious about what ctypes' supposed inferiorities are. Last time I used it was to implement native printing in Windows (long story short: needed to print to a ZPL printer, and Qt doesn't seem to offer any ability to just send raw data to a printer, so I implemented it myself with the Winspool API), and it was a breeze (comparatively speaking, i.e. as much of a breeze as Windows API programming can be).
If ctypes can handle the Windows APIs, then I'm convinced it can handle just about anything.
ctypes allows you to trivially mistype C objects, send the wrong types and number of arguments to C functions, etc. Cffi just imports C headers and gives you more or less what TFA illustrates (without D).
I mean, so does C. Pretty sure that's a "feature". ;)
But yeah, fair point; it's definitely nice when an FFI wrapper is able to actually read the header files and figure out function/type signatures.
My problem with these is that in my use case, it completely broke on Windows...
I mentioned two completely different solutions, it’s not clear what “these” is referring to (both?). Also, both are clearly supported on Windows and used by tons of successful libraries and applications, so without details about your use case it’s not a very useful comment.
After a week, I've finally had time to look at cffi. It's not even remotely as easy. I tried using it just now to call nanomsg and gave up immediately. I can't just give it the headers, I have to pass it a string which is valid C code. I can't read the headers (separately!) and pass that in either because it can't handle preprocessor directives. It's a mess.
In the article, I do this:
``` #include "nanomsg/nn.h" #include "nanomsg/pipeline.h" ```
That's it. I don't think the approaches can be compared.
I don't understand how the OP could write this article and not talk about FFI? What a glaring oversight. (FWIW: The FFI module works extremely well in NodeJS as well.)