Every language has unsafe bits if you look deep enough. CPython and Java's garbage collector are written in C, and we have ctypes and JNI, but I think most people would still consider those memory safe languages. The important question is: is there a well-defined safety boundary, somewhere between the CPU and your application code, where memory is guaranteed* not to be corrupted and the programmer is freed from worrying about low-level bugs like segfaults?
*assuming no outside bugs such as CPU bugs, bugs in the GC, or bugs in an unsafe block
This doesn’t make sense. So rust let’s you write unsafe code, and now that’s the same as any language that calls C in any way?
Why replace C++ then? After all, rust is just as unsafe by your definition.
Having a language that is default safe and can only be unsafe in rare situations is way better than one that lets libraries write unsafe code, which is better than a language with no safe boundary.
Having a language that is default safe and can only be unsafe in rare situations
This describes both Python and Rust. They are default safe, with escape hatches (ctypes or unsafe blocks) for rare situations. Writing unsafe code in Rust should be as rare as using ctypes in Python.
is way better than one that lets libraries write unsafe code, which is better than a language with no safe boundary.
Python lets libs use ctypes to poke at raw memory. Solution: avoid those libraries
Rust lets libs use unsafe blocks. Solution: avoid those libraries (and optionally use tools like cargo-geiger or forbid(unsafe) if you care enough
If you don’t seem a parallel between what you just said: “just use these tools to catch the unsafe parts of rust” and the C++ crowd saying “just use these tools or do these things to catch the unsafe parts of cpp” there’s no point in continuing this discussion.
Python ctypes are not at all as common as rust unsafe code and geiger only goes so deep in search for unsafe code if I remember correctly.
So your opinion is that safety holes are only important if the syntax used is "common"? IME unsafe Rust code in the wild is rare and there's a strong culture against its use. Meanwhile in Python, ctypes and C modules are very common -- just about all popular libs use C under the hood, numpy, scikit, django, etc. It's tough to do anything useful without it.
Does Python have any similar tooling for managing all that unsafe code? Or is the only option to hope and pray that the memory vulnerabilities are not too common?
You can say “python calls C and that makes it memory unsafe” but so does every language, including rust. It is well known GCs are memory safe, all those articles about memory safety push for GCs first, and rust second.
About half of Python libraries in PyPI may have security issues
Pretty bad, right? I'll take the ecosystem with 265 issues over the one with 749000 issues any day.
so does every language
Now you're getting it! Every language is unsafe, and always will be. They all run on a CPU with bits and bytes. The difference between Rust and Python is a difference in degree, not in kind. The only hope a programmer has is to constrain that necessary unsafety to be as small as possible. Rust handles this with unsafe blocks - they are very rare, easily greppable and easily auditable. Python has, well, nothing. It's built on a mountain of C, the interpreter, stdlib, and most libraries. The mountain is unsafe by default, and there's no way to decompose the problem to smaller pieces. This pile of unsafe C doesn't magically become safe just because you used a scripting language to call into it. (Just like using Python to run an external C program doesn't make the C program safe)
You came into this thread wondering why you get downvoted when you talk about this? It's probably because you're applying inconsistent standards to different languages. You decided Python is "safe", and when you find out about the C underneath, you ignore it and stick your head in the sand and make excuses for the memory vulnerabilities. Meanwhile you decided Rust is "unsafe", and use any rare counterexample to discredit the practical improvement over its predecessors. I believe this is the fallacy currently on display: https://en.wikipedia.org/wiki/Nirvana_fallacy
You didn’t even read the rust article. The python problems article aren’t specifically memory violations, my point was that rust claims to be “safe” and yet all those vulnerabilities were memory safety violations. Those wouldn’t be possible in python.
I would argue your the one falling into that fallacy. You decided because Python calls C, its unsafe and so is Rust.
My issue is not with that really, but the odd double standard. Comparing calling C code to being able to directly write to memory is just odd. It’s not at all the same in practice.
It’s like memory safety is black and white when you compare rust and C, but than all grey when it’s rust and anything else. Marking things “unsafe” in rust doesn’t magically make it better, and unsafe rust code is far more common than memory violations in GC languages.
But it’s clear you have decided you are much smarter, I don’t see the point in continuing this discussion.
Nit: Django and all its required dependencies are pure Python. (It does have an optional dependency on Argon2 for password hashing, though, which uses C.)
Looks like ctypes are used in gdal/geos, core/locks, the orcale db driver, and the test runner (but just c_int??). (And I'm ignoring auth where ctypes stands for "content types")
Huh, yeah. I think geos is the only one of these that's actually calling native code not provided by the platform (it calls a GIS library written in C++). The file-locking code calls into the Win32 API to do something that on Unix is provided by the Python standard library. The Oracle driver code is working around a bug in Cygwin. And the test runner isn't doing FFI at all; it's setting up some multi-process shared memory, and the API requires that a ctypes type be used for that, since you don't want to use regular Python types that can allocate at will (and might do so in the wrong part of memory) for that purpose.
The typical Django deployment uses none of these, but on the other hand, it does use a third-party database driver written in C, so I guess the point stands.
Comments
Every language has unsafe bits if you look deep enough. CPython and Java's garbage collector are written in C, and we have ctypes and JNI, but I think most people would still consider those memory safe languages. The important question is: is there a well-defined safety boundary, somewhere between the CPU and your application code, where memory is guaranteed* not to be corrupted and the programmer is freed from worrying about low-level bugs like segfaults?
*assuming no outside bugs such as CPU bugs, bugs in the GC, or bugs in an unsafe block
This doesn’t make sense. So rust let’s you write unsafe code, and now that’s the same as any language that calls C in any way?
Why replace C++ then? After all, rust is just as unsafe by your definition.
Having a language that is default safe and can only be unsafe in rare situations is way better than one that lets libraries write unsafe code, which is better than a language with no safe boundary.
This describes both Python and Rust. They are default safe, with escape hatches (ctypes or unsafe blocks) for rare situations. Writing unsafe code in Rust should be as rare as using ctypes in Python.
Python lets libs use ctypes to poke at raw memory. Solution: avoid those libraries
Rust lets libs use unsafe blocks. Solution: avoid those libraries (and optionally use tools like cargo-geiger or forbid(unsafe) if you care enough
If you don’t seem a parallel between what you just said: “just use these tools to catch the unsafe parts of rust” and the C++ crowd saying “just use these tools or do these things to catch the unsafe parts of cpp” there’s no point in continuing this discussion.
Python ctypes are not at all as common as rust unsafe code and geiger only goes so deep in search for unsafe code if I remember correctly.
So your opinion is that safety holes are only important if the syntax used is "common"? IME unsafe Rust code in the wild is rare and there's a strong culture against its use. Meanwhile in Python, ctypes and C modules are very common -- just about all popular libs use C under the hood, numpy, scikit, django, etc. It's tough to do anything useful without it.
Does Python have any similar tooling for managing all that unsafe code? Or is the only option to hope and pray that the memory vulnerabilities are not too common?
I can’t believe we are arguing about whether python is memory safe.
Practically, we can measure memory safety vulnerabilities and they are find often in rust crates:https://www.infoq.com/news/2021/11/rudra-rust-safety/
You can say “python calls C and that makes it memory unsafe” but so does every language, including rust. It is well known GCs are memory safe, all those articles about memory safety push for GCs first, and rust second.
You can cherry pick stats like that about any language: https://www.theregister.com/2021/07/28/python_pypi_security/
Pretty bad, right? I'll take the ecosystem with 265 issues over the one with 749000 issues any day.
Now you're getting it! Every language is unsafe, and always will be. They all run on a CPU with bits and bytes. The difference between Rust and Python is a difference in degree, not in kind. The only hope a programmer has is to constrain that necessary unsafety to be as small as possible. Rust handles this with unsafe blocks - they are very rare, easily greppable and easily auditable. Python has, well, nothing. It's built on a mountain of C, the interpreter, stdlib, and most libraries. The mountain is unsafe by default, and there's no way to decompose the problem to smaller pieces. This pile of unsafe C doesn't magically become safe just because you used a scripting language to call into it. (Just like using Python to run an external C program doesn't make the C program safe)
You came into this thread wondering why you get downvoted when you talk about this? It's probably because you're applying inconsistent standards to different languages. You decided Python is "safe", and when you find out about the C underneath, you ignore it and stick your head in the sand and make excuses for the memory vulnerabilities. Meanwhile you decided Rust is "unsafe", and use any rare counterexample to discredit the practical improvement over its predecessors. I believe this is the fallacy currently on display: https://en.wikipedia.org/wiki/Nirvana_fallacy
You didn’t even read the rust article. The python problems article aren’t specifically memory violations, my point was that rust claims to be “safe” and yet all those vulnerabilities were memory safety violations. Those wouldn’t be possible in python.
I would argue your the one falling into that fallacy. You decided because Python calls C, its unsafe and so is Rust.
My issue is not with that really, but the odd double standard. Comparing calling C code to being able to directly write to memory is just odd. It’s not at all the same in practice.
It’s like memory safety is black and white when you compare rust and C, but than all grey when it’s rust and anything else. Marking things “unsafe” in rust doesn’t magically make it better, and unsafe rust code is far more common than memory violations in GC languages.
But it’s clear you have decided you are much smarter, I don’t see the point in continuing this discussion.
Nit: Django and all its required dependencies are pure Python. (It does have an optional dependency on Argon2 for password hashing, though, which uses C.)
Huh, yeah. I think geos is the only one of these that's actually calling native code not provided by the platform (it calls a GIS library written in C++). The file-locking code calls into the Win32 API to do something that on Unix is provided by the Python standard library. The Oracle driver code is working around a bug in Cygwin. And the test runner isn't doing FFI at all; it's setting up some multi-process shared memory, and the API requires that a ctypes type be used for that, since you don't want to use regular Python types that can allocate at will (and might do so in the wrong part of memory) for that purpose.
The typical Django deployment uses none of these, but on the other hand, it does use a third-party database driver written in C, so I guess the point stands.