Lots of Spark workloads are executed with the C++ Photon engine on the Databricks platform, so we ironically have partially moved back to C++. Disclosure: I work for Databricks.
The continued use of C++ is not exactly something to be proud of, although in this case at least it presumably is for short-running jobs, not for long-running services that accumulate leaks.
There is a ton of reliable load-bearing software out there written in C++. I don't think the fact that a piece of software is written in C++ is enough to presume that it has memory leaks.
Comments
Many companies have 100k+ of lines of Spark code. It's not trivial to rewrite all of that in another query framework.
Following that logic, we should have stuck with C/C++ for everything. /s
Lots of Spark workloads are executed with the C++ Photon engine on the Databricks platform, so we ironically have partially moved back to C++. Disclosure: I work for Databricks.
The continued use of C++ is not exactly something to be proud of, although in this case at least it presumably is for short-running jobs, not for long-running services that accumulate leaks.
There is a ton of reliable load-bearing software out there written in C++. I don't think the fact that a piece of software is written in C++ is enough to presume that it has memory leaks.
Python would be just another PHP level language if it wasn't for C++.
It's what powers all of the DE/ML/AI libraries.