r/programming • • May 14 '21

Python programming: We want to make the language twice as fast, says its creator

https://www.tectalk.co/python-programming-we-want-to-make-the-language-twice-as-fast-says-its-creator/
780 Upvotes

260 comments sorted by

View all comments

Show parent comments

9

u/latkde May 15 '21

The issue with C interop is that Python has to stick with the PyObject data structure. This can have far-reaching consequences. If ownership of a Python object is shared between optimizable Python code and a native module, this effectively means the Python code must also interact with PyObject and can't really be optimized.

This doesn't have to be an insurmountable problem though:

  • Could only optimize code where an escape analysis indicates that it's safe to do. This will do wonders on more mathy or more algorithmic code, but will be disappointing in many real-world scenarios.

  • Break compatibility and completely redo the C data model. However, this would be a regression to the early 2.x series. One of the big features of 2.6 and 3.x was that all values can be treated as normal objects.

2

u/dreugeworst May 15 '21

Couldn't you box the data again at the abi boundary if needed?

8

u/latkde May 15 '21

Kinda, yes, but my concern is that the C function is a black box so you can't make assumptions about ownership of the object afterwards:

x = create_some_object()
x.do_something()  # safe per escape analysis
call_c_function(x)  # could do ANYTHING
x.do_something()  # no safe assumptions

Calling the C function would require some kind of boxing, but the C function could retain a reference to the boxed object. So the second x.do_something() must also operate on the boxed representation, in order to propagate changes to the C code. This can be avoided only for some built-in immutable types like int or str.

Of course this isn't quite true, e.g. a JIT compiler could compile a fast path for the assumption that the object's refcount didn't change, and a fallback otherwise. And maybe it's possible to annotate at least the built-in C functions with semantics that allow for safe and fast calls.

There's also an argument that C interop is not performance-sensitive, and that performance in pure-Python codebases like Django webapps is much more relevant. As JavaScript has demonstrated, a lot of performance is possible in moderately dynamic code.