r/Python • u/Thin-Parsley-1660 • 16d ago
Discussion Feasibility check: Writing a constrained Python-to-Rust transpiler for numeric code
so i would want to know about the feasibility of such a tool/project and whether it has any real uses or is it the equivalent of re inventing a worse wheel , basically core idea is to somehow transpile python to rust but not all of python ,instead more similar to numpy but instead the whole specified block of code is compiled for maximum speed almost like numba?? any feedback would be appreciated
9
3
3
2
1
u/teerre 16d ago
I mean, there are python to GPU kernel compilers, so obviously to Rust is also possible. Now, how useful it is it's a different question. "Numerical code" is already executed in a low level language if you use anything remotely standard, so benefits are dubious
And of course, this is without considering the elephant in the run which is just because you can write a transpiler, it doesn't mean it will be a good one
1
u/Beginning-Fruit-1397 16d ago
It's trivial to do directly in rust with pyo3 and the existing numpy bindings. Contrary to popular belief rust is not that complicated and not that far away from python idioms.
1
u/snugar_i 15d ago
What would the result of the transpilation be? An executable binary? A blob of Rust code? How would it call other libraries like numpy etc.? I'm having a hard time seeing the bigger picture here
2
u/donk8r 14d ago
the output is rust source, not a binary — the point is the numeric block gets rewritten while the rest stays python, so the artifact is python calling into rust. that's the bigger picture: you keep writing python, one block runs as compiled rust.
numpy doesn't get reimplemented. the generated rust calls the existing numpy bindings through pyo3. which is why the subset has to be declared. the hard part isn't the call, it's type inference: python lets you mix int and float in ways that run fine there and panic in rust. numba and cython narrowed the subset and made types explicit. the value is in the constraint, not the translation.
3
u/snugar_i 14d ago
So what's the workflow? I write a Python function, then your thing transpiles it into Rust, then I delete the Python version, build a wheel or three, and start calling the Rust one? That sounds quite painful...
And please don't use AI to write your response - if you can't be bothered using your own brain, I can't be bothered spending my time
1
u/donk8r 13d ago
nothing gets deleted. the function stays in your package and the compiled block would sit beside it as an extension module that same package imports, so your call sites keep calling the same name. you regenerate the rust when the block changes and you don't hand-edit it.
the wheel gets built when you ship. between releases you're still running the python path, and your tests run both and compare the outputs.
0
u/really_not_unreal 16d ago
Not feasible. Rust's borrow checker would mean that very clear object ownership would be required, which would not be doable in a reference counting language like Python. While you could wrap everything in a refcell, that would cost you a pretty significant amount of the performance that you're trying to gain from using rust.
2
-1
u/RuinWMD 16d ago
I built this.
It is an agent focussed python to rust compiler that focusses on speed and safety.
Also has its own programming language for AIs that is working very well but the best gains are from direct pyton to rust.
The native language is still being worked on and is super fun.
I have passed it over a few of my own python pipelines and it has resulted in some pretty great gains.
0
0
u/Khavel_dev 15d ago
The hardest part won't be the transpilation itself, it'll be type inference. Python's numeric code looks simple but the moment someone passes a numpy array with an unexpected dtype or mixes int and float in a way that works fine in Python but panics in Rust, you're in trouble. Numba solved this by adding type annotations via decorators and JIT compiling a subset, and Cython went a similar route. If your scope is literally "numpy-shaped arrays with explicit dtypes doing math," it might be doable as a research project. But the gap between that and anything people would trust in production is huge. I'd look at what Mojo is doing, they're basically tackling the same problem space but with a new language instead of transpilation.
19
u/me_myself_ai 16d ago
Super uninformed take, in case it's helpful:
This definitely sounds interesting and the Python AST is nice and elegant, but I'm not sure what it would win us. AFAIU the numeric libraries are already doing everything they possibly can in C directly, which is faster than Rust in the typical scenario. They're also some of the most mature libraries in the world (somehow not hyperbole?) in terms of hours invested, much of which was spent specifically on speeding things up for various workload/system combos.
Rust is safer ofc, but if you're doing something this jank/hacky to make the rust code, that feels like a silver lining at best. I suppose numeric code could be security-relevant, but again, you prolly wouldn't want autogenerated rust code for that kind of application anyway!
Hopefully someone comes in to prove me wrong and validate your idea. It would set the message boards aflame in a fascinating way, which is one undeniable upside!
EDIT: I suppose if it's transpiled it could be faster cause there's no python left at all, at that point. So I guess it could be somewhat faster?
At some point you've gotta ask yourself why you just don't write the code in rust from the jump tho, no? Especially these days?