r/Python • • Nov 13 '20

News Python swallows Java to become second-most popular programming language

https://www.theregister.com/2020/11/12/python_swallows_java_to_become/
2.3k Upvotes

205 comments sorted by

View all comments

Show parent comments

169

u/harylmu Nov 13 '20 edited Nov 13 '20

I think this quote is really not accurate because speed isn't the most important (well, usually), but a good concurrency story can save big dollars. I rewrote one of our Python app in .NET Core 5 and I wasn't able to kill it with load testing (2000 req, 200 concurrency), while the Python 3.8 app with FastAPI (literally) hanged around 25 requests per second. And to be clear: I made sure that all IO operations are async in the Python version (database queries, Consul queries, SNS publish, S3 upload - even though aioboto3 isn't too stable yet). This alone means that we're able to save thousands (millions on long term?) of dollars by running less instances.

Other than that, the speed difference between Python and typed languages is really NOT nanoseconds. I don't know what to say, test it yourself.

0

u/[deleted] Nov 13 '20

[deleted]

2

u/reddisaurus Nov 14 '20

Pandas doesn’t have C wrappers, it just wraps numpy. Pandas is slower than plain Python that just calls numpy directly, by a factor of 10 - 500 depending on the task.

Pandas provides abstraction for the sake of programmer time, but there’s a cost to that. Use a data class or NamedTuple and watch your code perform much better.

2

u/ShanSanear Nov 14 '20

Had exact same issue with overengineered script that did number crunching using pandas. Turned out that recursion combined with slowness of pandas at some tasks became big factor. We got x400 of the old speed, just by ditching pandas almost completly and dropping recursion whenever possible