r/Python • • Nov 13 '20

News Python swallows Java to become second-most popular programming language

https://www.theregister.com/2020/11/12/python_swallows_java_to_become/
2.3k Upvotes

205 comments sorted by

View all comments

Show parent comments

169

u/harylmu Nov 13 '20 edited Nov 13 '20

I think this quote is really not accurate because speed isn't the most important (well, usually), but a good concurrency story can save big dollars. I rewrote one of our Python app in .NET Core 5 and I wasn't able to kill it with load testing (2000 req, 200 concurrency), while the Python 3.8 app with FastAPI (literally) hanged around 25 requests per second. And to be clear: I made sure that all IO operations are async in the Python version (database queries, Consul queries, SNS publish, S3 upload - even though aioboto3 isn't too stable yet). This alone means that we're able to save thousands (millions on long term?) of dollars by running less instances.

Other than that, the speed difference between Python and typed languages is really NOT nanoseconds. I don't know what to say, test it yourself.

42

u/tr14l Nov 13 '20

You wrote bad Python. The difference is NOT 2000 vs 25 unless you did something terribly wrong. There will certainly be a difference, but that was your code running that slow, not Python itself.

3

u/harylmu Nov 13 '20 edited Nov 13 '20

What it did was: decode request body with messagepack, get stuff from Cassandra, save stuff to Cassandra, get stuff from Consul, publish stuff to SNS, encrypt an object then upload it to S3. The request takes ~2 seconds to complete on both .NET and Python (.NET is a bit faster).

As you can see quite a few things waits for IO here. I wrote everything with asyncio-based libs (Cassandra, Consul, aioboto3). I don't know if I can optimize it better. Please let me know what are some concurrency-traps that I might have gone into.

Either way, even the possibility to write slow Python means something.

1

u/cant_have_a_cat Nov 23 '20

even the possibility to write slow Python means something.

oof - it's much easier to write slow low-level code than high-level code, so I have no idea what you mean by this.