r/Python • • Nov 13 '20

News Python swallows Java to become second-most popular programming language

https://www.theregister.com/2020/11/12/python_swallows_java_to_become/
2.3k Upvotes

205 comments sorted by

View all comments

585

u/[deleted] Nov 13 '20

C is #1 according to this article, if anyone wants the TL;DR.

A quote at the end that I found interesting:

"And Python's 'slowness' is overstated. Who cares about some other language that's a few nanoseconds faster if you spend hours (or days or weeks) trying to figure out how to use it? A human's time is more valuable than a computer's."

I think this quote just about sums up the creator's POV on other languages, specifically Java. I don't think he's really comparing Python to something like C/C++ because those languages will run circles around Python with more than just "nanosecond" margins if used properly. Not to mention other languages like C/C++ are mature and are used to build projects that last for decades, even longer. I still see companies struggling to move their technologies from Python2 to Python3 due to it's end of support.

I love Python and use it often, but in certain fields people have to care about speed, performance and stability. Sometimes it's a matter of life and death. I wouldn't even trust Java for those types of tasks tbh let alone Python. It's easy to learn and use and it's fast enough to do most tasks that people need, but I still think languages like C/C++ are superior.

169

u/harylmu Nov 13 '20 edited Nov 13 '20

I think this quote is really not accurate because speed isn't the most important (well, usually), but a good concurrency story can save big dollars. I rewrote one of our Python app in .NET Core 5 and I wasn't able to kill it with load testing (2000 req, 200 concurrency), while the Python 3.8 app with FastAPI (literally) hanged around 25 requests per second. And to be clear: I made sure that all IO operations are async in the Python version (database queries, Consul queries, SNS publish, S3 upload - even though aioboto3 isn't too stable yet). This alone means that we're able to save thousands (millions on long term?) of dollars by running less instances.

Other than that, the speed difference between Python and typed languages is really NOT nanoseconds. I don't know what to say, test it yourself.

45

u/aes110 Nov 13 '20

I dont know anything about what your app did, but 25req/s hanged? are you sure there wasn't any bug there?

I rewrote my previous company's system from sync (flask) to async (sanic) and saw huge improvements, we saved ~7K dollars monthly on instances because of this.

In my current company, I'm also using sanic and my server easily handles 400 reqs per second, didnt really tried pushing more to it.

In anycase typed/compiled languages sure are much faster, but I think asyncio cand easily handle most stuff

0

u/[deleted] Nov 13 '20

[deleted]

13

u/aes110 Nov 13 '20

I prefer sanic, for the current project that I'm working on I had time to decide so I investigated both of them, for me FastAPI has good pros:

  1. Seamless integration with Pydantic and type hints

  2. Amazing documentation which is extremely beginner friendly.

  3. Seamless support for sync methods, however while this is good for beginners I dont think you should use it, when mixing sync and async code it's important to know what you are doing.

However I feel like FastAPI takes too many decisions for you so its much less flexible.

I wrote my own small sanic integration with pydantic that I carry around from my previous company as well.

Tbh I cant think of something specific to highlight as sanic's pros, it just works really well

Although the two are not different to such extremes, but in a sense you could say that sanic and FastAPI are like Flask and Django

45

u/Metalsand Nov 13 '20

While it does bear noting that RetiredMandalorian is being a bit facetious and exaggerating, and Python vs Java is mostly about "I need a quick thing" not about larger apps, that is the direction we are slowly heading into. Electron for example is widely known as inefficient and yet modern computers are powerful enough that those efficiencies don't matter as much as the development time does for simple applications.

38

u/[deleted] Nov 13 '20

It still matters. The problem with this view is that computers really haven’t gotten that much faster over the past decade and yet the software inefficiency is insane. Python can be built for larger applications too... but packaging is a nightmare (and is unlikely to ever get addressed) and the performance really does matter when it matters (which is becoming more and more important as software bloats)

14

u/cheese_is_available Nov 13 '20

It really does matter. zoom took off because it just works while skype does not. If your program is slow and there is a fast alternative with similar features, your program is basically dead even if you're Microsoft.

→ More replies (1)

13

u/grimonce Nov 13 '20

Speed doesn't come from type inference only. Other than that I 100% agree. Python is the main tool on our shop but I am really starting to hate it. Java really isn't all that bad as people paint it, it is fine.

Maybe thats my point of view because I actually have a degree in electronics (rf specialization), I actually don't mind C or C++ either, I just dislike to he competing cross platform dependency management systems, cmake is ugly but it gets the job done.

Pythons ecosystem is against the zen of python, where it says explicitly to try and have just one way to do something, Java does this way better that Python. Python provides infinite ways to make your project management a hell worse than npm dependency hell.

8

u/DanKveed Nov 13 '20

I recommend learning rust. A little bit of rust in your python can give massive speed gains. And the rust tooling of light years ahead of c++, in fact packaging is even better than python. Very similar to JS actually

→ More replies (2)

0

u/[deleted] Nov 13 '20

[deleted]

→ More replies (10)

42

u/tr14l Nov 13 '20

You wrote bad Python. The difference is NOT 2000 vs 25 unless you did something terribly wrong. There will certainly be a difference, but that was your code running that slow, not Python itself.

→ More replies (9)

4

u/pipai_ Nov 13 '20

That seems very strange. Were you running it with full gunicorn setup?

→ More replies (2)

9

u/Doomphx Nov 13 '20

I'm ditching django for .Net 5 Asp.net and you're not lying. My story is similar to yours where I couldn't even possibly lag my new server with testing.

My next move is to move all my django channels over to SignalR where I can guarentee less than 100ms response times. Dev is looking good with nothing really taking more than 20ms so fingers crossed!

The performance, productivity, and ease of deployment I get with .Net has been life changing.

Git push, pull .Dotnet publish, restart server, all done :)

→ More replies (1)
→ More replies (12)

20

u/mrsmiley32 Nov 13 '20

Hmm, I feel like someone should have already have done language benchmarks in a way that you could do a code review so that you could create a PR in case they screwed up a language setting.

Oh a quick Google presents many, here's one.

https://github.com/kostya/benchmarks

6

u/santiacq Nov 13 '20

That's interesting, it really puts into perspective how performant C/C++ is. It's also cool to see how results show that compiled languages are generally an order of magnitude faster than interpreted ones and those with garbage collector tend to be slower than those without

→ More replies (1)

10

u/ChickenNuggetSmth Nov 13 '20

I'd take that benchmark with a huge grain of salt. I had a quick look at the matmul in python, and what they did was pretty bad: They reimplemented what is already efficiently written.

In python you can have dramatic performance improvements if you use the available fast libraries like numpy. They are written in a faster language like fortran, decreasing the python performance overhead by a ton while still providing the convenience of python.

4

u/satireplusplus Nov 13 '20 edited Nov 13 '20

In python you can have dramatic performance improvements if you use the available fast libraries like numpy. They are written in a faster language like fortran, decreasing the python performance overhead by a ton while still providing the convenience of python.

This is how all the Python ML stuff (numpy, pytorch, ...) uses matrix multiplications, with a BLAS library. And these libraries are crazily optimized and hand vectorized, you wont be able to compete with your own matrix mult routines in C. I bet its an order of magnitude faster than the naive O(n3) C code in the benchmark.

Its also typical that the matrix mult kernel will be optimized to register / cache size and instructions (vectors) for each processor generation, as these Intel optimization in OpenBLAS:

dgemm_kernel_16x2_haswell.S
dgemm_kernel_4x4_haswell.S
dgemm_kernel_4x8_haswell.S
dgemm_kernel_4x8_sandy.S
dgemm_kernel_6x4_piledriver.S
dgemm_kernel_8x2_bulldozer.S
dgemm_kernel_8x2_piledriver.S

Its probably also doing something better than the naive algorithm, such as Strassen or Coppersmith–Winograd.

In other words that benchmark is totally bullshit and a waste of time.

→ More replies (4)

10

u/satireplusplus Nov 13 '20

don't think he's really comparing Python to something like C/C++ because those languages will run circles around Python with more than just "nanosecond" margins if used properly.

The python is slow meme is still strong. But for most really compute intensive applications, you're just using Python as a nice and fancy interface to some C, C++ or Fortran library. That's all of numpy, scipy, sklearn (machine learning) and all the deep learning stuff. Doesn't really matter if you're using C++, C or Python since the routines that are computentioanlly expensive are all in the BLAS libraries optimzied to the last nanosecond with AVX vector instructions. If you have a critical code path that is not covered by the above and you need the speed, you'd write a C/C++ extension or you use Cython.

→ More replies (2)

6

u/iiMoe Nov 13 '20

Literally better comment than the article

8

u/[deleted] Nov 13 '20

Isn't Java used over C/C++ precisely because of it's stability?

9

u/utdconsq Nov 13 '20

Define 'stability'. Ive written a lot of Java, C, and C++. If you mean 'something that doesn't crash easily', Java is better than C or C++ typically because it helps you avoid out of bounds crashes and so on, but at the same time, until very recently it would blow up in your face due to NullPointerExceptions because you need to manually guard every bloody variable that isn't a primitive. Kotlin and others helped them realise that was shit, so they have changed the rules now. Meanwhile, in C, you are responsible for all checks on all things unless you use someone's library that does it for you. The standard library is super lightweight to the point of being unhelpful. C++ is better, with RAII and the STL, but due to the desire to be a superset of C, you can still shoot your foot off with raw pointers if you make the mistake of treating it like actual C instead of using the modern features of the language. For my money, JVM stuff has succeeded well because of things like Maven (love it or hate it), and nice stack traces. Never underestimate how productive you can be made if you can easily discover what in god's name broke!

→ More replies (26)

3

u/gradi3nt Nov 13 '20

Comparing C/C++ to python is apples to oranges. It's not like this is some competition that will end up with one universal language for giving computers instructions -- there will always be a whole toolbox full of them.

→ More replies (2)

3

u/gromain Nov 13 '20

I'm not sure why you say C/C++ is superior. In my opinion, it's two different tools that you would use for two different purposes. None is superior to the other, just like a hammer and a screwdriver are not superior to one another.

→ More replies (16)