r/Python • • Nov 13 '20

News Python swallows Java to become second-most popular programming language

https://www.theregister.com/2020/11/12/python_swallows_java_to_become/
2.3k Upvotes

205 comments sorted by

252

u/[deleted] Nov 13 '20

Python "swallows" java.

I see what you did there ( ͡° ͜ʖ ͡°)

90

u/[deleted] Nov 13 '20

Just a regular snake that drinks regular coffee.

14

u/[deleted] Nov 13 '20

Get because python is a snake

6

u/ChonkyXL Nov 14 '20

Yeah and Java is a coffee mug, get it? get it?

585

u/[deleted] Nov 13 '20

C is #1 according to this article, if anyone wants the TL;DR.

A quote at the end that I found interesting:

"And Python's 'slowness' is overstated. Who cares about some other language that's a few nanoseconds faster if you spend hours (or days or weeks) trying to figure out how to use it? A human's time is more valuable than a computer's."

I think this quote just about sums up the creator's POV on other languages, specifically Java. I don't think he's really comparing Python to something like C/C++ because those languages will run circles around Python with more than just "nanosecond" margins if used properly. Not to mention other languages like C/C++ are mature and are used to build projects that last for decades, even longer. I still see companies struggling to move their technologies from Python2 to Python3 due to it's end of support.

I love Python and use it often, but in certain fields people have to care about speed, performance and stability. Sometimes it's a matter of life and death. I wouldn't even trust Java for those types of tasks tbh let alone Python. It's easy to learn and use and it's fast enough to do most tasks that people need, but I still think languages like C/C++ are superior.

169

u/harylmu Nov 13 '20 edited Nov 13 '20

I think this quote is really not accurate because speed isn't the most important (well, usually), but a good concurrency story can save big dollars. I rewrote one of our Python app in .NET Core 5 and I wasn't able to kill it with load testing (2000 req, 200 concurrency), while the Python 3.8 app with FastAPI (literally) hanged around 25 requests per second. And to be clear: I made sure that all IO operations are async in the Python version (database queries, Consul queries, SNS publish, S3 upload - even though aioboto3 isn't too stable yet). This alone means that we're able to save thousands (millions on long term?) of dollars by running less instances.

Other than that, the speed difference between Python and typed languages is really NOT nanoseconds. I don't know what to say, test it yourself.

45

u/aes110 Nov 13 '20

I dont know anything about what your app did, but 25req/s hanged? are you sure there wasn't any bug there?

I rewrote my previous company's system from sync (flask) to async (sanic) and saw huge improvements, we saved ~7K dollars monthly on instances because of this.

In my current company, I'm also using sanic and my server easily handles 400 reqs per second, didnt really tried pushing more to it.

In anycase typed/compiled languages sure are much faster, but I think asyncio cand easily handle most stuff

0

u/[deleted] Nov 13 '20

[deleted]

13

u/aes110 Nov 13 '20

I prefer sanic, for the current project that I'm working on I had time to decide so I investigated both of them, for me FastAPI has good pros:

  1. Seamless integration with Pydantic and type hints

  2. Amazing documentation which is extremely beginner friendly.

  3. Seamless support for sync methods, however while this is good for beginners I dont think you should use it, when mixing sync and async code it's important to know what you are doing.

However I feel like FastAPI takes too many decisions for you so its much less flexible.

I wrote my own small sanic integration with pydantic that I carry around from my previous company as well.

Tbh I cant think of something specific to highlight as sanic's pros, it just works really well

Although the two are not different to such extremes, but in a sense you could say that sanic and FastAPI are like Flask and Django

45

u/Metalsand Nov 13 '20

While it does bear noting that RetiredMandalorian is being a bit facetious and exaggerating, and Python vs Java is mostly about "I need a quick thing" not about larger apps, that is the direction we are slowly heading into. Electron for example is widely known as inefficient and yet modern computers are powerful enough that those efficiencies don't matter as much as the development time does for simple applications.

38

u/[deleted] Nov 13 '20

It still matters. The problem with this view is that computers really haven’t gotten that much faster over the past decade and yet the software inefficiency is insane. Python can be built for larger applications too... but packaging is a nightmare (and is unlikely to ever get addressed) and the performance really does matter when it matters (which is becoming more and more important as software bloats)

14

u/cheese_is_available Nov 13 '20

It really does matter. zoom took off because it just works while skype does not. If your program is slow and there is a fast alternative with similar features, your program is basically dead even if you're Microsoft.

3

u/[deleted] Nov 14 '20

The problem with Microsoft is that they don't really care about their side products. At least not in the same way as a single focus company like Zoom Video Communication Inc does

13

u/grimonce Nov 13 '20

Speed doesn't come from type inference only. Other than that I 100% agree. Python is the main tool on our shop but I am really starting to hate it. Java really isn't all that bad as people paint it, it is fine.

Maybe thats my point of view because I actually have a degree in electronics (rf specialization), I actually don't mind C or C++ either, I just dislike to he competing cross platform dependency management systems, cmake is ugly but it gets the job done.

Pythons ecosystem is against the zen of python, where it says explicitly to try and have just one way to do something, Java does this way better that Python. Python provides infinite ways to make your project management a hell worse than npm dependency hell.

8

u/DanKveed Nov 13 '20

I recommend learning rust. A little bit of rust in your python can give massive speed gains. And the rust tooling of light years ahead of c++, in fact packaging is even better than python. Very similar to JS actually

2

u/proverbialbunny Data Scientist Nov 14 '20

A little bit of rust in your python

Like, inline Rust?

3

u/DanKveed Nov 14 '20

Possible but I was thinking of importing it as a package using pyo3

42

u/tr14l Nov 13 '20

You wrote bad Python. The difference is NOT 2000 vs 25 unless you did something terribly wrong. There will certainly be a difference, but that was your code running that slow, not Python itself.

2

u/DanKveed Nov 13 '20

It depends on use case too. A heavily multithreaded CPU intensive app can be way more than 2000vs 25 no matter how well you write it, especially when you compare it with rust/c++

2

u/tr14l Nov 13 '20

If you use a single thread running on a couple dozen cores, sure, no doubt. But why would you do that?

8

u/DanKveed Nov 13 '20

I am talking about multithreading. Python sucks ass at true CPU intensive tasks. That's why libraries like numpy are written in higher performance languages like c++. And why would you need that.... Well numpy for starts, anything control system related, AI, simulation, CAD, video editing, the use cases are Endless

2

u/tr14l Nov 13 '20

Yeah, that's an optimization for an interpreted language. I have made lots of multithreaded apps in compiled and python languages. I've never seen a 100x difference. Ever.

0

u/[deleted] Nov 13 '20

[deleted]

1

u/ShanSanear Nov 14 '20

I ran pypy on one of our inefficently written python projects (runtime of about 10hours). Pypy was 30-40% slower than python 3.8 using exactly the same code. Pypy isn't always the answer.

3

u/harylmu Nov 13 '20 edited Nov 13 '20

What it did was: decode request body with messagepack, get stuff from Cassandra, save stuff to Cassandra, get stuff from Consul, publish stuff to SNS, encrypt an object then upload it to S3. The request takes ~2 seconds to complete on both .NET and Python (.NET is a bit faster).

As you can see quite a few things waits for IO here. I wrote everything with asyncio-based libs (Cassandra, Consul, aioboto3). I don't know if I can optimize it better. Please let me know what are some concurrency-traps that I might have gone into.

Either way, even the possibility to write slow Python means something.

1

u/cant_have_a_cat Nov 23 '20

even the possibility to write slow Python means something.

oof - it's much easier to write slow low-level code than high-level code, so I have no idea what you mean by this.

4

u/pipai_ Nov 13 '20

That seems very strange. Were you running it with full gunicorn setup?

1

u/harylmu Nov 13 '20

No, I haven't used a process manager. I could try but I doubt it'd be much better.

4

u/pipai_ Nov 14 '20

It’s actually very important. Because python has a GIL, you will end up with only running a single thread. Of course the performance will suffer accordingly. You need to set it up like in production, in order to do a real comparison.

Other languages don’t find this necessary because they don’t have a GIL.

9

u/Doomphx Nov 13 '20

I'm ditching django for .Net 5 Asp.net and you're not lying. My story is similar to yours where I couldn't even possibly lag my new server with testing.

My next move is to move all my django channels over to SignalR where I can guarentee less than 100ms response times. Dev is looking good with nothing really taking more than 20ms so fingers crossed!

The performance, productivity, and ease of deployment I get with .Net has been life changing.

Git push, pull .Dotnet publish, restart server, all done :)

3

u/tomatotomato Nov 13 '20

With Azure Devops, it's just git push.

3

u/supremacyofthelaces Nov 13 '20

I don't really know any languages other than Python, and it mostly takes a while because my laptop is an old piece of junk and I always have a million things open at once. What kind of scale sre you talking here?

11

u/Aidtor Nov 13 '20

Production environments or places where you need a lot of compute.

Production servers these days are typically cloud based. Horizontal scaling (adding more machines to the pool of resources handling requests) is very easy with rented compute but the cost can really add up for a business over time.

Tasks that require heavy compute, like ML, typically use python as an interface to code written in faster languages.

4

u/--Shade-- Nov 13 '20

Python is an exellent language to call C from, but a terrible language to write C in. If you have a nice pythonic library that wraps some 'C thing' then Python shouldn't be your bottleneck. If it is, then isolate and profile (and apply the common speed up tricks where neccicary).

2

u/Aidtor Nov 13 '20

Sure if you have the time for that. When I’m prototyping a model or I’m given some crazy deadline to ship a feature I’m not going to reach for C.

4

u/--Shade-- Nov 13 '20

That's fair enough. I can't recall the last time I've voluntarily written a line of C, lol. What I was getting at is that Python already has a huge ecosystem of libraries that work as high level wrappers for things written in lower level languages. If you write decent Python (by Python standards), and leverage existing libraries, then Python (mostly) shouldn't be the bottleneck. If it is, the first step should be to take a look at potential hotspots in your Python code (not write a C library).

2

u/Aidtor Nov 14 '20

Oh my bad totally misunderstood what you were trying to say. I completely agree.

2

u/latrociny Nov 13 '20

What is your opinion on Julia? It is touted to be superior than python wrt speed and speeds comparable to C especially for ML tasks

4

u/Aidtor Nov 13 '20

My opinions are mixed.

On a personal level I deeply disagree with 1-indexing.

But my personal quibbles aside, it is superior to python. It’s faster and you can actually type things which is a huge deal for large codebases. The interop is great! The writing code like mathematical expressions is overblown.

Overall I’m bullish on Julia. I don’t think it will ever “overtake” python, but they will coexist.

I do have some concerns though for Julia’s future. And that concern is called JAX.

2

u/satireplusplus Nov 13 '20

For ML tasks you want good libraries and abstractions. Those are lacking in Julia. Execution speed of the language doesn't really matter and these benchmarks are missing the point for ML. PyTorch, Tensorflow, numpy etc. handle the matrix multiplications in a BLAS library (C/fortran) or with CUDA. Its gonna be faster than anything you can hack together on your own from scratch, no matter what language you pick.

2

u/hilomania Nov 13 '20

In python today you wouldn't solve that with threads but with AsyncIO.

1

u/harylmu Nov 13 '20

I used asyncio for all IO based operations in my Python app. I just edited my original comment.

2

u/Max_Insanity Nov 14 '20

*fewer instances

After only just barely understanding all of that, my ego needs sth. where I could correct you :P

0

u/[deleted] Nov 13 '20

[deleted]

2

u/reddisaurus Nov 14 '20

Pandas doesn’t have C wrappers, it just wraps numpy. Pandas is slower than plain Python that just calls numpy directly, by a factor of 10 - 500 depending on the task.

Pandas provides abstraction for the sake of programmer time, but there’s a cost to that. Use a data class or NamedTuple and watch your code perform much better.

2

u/ShanSanear Nov 14 '20

Had exact same issue with overengineered script that did number crunching using pandas. Turned out that recursion combined with slowness of pandas at some tasks became big factor. We got x400 of the old speed, just by ditching pandas almost completly and dropping recursion whenever possible

1

u/pytrashpandas Nov 15 '20

Hey so to preface, I’m a bit biased as a hardcore pandas user for about 6 years as well as a contributor to the core library. Pandas isn’t perfect when it comes to speed/efficiency, but in my entire experience with pandas I’ve never seen any significant and properly vectorized pandas operations be anywhere near 2-3 orders of magnitude slower than properly vectorized numpy code. And you especially couldn’t get better performance from native python collections (re: namedtuples/dataclasses). The timings you provided suggest to me that you might be iterating over your dataframes or using pandas‘ apply methods which are not the right way to use pandas. Same goes for u/ShanSanear. If I’m wrong, then I would actually be really interested in the cases you guys have seen if you wouldn’t mind sharing, either way one of us would learn something new.

1

u/ShanSanear Nov 15 '20

Yep, apply/getting rows from dataframe was the main reason for slow code in our case

1

u/pytrashpandas Nov 16 '20

Ah, yea that would be why. apply is almost always not the right way to do things in pandas. It's really more of a last ditch resort and very slow. See this quote from this blog post from one of the main pandas contributors.

You rarely want to use DataFrame.apply and almost never should use it with axis=1.

If you don't mind providing an example of some of the kind of stuff you're trying to do that was slow, I'd be happy to show how to do things in a more "pandonic" way.

1

u/ShanSanear Nov 16 '20

Actually what we did was heavy overengineering and pandas was mostly used for loading data (I know, heresy). Don't have code on me, but problem was something along the lines of this:

  1. Load multiple sets of related objects that are represented as CSV files (1 line - 1 object)

  2. Find this relation (could be any kind, including recursion of the same type)

  3. Depending on some state of the object, do the calculation

During loading, we got each Series from dataframe for each type of objects and created classes for them. Then the references. And then - the calculations. In the hindsight - yep, that was the worst of it all.

After actually doing profiling, we saw that majority of cpu time was used by pandas in many different places.

That was one thing, but even algorithm that we implemented was quite unoptimized. We started at the root of the tree, then recursively went deeper into the relations of each object to extract required numbers. Apparently going from the leafs and then up was much better approach in every case. Now we only need to figure out why the numbers differ between implementations, and that will be the hardest part.

Especially when the guy who actually wrote this (I am mostly overseeing and providing some help) is stubborn enough to not do any kind of testing.

Thanks for interest - I can imagine hearing "pandas is slow" was heresy but the again - we misused it, and that is our fault, not the library itself.

2

u/[deleted] Nov 17 '20 edited Nov 17 '20

Hey, so obviously can't give you much insight on your stuff, but one thing to keep in mind is that when using pandas, it's best to stay entirely within a pandas/numpy/numeric style ecosystem. You don't usually ever want to mix pandas objects into custom python objects. Pandas structures and operations should for the most part be standalone. You should think of solving problems in pandas more similar to how you would solve problems with sql. You normally wouldn't ever want to run a recursive style solution on a dataframe, you could probably instead use some merges to do what you want.

→ More replies (1)

1

u/reddisaurus Nov 16 '20

I’m referring to storing numpy arrays within the dataclass, or using a NamedTuple for smaller data sets instead of just reaching for pandas every time. Both of these will provide faster access than pandas.

Pandas .itertuples() is about as fast as it can get to iterate over records and it does perform well.

As for any operation which acts on the DataFrame or Series objects, yes those operations are orders slower. It’s just often the case that the operation itself isn’t repeated much, so the large overhead of accessing the Series object isn’t noticeable.

There’s no case in which Pandas is faster than the simper alternatives. And that makes sense, because it’s there to provide abstraction over column-wise data. The problem is that many users immediately reach for it for any problem. Not only is it overused, but pandas lends itself to poor code as well. The DataFrame is a black box of typing! Code cannot by statically checked like it could if one used a dataclass. It’s like porting around a mutable global state to every function to which a DataFrame is passed. Who knows what structure the object should have? It’s often not in the code at all.

1

u/pytrashpandas Nov 16 '20 edited Nov 16 '20

The problem is that many users immediately reach for it for any problem

Totally agreed. I've definitely seen it used in places where it's completely unnecessary, and honestly probably irresponsible. I will say though, that if you ever find yourself needing to use itertuples or iterrows on a dataframe then you're either using pandas very wrong, or you shouldn't be using pandas in the first place.

those operations are orders slower

I agree that pandas is slower than pure numpy in most cases, but it is no where near multiple orders of magnitude. Again if you are seeing this happening, then I can guarantee it is because pandas is not being used correctly. If you would care to provide me with an example that you think demonstrates this, I would be happy to show you how it can be done faster

There’s no case in which Pandas is faster than the simper alternatives

The thing is that in many cases there are no simpler alternatives, especially that makes ease of development worth the potential speedups you could get otherwise. Especially when it comes to working with heavily labeled timeseries data.

pandas lends itself to poor code as well...It’s like porting around a mutable global state to every function to which a DataFrame is passed.

Yes, in the same way that python and standard data structures lend themselves to poor code. If used improperly, without an understanding of the underlying concepts and when to best apply it, it can lead to a mess of a program. If used properly it is extremely powerful and easy to express concepts that would otherwise be difficult to express. The problem I think is that most people try to treat dataframes and series as a drop-in replacement for dictionaries/lists etc. and structure their python code the same way they would otherwise. This is not how you should be using pandas.

Again I am inherently biased on this topic. I heavily use both pandas and numpy and other numeric python libraries and feel very passionately about this area of python. I'd be very happy to hear any examples you have and provide counter examples that can show you the power of pandas.

Also you may (or may not :) ) enjoy the xarray library. It's a much thinner wrapper around numpy that provides similar labelling capabilities as pandas (although it's still has a lot of room for improvement).

20

u/mrsmiley32 Nov 13 '20

Hmm, I feel like someone should have already have done language benchmarks in a way that you could do a code review so that you could create a PR in case they screwed up a language setting.

Oh a quick Google presents many, here's one.

https://github.com/kostya/benchmarks

6

u/santiacq Nov 13 '20

That's interesting, it really puts into perspective how performant C/C++ is. It's also cool to see how results show that compiled languages are generally an order of magnitude faster than interpreted ones and those with garbage collector tend to be slower than those without

10

u/ChickenNuggetSmth Nov 13 '20

I'd take that benchmark with a huge grain of salt. I had a quick look at the matmul in python, and what they did was pretty bad: They reimplemented what is already efficiently written.

In python you can have dramatic performance improvements if you use the available fast libraries like numpy. They are written in a faster language like fortran, decreasing the python performance overhead by a ton while still providing the convenience of python.

4

u/satireplusplus Nov 13 '20 edited Nov 13 '20

In python you can have dramatic performance improvements if you use the available fast libraries like numpy. They are written in a faster language like fortran, decreasing the python performance overhead by a ton while still providing the convenience of python.

This is how all the Python ML stuff (numpy, pytorch, ...) uses matrix multiplications, with a BLAS library. And these libraries are crazily optimized and hand vectorized, you wont be able to compete with your own matrix mult routines in C. I bet its an order of magnitude faster than the naive O(n3) C code in the benchmark.

Its also typical that the matrix mult kernel will be optimized to register / cache size and instructions (vectors) for each processor generation, as these Intel optimization in OpenBLAS:

dgemm_kernel_16x2_haswell.S
dgemm_kernel_4x4_haswell.S
dgemm_kernel_4x8_haswell.S
dgemm_kernel_4x8_sandy.S
dgemm_kernel_6x4_piledriver.S
dgemm_kernel_8x2_bulldozer.S
dgemm_kernel_8x2_piledriver.S

Its probably also doing something better than the naive algorithm, such as Strassen or Coppersmith–Winograd.

In other words that benchmark is totally bullshit and a waste of time.

5

u/kazi1 Nov 13 '20

That's the whole point though. You're not testing Python's speed if you use numpy, you're testing C. The benchmarks would just be every language using C libraries if that was allowed.

4

u/ChickenNuggetSmth Nov 13 '20 edited Nov 13 '20

To preface this: I'm not super knowledgeable about what happens "behind the scenes" or even if numpy is considered a standard library or not.

I think libraries like numpy are the reason python is as good as it is. It is well integrated, for matrix multiplication you even have the @ operator that makes it super readable. If you benchmark python without those features, you are not representing a typical use-case. If you use python in a way it was never meant to be used it will look awful, but imo those results are pretty useless.

Edit: I'm seeing that numpy isn't part of the standard library, but the standard lib is also partially written in C. So I'm not sure where 'true' python starts.

1

u/Armaliite Nov 13 '20

The reference implementation is in C, but there are many other implementations. For example PyPy which is in the benchmarks and is also much faster than CPython.

1

u/kazi1 Nov 13 '20

Yeah a fair point. I feel like these benchmarks can get pretty "handwavy" sometimes with what's allowed and what's not haha...

Tbh I'm surprised Rust performed as badly as it did in those stats - I thought it was supposed to be as fast as C, meanwhile it's almost neck and neck with Go.

10

u/satireplusplus Nov 13 '20

don't think he's really comparing Python to something like C/C++ because those languages will run circles around Python with more than just "nanosecond" margins if used properly.

The python is slow meme is still strong. But for most really compute intensive applications, you're just using Python as a nice and fancy interface to some C, C++ or Fortran library. That's all of numpy, scipy, sklearn (machine learning) and all the deep learning stuff. Doesn't really matter if you're using C++, C or Python since the routines that are computentioanlly expensive are all in the BLAS libraries optimzied to the last nanosecond with AVX vector instructions. If you have a critical code path that is not covered by the above and you need the speed, you'd write a C/C++ extension or you use Cython.

3

u/GenericRedditor12345 Nov 13 '20

But that doesn’t conform to the circlejerk!

Seriously even for webservers python can compete with the best (starlette & uvicorn).

2

u/nuephelkystikon Nov 14 '20

You shouldn't invoke your feature-length film rendering process via bash, it parses the initial CLI arguments slower than sh.

6

u/iiMoe Nov 13 '20

Literally better comment than the article

8

u/[deleted] Nov 13 '20

Isn't Java used over C/C++ precisely because of it's stability?

9

u/utdconsq Nov 13 '20

Define 'stability'. Ive written a lot of Java, C, and C++. If you mean 'something that doesn't crash easily', Java is better than C or C++ typically because it helps you avoid out of bounds crashes and so on, but at the same time, until very recently it would blow up in your face due to NullPointerExceptions because you need to manually guard every bloody variable that isn't a primitive. Kotlin and others helped them realise that was shit, so they have changed the rules now. Meanwhile, in C, you are responsible for all checks on all things unless you use someone's library that does it for you. The standard library is super lightweight to the point of being unhelpful. C++ is better, with RAII and the STL, but due to the desire to be a superset of C, you can still shoot your foot off with raw pointers if you make the mistake of treating it like actual C instead of using the modern features of the language. For my money, JVM stuff has succeeded well because of things like Maven (love it or hate it), and nice stack traces. Never underestimate how productive you can be made if you can easily discover what in god's name broke!

8

u/x3r0x_x3n0n Nov 13 '20

We had a contest among us friends to write the best version of a breadth first search to solve a maze of 5000x5000 now i dont know about the capabilities of my peers they chose java i chose pure C. both ran it and I can tell you we gave up on the java programs and went out to have coffee. Legend has it they are still running to this day :D

0

u/[deleted] Nov 13 '20

[deleted]

11

u/panderingPenguin Nov 13 '20

All you've proved is that a language that's good for writing huge business systems isn't great for writing super tight, highly performant, implementations of search algorithms.

It doesn't even prove that. It shows that his friends probably screwed up and wrote something really slow or possibly even an infinite loop seeing as he suggests it may have never terminated.

→ More replies (7)
→ More replies (3)

2

u/midnightcom Nov 13 '20

I believe Java is type safe where C/C++ are not. It's very easy to make a critical error in C if you don't know what you're doing. However the overhead of Java makes it impractical for embedded systems over C.

0

u/[deleted] Nov 13 '20

[deleted]

3

u/panderingPenguin Nov 13 '20

As someone who writes C++ for a living, fucking loooool

3

u/x3r0x_x3n0n Nov 13 '20

yep if you know the ropes C/C++ has got to be one of the most flexible languages out there.

3

u/gradi3nt Nov 13 '20

Comparing C/C++ to python is apples to oranges. It's not like this is some competition that will end up with one universal language for giving computers instructions -- there will always be a whole toolbox full of them.

2

u/VisibleSignificance Nov 13 '20

Comparing C/C++ to python is apples to oranges

The best programming language is a smoothie made from both.

2

u/gradi3nt Nov 14 '20

Cython is great for some things, but just like all other languages it has ups and downs. There is no best programming langauge!

3

u/gromain Nov 13 '20

I'm not sure why you say C/C++ is superior. In my opinion, it's two different tools that you would use for two different purposes. None is superior to the other, just like a hammer and a screwdriver are not superior to one another.

1

u/tylercoder Nov 13 '20

How does c# speeds compare to c++?

4

u/PowershellAdept Nov 13 '20

It depends what benchmarks you look at, but a good c++ implementation is always faster than c#. If you look at tech empower c# is one of the faster JIT languages since they added span<T> and memory<T>. Benchmark games has them all over the place. Unity game engine compiles IL into cpp so take that for what its worth.

Ultimately, you can do very fast stuff with c# if you want to pay attention to data, value types vs ref types, allocation, garbage collection, cache, etc. etc. but idiomatic c# isn't any better than Java or other similar languages. In my opinion c# strength is async, maturity, and backing from a huge company. Cross platform and deployment capabilities are a plus since .net core or .net 5.

1

u/stars9r9in9the9past Py2.7 gang Nov 13 '20

I saw a post like this I think last week. Was this the same article claiming Python as 2nd most-popular but used internet searches to rank popularity? If so, it just seemed like an interesting claim as plenty of things could explain search frequency, like popularity, but also unfamiliarity (like asking what is Py?) or just looking for quick help (like searching a refresher, learning guide, or just a quick question). While that still may correlate with popularity, that just didn't seem like the best form of measurement

I love python and it is pretty popular, but I was curious of the methods used to rank the various languages

1

u/fatnote Nov 13 '20

I have spent (and continue to spend) roughly equal amounts of time learning Python vs. Java. Both have their strengths, both have gotchas and little (or big) annoyances. This matter-of-fact implication that Python is indubitably easier to use/learn than any other language is extremely misleading and unhelpful. It sets unreasonable expectations for those curious about Python and puts off those curious about other languages.

1

u/chulala168 Nov 13 '20

You forgot FORTRAN.

1

u/grismar-net Nov 14 '20

I was with you until that last remark. C/C++ are superior, if speed is your yardstick. There is no such thing as a superior language in general (although you're free to have a personal preference, of course)

1

u/pyer_eyr Nov 14 '20

Python is good for me, as business analyst/ data science guy -- I have to quickly prototype projects and then my team can assesses its viability. If it's feasible then developers can take the prototype to production with performant languages.

1

u/[deleted] Nov 14 '20

For numerical code, Python has a JIT library called Numba, which generates extremely fast code on par with C/C++.

1

u/Smallpaul Nov 13 '20

I find it weird that you think that one tool is "superior" to another. Screwdriver or hammer: which is superior?

0

u/[deleted] Nov 14 '20

[deleted]

1

u/Smallpaul Nov 14 '20 edited Nov 14 '20

When I said "superior" I was referring to raw speed, performance and stability (

as stated in my comment), which in that case, is far superior.

"Raw speed" and "performance" are basically the same thing. "Stability" is essentially a function of the programmer's skill and the environment. There is nothing more intrinsically "stable" about C than Python and given that one is prone to buffer overruns, if you absolutely had to pick one for "stability", Python would probably be the better bet.

You tell me how much better Python is at doing these tasks.

If I did, I'd be contradicting my own point, which is that its idiotic to rank tools designed for different purposes. You make lip service to understanding that its idiotic here: "I'm not here to say that C can do Python-based projects better"

But then you turn around and list a bunch of projects that are C-oriented and seem to imply that if Python can't do them better then its inferior. Which just shows that you still don't get it. You use the right tool for the job. It would be dumb to write a device driver in Python and equally dumb to write YouTube in C.

I know how important C is for the reasons I explained, apparently you do not.

I didn't say that C was not important. You're projecting. I said that C is important for different kinds of applications than Python is important for. Hard to believe that there are technology professionals who would disagree with that.

1

u/johnxreturn Nov 14 '20

I make your words mine. I love Python but implying it’s only “slightly” slowest than faster languages is a disservice to the truth. C/C++ are far superior when the subject is speed. Though the learning curve is steeper.

0

u/nlantau Nov 13 '20 edited Nov 13 '20

I'm actually taking an algorithms and data structures course right now, having about one year of embedded C programming and being a TA in fundamental programming for Python prior. I've implemented a couple of sorting algorithms in both languages (and in Java), to see the difference and I've been blown away every time. Some algorithms won't finish (within reasonable time (~< 1hrs)) in Python whilst in C I've got a runtime < 2 seconds. I've compared the implementations many times, simply to see if my implementations differ, but no. It's all about the speed of the languages.

A friend of mine was asked to run a 3x nested for loop in Java, by the teacher. He was supposed to sum++ in the innermost loop, having the iterated variable n set to 10,000 (the conditions were a bit messy, but we concluded O(N3 )). He finished in 1 m 06 sec. My C programming finished in 0.08 sec. Same hardware.

My own tests have shown such an enormous difference that it has motivated me to continue to work with both C and Python throughout the course, prioritising C by all means.

Edit: I love Python. Don't get me wrong, it's one of my absolute most useful sw tools. I do enjoy C as well, because it really tests my knowledge.

Edit 2: typo

0

u/chalbersma Nov 13 '20

Using python 3.8 or higher brings a lot of speed improvements out of the box. I think the characterization of python as slow in the 2.x days or the 3.5 and less days was accurate. But it's so much snappier now.

24

u/[deleted] Nov 13 '20

I like Python but I’m sure it’s not the right choice for every problem.

31

u/hermeticwalrus Nov 13 '20

The advantage of Python is that it’s a good enough choice for most problems, as well as being excellent for rapid prototyping.

10

u/Vionik Nov 13 '20

Like every language

2

u/[deleted] Nov 13 '20 edited Nov 14 '20

There’s always a few vocal hardcore people who seem to think their language should be used for everything. I kind of like Go but realize I wouldn’t use it for everything either. Rust also looks interesting but I’m not learning that until I find that employers in my area desire that skill

Sorry for using hyperbole here.

→ More replies (1)

0

u/Smallpaul Nov 13 '20

What language is "the right choice for every problem"?

3

u/[deleted] Nov 14 '20

Assembly

0

u/Smallpaul Nov 14 '20

Which specific Assembly language is "the right choice for every problem"?

3

u/[deleted] Nov 14 '20

It was a joke

10

u/[deleted] Nov 13 '20

The best thing about python is you can do whatever you want to do with it.

12

u/secretaliasname Nov 13 '20

Also the worst thing. I work on a lots of projects maintained by folks who are not primarily software developers. The lack of first class typing and interfaces makes python easy to learn but also much harder IMO to learn how to write maintainable code.

8

u/[deleted] Nov 13 '20

If python is gaining in popularity I'd hope it's because more people are gaining an interest in programming and using python as a front door.

2

u/proverbialbunny Data Scientist Nov 14 '20

Elementary school kids are being taught programming today. It's really a great age to learn this kind of stuff. Kids surprisingly have an easier time learning how to program than adults do.

26

u/BigTheory88 Nov 13 '20

As great as Python is, it's just not going to cut the durability of software you can write with Java.

17

u/mooburger resembles an abstract syntax tree Nov 13 '20

define "durability"? One of the aims of the Python style guide (PEP8) is enforcing readability and maintainability.

7

u/Fledgeling Nov 13 '20

Well, some of the most annoying parts of Java that make me hate the language exist to force large teams of users to use design patterns that are enforced by the language for usability.

Half the developers I talk to don't even know what pep8 is and think global variables are fine. Granted, that's fine for the work I'm doing lately.

1

u/mooburger resembles an abstract syntax tree Nov 13 '20

I have never worked on an "enterprise" Java app where there was not already an overriding style guide that the code/PR reviewers enforce to begin with, so I think documentation, the test framework and maybe some other pipeline tools (like if you want to do some type sanity checking with mypy) will get folks most of the way there. To be Pythonic is to EAFP, and the language is specifically built around that pattern; Guido's whole idea is "we're all adults here".

2

u/Fledgeling Nov 14 '20

Yup.

And the idea that we are all adults is something I have had success with in practice.... While avoiding anything too corporate feeling.

1

u/mooburger resembles an abstract syntax tree Nov 14 '20

Yeah my experience with corporate Java is that it is definitely and enabler of cargo cult programmers instead of investing in actual software engineers.

1

u/maikindofthai Nov 14 '20

When I think of "durability" WRT a programming language, the types of things covered in a style guide aren't really what comes to mind...

I think of things like the strictness/explicitness of the type system, how robust the error handling is, what the runtime and its backwards compatibiilty story looks like, what the dependency/tooling ecosystem looks like, etc. Python fares well in a few of these categories, but if you're writing a large application with a team of developers, Python makes it a whole lot easier to make a giant mess of things than something like Java.

Python's a tool like any other programming language -- it's not always going to be the best one for the job.

1

u/mooburger resembles an abstract syntax tree Nov 14 '20

The whole philosophy behind python data structures and methods is runtime typing/late binding. It's an entirely different way of thinking from compile time strong typing. In fact, other languages are starting to use it, not just the entire ECMAScript ecosystem but even .NET implemented duck typing shim in order to reduce interface and getter/setter boilerplate. With great freedom comes great responsibility, why should the language be restricting you?

0

u/CausticTitan Nov 13 '20

Resilience to scale.

12

u/x3r0x_x3n0n Nov 13 '20

Niether is gonna beat C level performance.

Python for quick and dirty stuff

C for production code

java for interviews

6

u/zeroviral Nov 13 '20

Python for interviews IMO. I code Java at work.

0

u/x3r0x_x3n0n Nov 14 '20

the reason i said it is because one can write some cyptic C and python. there is only the straight forward way to write java.

0

u/zeroviral Nov 14 '20

Yeah I guess. But I will bet money I can write an algorithm 10x faster in python than in Java. Also, easier to read.

What do you mean “cryptic”? I’ve passed interviews for G, FB, Amazon and Bloomberg using python. And I’ve never heard “cryptic” thrown around.

→ More replies (3)

1

u/BigTheory88 Nov 14 '20

Depends on the use case really

1

u/x3r0x_x3n0n Nov 14 '20

Describe the java features which make it robust.

0

u/[deleted] Nov 14 '20

[deleted]

1

u/x3r0x_x3n0n Nov 14 '20

Thats a good one. But wont that impose restrictions.

0

u/[deleted] Nov 14 '20

[deleted]

1

u/x3r0x_x3n0n Nov 14 '20

What you sayin is absolutely true. But i believe programmers should know what they are doing so i like more of the C paradigm of things but thats just me tho.

0

u/rzet Nov 13 '20

We need to go deeper... Died while reading Assembly

1

u/x3r0x_x3n0n Nov 14 '20

Amen. the world would loose color without x86.

6

u/khfung11 Nov 13 '20

Of course, the most easy to learn and more importantly, readable language.

5

u/JustAddingToThePile Nov 13 '20

And Visual Basic (number 6, 4.01% rating) is still holding its lead over JavaScript (number 7, 2.03% rating).

I think their methodology might need a little tweaking.

TIOBE index

5

u/recursiveG Nov 13 '20

There's one of these posts almost daily now. The reason is because data scientists have to do more searches on "how to do X in python" than professional developers. All these extra searches and additional stack overflow questions are counted.

4

u/MaaiKaLaal Nov 13 '20

This code block picture doesn't make any sense. Especially the last line.

7

u/2readitol Nov 13 '20

The code on the picture is not that good to show off 🤓

6

u/samdof Nov 13 '20

But it still doesn't run on a 3 quadrillion devices, so there's that...

2

u/Fledgeling Nov 13 '20

Are you talking about tiny embedded systems?

2

u/Mr_Memes_no_is_here Nov 13 '20

Very nice title

2

u/demdillypickles Nov 13 '20

Woah, again?

2

u/daravenrk Nov 13 '20

Python is a c wrapper now days. So yeah number 2 as I hate writing c if I can write python.

2

u/a_cuppa_java Nov 14 '20

Ah yes, the proverbial Python hath swallowed me

4

u/[deleted] Nov 13 '20

Whats the use of placing a language before another. Python is easy to learn and most of the beginner fall for it. It has also huge popularity on stackoverflow because most of them are novices and they find it overwhelming by looking python popularity and robust community. But popularity doesn't mean shit for industry usage. Java is a battle tested language and enterprises still stick to it despite jokes on the internet.

2

u/Isvara Nov 14 '20

Whats the use of placing a language before another.

There isn't any. The relevant questions are:

  • Is it productive to use it?

  • Can I easily hire people who can use it?

  • Is it well documented?

  • Is it maintained?

  • Does it have actively developed, well-documented libraries that do things I need?

  • Is there an active community I can go to for help?

The specific position doesn't answer those questions, but the fact that it's in the top n suggests answers. But popularity can be misleading, like JavaScript's increased popularity due to its inclusion in browsers.

1

u/b10y Nov 13 '20

I believe the most of Python community in Reddit has never implemented a production system where more than 5 developers (senior junior mixed) worked on. I would have cut myself if I had encountered a production bug in Python that would have been a compile error in a statically typed language.

2

u/soulstuff_ Nov 13 '20

why does the title sounds so dirty?

2

u/Hammar_Morty Nov 13 '20

I like python way way more than Java but i don't use it as much now that I am a better programmer. My biggest complaint is how much bad legacy code I've had to deal with that has zero styling, poor documentation or validation. I use bash for my quick or system scripts, golang for my CLI tools and servers, I use python for machine learning training training, quick data analysis and plotting.

3

u/1842 Nov 13 '20

Bad legacy code exists wherever programming languages exist.

I've worked with some horrible Java, PHP, and JavaScript code. Legacy code isn't necessarily worse in any particular language, but in less strict languages, legacy code can get weird.

2

u/[deleted] Nov 13 '20

fuck java,

2

u/Practical_Ear_4307 Nov 13 '20

Thanks for the news but Python not swallow Java yet :)

13

u/thanieel Nov 13 '20

Stage 1. Denial

1

u/[deleted] Nov 13 '20

Some good news during the pandemic LOL

1

u/DamiansBigBrain Nov 13 '20

More like python d33pthroats Java. Lmao

1

u/[deleted] Nov 13 '20

How do they know that? How it's measured?

1

u/daggerdude42 Nov 14 '20

Wow this isn't news at all. Been out for a few weeks now actually

1

u/0dte Nov 14 '20

Hm, I enjoy Java because I learned it after Python and it forced me to think in advance about things rather than facerolling through projects.

1

u/[deleted] Nov 14 '20

You could say it “eclipsed” it

1

u/Diogonni Nov 14 '20

I recommend this programming language to beginners. Start with one of the easiest ones and then later on when you get better, you can tackle the harder languages.

1

u/Poddster Nov 16 '20

TIOBE has never been accurate.

0

u/The_DashPanda Nov 13 '20

Get it guys?

Cause Pythons are snakes!

And snakes are animals!

And animals swallow things!

And Java is coffee!

And coffee is food!

And food gets swallowed!

And this programming language named after an animal is literally SWALLOWING a programming language named after a food!

HAHA! Aren't journalists so clever??!?!!?!

0

u/zeroviral Nov 13 '20

When python takes over FinTech I’ll be a believer

-1

u/[deleted] Nov 13 '20

Sad

0

u/[deleted] Nov 13 '20

[deleted]

20

u/zeroviral Nov 13 '20

Took the words out of my mouth. I love python, but Java jobs are much higher paying consistently. I still use python for whiteboard FAANG interviews tho if I ever company hop.

6

u/[deleted] Nov 13 '20

I’ll probably get a lot of shit for this, but I don’t think that Python is that great for web development. I love Python and use it for nearly everything, but I used to program in C# and have written some decently large web APIs. Python (Flask or Django) is great for small APIs, but C# was so much better when it came to managing more complex types and structures of large web applications.

Most of my programming now is for scientific/numerical applications. Areas of my code that need high performance can easy be written in C extensions and even sometimes just wrapped with numba. So far, the web APIs are the only real downside I’ve experienced in Python.

1

u/axonxorz pip'ing aint easy, especially on windows Nov 13 '20

Python (Flask or Django) is great for small APIs, but C# was so much better when it came to managing more complex types and structures of large web applications.

I'm largely in this boat, and I like it, it's not going to change anytime soon. I've got a bunch of Pyramid, one Django and a few Flask applications that I work on regularly. The codebases are getting too large and the flexibility of Python is definitely starting to make things more difficult. With everything moving to the frontend for better web-app experiences, I'm hoping things like FastAPI and the strong-typing that it encourages makes this a little easier.

Now, to get all these projects off 2.7..............

1

u/proverbialbunny Data Scientist Nov 14 '20

Custom mobile app development on iOS and Android was not Python’s strong suit in the past— but things are changing rapidly. In the early times, Python did not have a strong story when it came to writing GUI for mobile apps. In fact, iOS and Android mobile app development were pretty much out of the question with only Python.

https://dzone.com/articles/why-python-has-become-a-popular-choice-for-mobile

Typically what you want to do is have Python run on the backend and then use another language for the frontend. However, this is not necessary as shown in the article.

0

u/[deleted] Nov 14 '20

[deleted]

1

u/proverbialbunny Data Scientist Nov 14 '20

Guido isn't tied to Python any more, just so you know.

Something to keep in mind is programming language rankings are determined by web searches. Python is a pain in the butt language in that the non-standardness of its libraries combined with it's not-so-great api documentation requires tons of googling around to figure out how to do anything. In comparison, languages like C are easy to learn inside and out so you rarely have to google anything anymore.

Python's ranking is totally inflated, but that ofc doesn't understate the popularity of the language.

0

u/matrinox Nov 13 '20

Mhm isn’t the interest limited to just machine learning / data science? Outside of that, it wouldn’t even break top 20

0

u/shinitakunai Nov 13 '20

You are wrong at the large desktop applications statement. I build them, sure it is PySide2 (bindings for C++ QT framework) but I do all my work on python.

0

u/[deleted] Nov 13 '20

[deleted]

18

u/gradi3nt Nov 13 '20

Almost any job can be a python developer job -- just start automating your work with python!

12

u/crazedizzled Nov 13 '20

Customer: fix my wordpress plugin

1

u/rzet Nov 13 '20

I speak python, landed job in new environment... End up doing endless analysis of why our shit java software crash under little stress, please help and run your great scripts again with this extra setup.. or simply do manual tests because other qa left.

0

u/[deleted] Nov 13 '20

[deleted]

114

u/[deleted] Nov 13 '20

Better performance?¯_(ツ)_/¯

20

u/gregy521 Nov 13 '20

You can swap CPython with pypy for an average 4x speedup. You can also offload to more efficient libraries like numpy for data crunching, or use Numba to JIT compile to machine code.

1

u/[deleted] Nov 13 '20

Sure but for the love of god it would be nice to be able to use a for loop instead of having to do vectorialization all the time. And I say this knowing how to vectorialize a ton of stuff. But it is just strange

1

u/Armaliite Nov 13 '20

Numba speeds up the loops though

25

u/JoelMahon Nov 13 '20

Just need to write something to turn python code into optimised c++ then, ez pz

Tbh, a human can do that part usually quite quickly, use python to invent, c++ for real time critical deliverables after python proves the concept

21

u/Darksteel213 Nov 13 '20

Cython exists, but then you have to specifically write your code to be compiled near 100% to C

10

u/ItsOkILoveYouMYbb Nov 13 '20

I could imagine indie game development taking off even more so than it already has if Python was usable with Unity and Unreal Engine.

Maybe we'll get a speed-critical but pleasant coding language like Python to make modern games with some day.

14

u/jabela Nov 13 '20

Godot has a Python like scripting system. It's definitely getting there.

1

u/shinitakunai Nov 13 '20

I tried godot and uninstalled it 5 days later because it is not really like python. It is similar but the differences are annoying

7

u/gregy521 Nov 13 '20

I believe that's the niche that Rust is trying to fill. Consistently ranked near the top in developer surveys, and runs as fast (or faster) as C/C++ in many cases.

Certainly not 'pleasant' in the way that python is, where it's like writing pseudocode and it has lots of high level features to make life easier, but much more pleasant than C/C++ I would argue.

2

u/SlappinThatBass Nov 13 '20

With pyo3, unlimited power!!! In the future

1

u/VisibleSignificance Nov 13 '20

where it's like writing pseudocode

related

5

u/xigoi Nov 13 '20

You can't do that because of eval() and friends.

1

u/VisibleSignificance Nov 13 '20

because of eval() and friends

You shouldn't really use those in runtime either.

And if it's at import-time in python (e.g. attrs), it should map into compile-time nicely.

1

u/asbox Nov 13 '20

Exactly that!..don't get why people don't understand how much efficient it is to work this way instead of other freaking languages complications.. Cpp + python is all you need.

6

u/PooPooDooDoo Nov 13 '20

Honestly, if I am going for performance at scale, I wouldn’t chose python. If I am write code that interfaces with other systems that do the heavy lifting, then I am 100% using python.

But yeah, it would be nice.

4

u/jebk Nov 13 '20

As an advanced amateur python user I think most people looking at python Vs say go is the ease of concurrency for stuff.

I imagine what most people searching 'how to make python faster' are lookjng for is 'how to make these 10 huge datasets load faster'.

Multi threading/async stuff isn't as obvious as in something like go (or even node).

1

u/f16f4 Nov 14 '20

Python is terrible for multithreading. It only actually works on one thread at a time because of the GIL.

-1

u/daravenrk Nov 13 '20

But the backend is mostly c. It’s pretty fast.

4

u/[deleted] Nov 13 '20

Oh sweet summer child...

1

u/daravenrk Nov 23 '20

Rofl. I know like saying a train makes good time.

0

u/[deleted] Nov 13 '20

[deleted]

10

u/gregy521 Nov 13 '20

But if you're running that app 1000 times a day, that sums to 30 and 200 seconds respectively. A much more appreciable difference.

Python is an extremely popular scientific computing language, too, and those don't care about 'every nanosecond', more the fact that cutting the runtime from 4 hours to 20 minutes is worthwhile.

3

u/psharpep Nov 13 '20

Sure, but if you'rr doing scientific computing in Python, you're using NumPy/Numba or similar, which gives runtimes on the order of C.

1

u/xigoi Nov 13 '20

If it takes 0.2 seconds, you can't serve more than 5 requests per second, which may be limiting.

3

u/mooburger resembles an abstract syntax tree Nov 13 '20

the real problem is that you're asking the app itself to do too much. Just about every production environment uses loadbalancers already. Why are we rehashing the same old arguments people made about RoR when it's obvious that the ecosystem & community support is what makes or breaks things?

1

u/SilkTouchm Nov 14 '20

Trading bots are bottlenecked by your latency towards the exchange's server. It's not really a cpu intensive task.

Also you might be exaggerating a bit with the nanoseconds. I don't think there's any webserver that can handle 1 billion requests per second.

2

u/Kranke Nov 13 '20

Easier deployment