r/Python • • Sep 09 '22

Discussion Async Python is not faster

https://calpaterson.com/async-python-is-not-faster.html
0 Upvotes

47 comments sorted by

View all comments

21

u/rouille Sep 09 '22

I think this misses the main appeal of async. It's not about speed but about the programming model. Async makes it much easier to work with and compose concurrency compared to the thread based model. Just the fact that you can cancel tasks makes it trivial to do proper timeouts, which nobody gets right in sync applications, for example.

It also allows things like structured concurrency.

3

u/rnike879 Sep 09 '22

Hey, I love your breakdown but could you explain more about the structured concurrency with async?

2

u/ManyInterests Python Discord Staff Sep 09 '22 edited Sep 09 '22

For a beautiful explanation suitable for beginners with simple examples, see Raymond Hettinger's concurrency keynote. David Beazley also has tons of talks on concurrency if you want to deep-dive.

To give you the two-minute version:

Basically, async is a cooperative model. Everybody designs their code to work together and "yield" control back to the event loop when you don't need to "be running" (like when waiting on OS calls or network for I/O -- that's the io in asyncio) so everything works in harmony. This ends up being very lighweight/efficient compared to alternatives.

By comparison, in traditional threaded models, you don't have to worry about being cooperative with your concurrent neighbors -- what gets to execute is handled mostly handled the OS thread scheduler. However, this incurs a good deal of overhead from the GIL/thread context switching for every thread. There are synchronization primitives (e.g., locks) when you need them, but they can be costly in terms of performance and those costs rise exponentially as you add more threads.

So the real benefit is not necessarily speed -- you can't crunch prime numbers faster using asyncio -- the real benefit is massive concurrency (usually in I/O-bound work, like web applications).

-5

u/benefit_of_mrkite Sep 09 '22

It’s also worth mentioning that not all operations make sense for async - file system ops are are blocking.

The main purpose of asynchronous in python is working around the GIL

4

u/alkasm github.com/alkasm Sep 09 '22 edited Sep 09 '22

The main purpose of asynchronous in python is working around the GIL

No, it isn't. The GIL isn't mentioned at all in any of the async PEPs:

Threading on a single core (a-la Python's GIL) isn't fundamentally that different from async code; with async, the programmer decides when to context switch (via await), whereas in threading, the OS decides when to context switch.

The primary motivation of async programming in Python is that it is an extremely useful, modern paradigm for concurrency, allowing for better event-driven processing than threads do. From PEP 492:

We believe that the changes proposed here will help keep Python relevant and competitive in a quickly growing area of asynchronous programming, as many other languages have adopted, or are planning to adopt, similar features

-2

u/benefit_of_mrkite Sep 10 '22

Go read the old python mailer.

I’m old.

0

u/[deleted] Sep 09 '22

[deleted]

-1

u/benefit_of_mrkite Sep 09 '22

I didn’t say it was the only reason I said it was the main reason. Don’t twist my words especially when you are quoting them.