r/Python • u/higherorderbebop • Sep 09 '22
Discussion Async Python is not faster
https://calpaterson.com/async-python-is-not-faster.html13
u/ominous_anonymous Sep 09 '22 edited 10d ago
The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.
Coherent kettle retire thought busy basket ink engine
4
u/higherorderbebop Sep 09 '22
This is not my article. Found the outcome surprising and wanted to know the community's thoughts on this.
Thanks for sharing these articles.
101
Sep 09 '22 edited Sep 09 '22
The issue is people don't understand how to properly compare the difference.
You CANNOT use a database that is on the same machine as your server. Because that dramatically reduces the latency for DB requests which then requires far less threads to saturate the CPU. The Asyncs performance advantage is entirely from requiring less OS bound thread switching.
The slower and slower your awaiting io time is the faster and faster async will perform relative to sync.
EDIT: Upon reading the code the async side is written incorrectly and has multiple bugs including but not limited to useless async calls clogging the task queue, non locked connection pool creation, and compares multiple independent json to bytes creation methods that have very different runtime.
Don't write Async code like you write sync code thread safety vs sync saftey has fully independent characteristics. Async has its own locking primitives that need to be used properly.
4
u/spoonman59 Sep 09 '22
That sounds like a broad brush.
For small data sets, all the database blocks will likely be cached anyway. Who has a large dataset and collocates the DB and the server?
async can provide overlapping of compute and I/O for situations where the I/o and compute are Mixed, that is neither one dominates the runtime.
It is true that async also reduces context switch and memory overhead of threads, but that is not the only (or entire, as you said) potential performance benefit of async.
Some programs will complete in less time using async than blocking operations, depending on workload.
7
Sep 09 '22
Not really sure what you care about the database blocks being cached for unless you are referring to local caching. The point was round tripping time has a pretty heavy cost. That cost is being hidden by the db being on the same physical machine.
I wasn't talking about all workloads I was talking about the web server workloads being discussed. Async isn't free. So even shaving off a 10th of a milliseconds of response time makes a dramatic difference. On 5000 requests per second that's half a second of dead time that threads need to make up.
Also reading the actual code there is some serious issues with it. They are getting their pool in every single async request asynchronously. Which doubles the number of events on the loop for no apparent reason. It's pretty easy to screw up async code if you don't understand how it works.
2
u/spoonman59 Sep 09 '22
I did misunderstand your point… my mistake.
I see what you mean after clarifying and I agree. There are definitely some issues with how the test was conducted. You make an excellent point that by hiding the latency and hosting the DB locally we are not representing a realistic workload, and it taints all of the conclusions as well.
ETA: not just not a realistic workload, but one which specifically portrays async worse than it normally would be.
10
Sep 09 '22
I also just realized he didn't use a lock on the pool creation. So there is actually a bug in every async program that there is going to be multiple pools created.
Don't write async code badly is a strong word of advice I have for everyone.
2
21
u/AlexMTBDude Sep 09 '22
The author of this piece doesn't understand the purpose of async. It's not supposed to be faster. It's supposed to be concurrent. Threads should not block each other.
21
u/rouille Sep 09 '22
I think this misses the main appeal of async. It's not about speed but about the programming model. Async makes it much easier to work with and compose concurrency compared to the thread based model. Just the fact that you can cancel tasks makes it trivial to do proper timeouts, which nobody gets right in sync applications, for example.
It also allows things like structured concurrency.
3
u/rnike879 Sep 09 '22
Hey, I love your breakdown but could you explain more about the structured concurrency with async?
2
u/ManyInterests Python Discord Staff Sep 09 '22 edited Sep 09 '22
For a beautiful explanation suitable for beginners with simple examples, see Raymond Hettinger's concurrency keynote. David Beazley also has tons of talks on concurrency if you want to deep-dive.
To give you the two-minute version:
Basically, async is a cooperative model. Everybody designs their code to work together and "yield" control back to the event loop when you don't need to "be running" (like when waiting on OS calls or network for I/O -- that's the
ioinasyncio) so everything works in harmony. This ends up being very lighweight/efficient compared to alternatives.By comparison, in traditional threaded models, you don't have to worry about being cooperative with your concurrent neighbors -- what gets to execute is handled mostly handled the OS thread scheduler. However, this incurs a good deal of overhead from the GIL/thread context switching for every thread. There are synchronization primitives (e.g., locks) when you need them, but they can be costly in terms of performance and those costs rise exponentially as you add more threads.
So the real benefit is not necessarily speed -- you can't crunch prime numbers faster using asyncio -- the real benefit is massive concurrency (usually in I/O-bound work, like web applications).
-5
u/benefit_of_mrkite Sep 09 '22
It’s also worth mentioning that not all operations make sense for async - file system ops are are blocking.
The main purpose of asynchronous in python is working around the GIL
4
u/alkasm github.com/alkasm Sep 09 '22 edited Sep 09 '22
The main purpose of asynchronous in python is working around the GIL
No, it isn't. The GIL isn't mentioned at all in any of the async PEPs:
Threading on a single core (a-la Python's GIL) isn't fundamentally that different from async code; with async, the programmer decides when to context switch (via
await), whereas in threading, the OS decides when to context switch.The primary motivation of async programming in Python is that it is an extremely useful, modern paradigm for concurrency, allowing for better event-driven processing than threads do. From PEP 492:
We believe that the changes proposed here will help keep Python relevant and competitive in a quickly growing area of asynchronous programming, as many other languages have adopted, or are planning to adopt, similar features
-2
0
Sep 09 '22
[deleted]
-1
u/benefit_of_mrkite Sep 09 '22
I didn’t say it was the only reason I said it was the main reason. Don’t twist my words especially when you are quoting them.
13
u/Salfiiii Sep 09 '22 edited Sep 10 '22
Your title is misleading.
Async might not be overwhelmingly faster - or even faster at all - for web frameworks, but definetly has use cases where it shines.
If you have to do a lot of isolated REST calls which you can prepare beforehand, you get major benefits from using async(io).
I’ve reduced a scripts runtime from 30 to 3 minutes by using concurrent, non blocking requests with asyncio. I even used a semaphore to only allow 10 concurrent requests, so there’s still potential left.
It added a little complexity though, but nothing to fancy if one read the python book about asyncio, which I can recommend.
6
u/spoonman59 Sep 09 '22
I can’t get past the title. Faster than what?
Obviously async is father than non-async for certain workloads. If a program doing something completes in less time, it is “faster.”
There’s nuance to the difference between concurrent and parallel, and situations where concurrency without parallelism provides no speed up (e.g., compute heavy async), but such an obviously click-bait headline is off putting.
ETA: If the claim is that async isn’t faster than threads, or something else specific, sure. But not faster than anything…?
-1
Sep 09 '22
That’s not how async works, if you’re waiting for io then the io is the bottleneck. Async allows you to get more out of your hardware for io bound tasks
6
u/spoonman59 Sep 09 '22
Ah, no, that’s not correct.
Imagine you read data for 100 ms and then do compute for 100 ms. In other words, for each 100ms of data read there is 100 ms to compute.
Imagine you process two blocks.
With blocking, you read for 100 ms. Then you compute for 100 ms. Next you read for 100 ms. Then Compute for 100 ms. Wall clock time? 400 ms.
Now async:
You read 100 ms. You start the compute process, which returns immediately. You then issue the read the next chunk. Python releases the GIL while it does the read. Finally when the first async call is done, the second one can begin as soon as the second read completes.
This example completes in 300 ms. So it is faster than non-async.
Now you are overlapping I/o and compute. The second example will complete in 300 ms instead of 400ms for the first.
Python release the GIL for I/o. So the threads give the exact same speed up as async.
ETA: Your statement is correct when the program execution time is entirely dominated by the i/o.
Async does not help performance when your application is almost all compute or almost all I/o. However, for a more balanced mix, it will allow progress to be made when a blocking program cannot make progress.
This is the difference between concurrency and parallelism.
-1
Sep 09 '22
You don’t need async to do that, you can use threads and may even be able to speed things up if you fork out sub processes because you can do the computations in parallel
4
u/spoonman59 Sep 09 '22
Your original claim was that’s not how async works and that async can only help you get more out of your hardware for I/o bound tasks.
I have disproven this claim trough a simple example presented in introductory operating system classes.
Operating systems like Linux, and others, have had async I/o operations for years. The concept is the same even if the operating mechanisms are different. It’s not something unique or specific to python. It’s a different model of concurrency from the threading model.
Now you are moving the goal posts. If you want to debate whether it is the right design to use for the task at hand, that’s a separate issue. But what you originally stated, that that is not how async works, is incorrect. It works as I described… unless you have evidence to the contrary?
4
2
Sep 09 '22
Async should not be faster or slower than sync, async allows you to get a better bang for your buck when it comes to hardware costs. Think of a web socket sever like a chat service, you may have a bunch of connections that are mostly idle. Here you can use async to sleep on those sockets until a message arrives.
2
u/dcbrown73 Sep 09 '22
lol at some of the crazy excuses being made about Python async.
Async is DEFINITELY NOT about creating "A pretty programming model".
It's about *efficiently handling of blocking operations* which *SOULD* enhance throughput and responsiveness of said application.
When people started talking about the asyncio Python module back before it was released, the expectations were sky high given what async programming is capable of. Python's asyncio never met those expectations related to expected speed improvements, but that doesn't mean it's bad or shouldn't be used.
If I recall, I believe there are supposed to be changes coming (if not already arriving) that should help amp up the efficiency of asyncio.
1
u/sanshinron Sep 09 '22
Yeah I used to update inventory via different APIs with requests and that would take minutes, rewrote it all with aiohttp and it takes 2.6 seconds.
1
u/caleb Sep 11 '22
Miguel Grinberg wrote a detailed response to this article: https://blog.miguelgrinberg.com/post/ignore-all-web-performance-benchmarks-including-this-one
If your system has no latency then asyncio is much less useful. But I haven't come across too many backend systems without latency.
0
u/guhcampos Sep 10 '22
I was going to bash the hell out of this article, but on a second thought the reason this kind of cluessness exists lies in async evangelists claiming its faster, to begin with.
The concept here should be dead simple to anyone with a CS degree:
For classic "sync" code you rely on the underlying system to schedule CPU time for you. How that time is split between competing computations is up to the system, and will generally be a best effort for the majority of use cases.
When you write async code you are taking back the control of CPU time scheduling from the system to your hands. You choose when to switch between different computations, because you believe you can do a better job than what the generic underlying system is doing. With great power, however, comes the great responsibility of managing that CPU allocation. If you are not carefully and intentionally managing the switch between your concurrent computations, you should not be using async at all.
If you don't understand the concept of cooperative concurrency, just use threads to avoid blocking - or go do some reading. Claiming one is "faster" than the other just make you sound stupid.
2
Sep 10 '22
Given the same cpu capacity and well written code, it IS faster if used in the right contexts at performing the same amount of tasks - you can’t make the request go any faster if it takes 2s for your request to finish. If I have one thread, and 100 requests to make, async will complete the task faster than sync will because those two seconds of idle time waiting for a query or processing the response can be used to execute another query or process another response - it won’t make the processing any faster, but it CAN make the overall task faster when used appropriately. The example I’ve outlined, you could finish all 100 requests in slightly more time that a sync system can process the first request because of blocking the thread.
That is objectively faster - but the definition of faster HAS to be clear, it won’t make processing your request any faster. It will make it possible to process more requests at once when you’re dealing with externally constrained performance limitations.
1
u/guhcampos Sep 10 '22
You can simply spawn 100 threads for the same effect. Since you are waiting for IO the GIL will be likely irrelevant.
1
Sep 10 '22
I think you’re right and I’m confusing what I remember about multiprocess performance vs threading performance compared against async and it’s completely separate and work in parallel.
I recall something about performance of threading vs asyncio still having the win for asyncio here but I don’t remember where I picked up that impression now that it’s called into question
57
u/runew0lf Sep 09 '22
Nobody said it was, the important thing is its non blocking..