r/Python • • Nov 30 '22

Resource Async Python is not faster - excellent analysis from Cal Paterson

https://calpaterson.com/async-python-is-not-faster.html
11 Upvotes

18 comments sorted by

View all comments

6

u/blabbities Dec 01 '22

I swear I can view a thousand Async/Concurrency blogs, papers, docs,and vids. I'll never really understand it.

In fact I have a Python page that "explains it well" on one of my VMs right now. this blog seem cool too but I guess. I've also made one python program that used it and saw the improved speeds. I also am learning GoLang in which Concurrency is like first-class thought in it's language design.... Tho somehow I just don't get it.

Maybe it's that I fear diving more into it because Im not doing big data projects and I'm afraid of the complex debugging issues.

Anyway good blog. Adding it to my saves 😂.

10

u/jorge1209 Dec 01 '22 edited Dec 01 '22

Async is best for situations where individual requests take a short amount of time and then there is a long I/O wait. For instance a web server. 10k requests come in, they take nanoseconds to perform initial processing, then you wait an eternity for that packet to cross the internet and a response to come back, before you can make another near instantaneous response.

You could implement that as threads, but then you need 10k threads one for each request, and each will then insert itself into the wake loop for the kernel network layer saying "wake this thread when you get a response on this particular connection", and that uses a lot of kernel resources.

Alternately you could have a single process that spin waits on the network device. That single process says: "give me all responses from this port and I will dispatch them".

That second approach can be a lot more performant (in certain situations) s the application knows exactly what it needs and frees the kernel from having to monitor all these threads. The challenge is how to implement this, because while the service is handling any given request it cannot easily handle others, and there are a variety of approaches.

One common approach is threadpools, but the downside is that you have to serialize your state: after making the initial response the thread will be used to handle other unrelated requests, and you need to save all the relevant details of the current request to some data structure so it can be looked up when the response comes back. Programmers don't like to have to think about serializing their state like this and just wish they could just say "continue from here".

Python Async uses cooperative multi-tasking where you write specially annotated functions that perform all the work they can, but then announce (via calling yield) when they have reached a point where they have to wait. The individual yield calls are exactly those annotations of "continue from here". At that point the event loop can check if anyone else can be dispatched, and at some remote point in the future come back to continue execution from the point of yield.

Go takes a different approach and basically manages threading within the runtime. When calling a goroutine it doesn't merely store the current registers of the CPU on its own execution stack, but actually allocates a new stack for that goroutine on the heap, and then it schedules the execution of that goroutine. [This structure is also found in "stackless python"]. The Go runtime provides the I/O functions that you might wait on and it knows that at those points (and a few other points specified in the language) will it be able to preempt the goroutine and repurpose the runtime thread to run another goroutine.

The go approach is nicer as it means that all the required information is present to run code like it was an actual thread, but without forcing the kernel to manage that thread, nor requiring the programmer to annotate the explicit points at which the code can and should yield, but since you aren't using a C call stack there is some added complexity when calling C code or libraries.

In other words:

  • In C and python, the state of your function (the value of all current variables, and who you would return to) is reflected in the call stack of the running program. You can't suspend a function call without suspending the program as a whole.

  • In async specifically annotated functions are stored on the heap just as any other managed object would be. In that sense they are no different from strings or arrays and the runtime can call or suspend them effectively at will, but you have to annotate the functions that will do so.

  • In Go and stackless, all functions are managed objects on the heap, and all functions are therefore eligible for this special treatment, no additional annotations are required.

1

u/onedirtychaipls Dec 01 '22

So I have a particular dilemma and I wonder if you have quick input. I've been making a script that asynchronously logs in users and has them do work. For load testing. But I keep hitting a bottleneck and it doesn't feel like it can actually do that. Is python the wrong choice? Or should I keep working on it?

1

u/jorge1209 Dec 01 '22

Python Async probably doesn't do what you think. It is still inherently single-threaded.

There is no difference between:

 async generate_load(id):
        req = make_request(id)
        yield
        read_answer(req)

 
asyncio.gather(generate_load(i) for i in range(1e5))

and

requests = [make_request(i) for i in range(1e5)]
answers = [read_answer(r) for r in requests]

except that the former is slower and more complex. In either case only 1 request is being made at any one time.

To fix this you would need a ThreadPool and then map the requests through the pool.

BUT BUT BUT BUT

Python interpreter has something called the GIL which exists to protect core interpreter data structures, and if make_request is a pure python function, then even if you have a ThreadPool, it will still be serialized at the level of python bytecode.

So in Python you probably need a MultiprocessingPool... and thats really heavy. To do this properly look either at gevent, or just don't use python.