In this post I will try to give you some insights in my mental model about parallelism when using asynchronous or threading style parallelism. And I also show how you can utilize both types simultaneous. <\/p>\n\n\n\n
This may not be a super valid article about parallelism in Python. It may also not be an proper academic definition of things I write here, but it’ll address some very basic understanding about how to write proper asynchronous code.
You will learn how to run async code at a next level leaving old design patterns from times of threading.
I also recommend this read to async beginners to avoid doing things wrong at the first step, because in many cases you will not recognized that there is a better way.<\/p>\n\n\n\n
Synchronous programming means that the control flow of the program runs synchronous. Each line of code is executed instruction by instruction and line by line from to bottom allowing only jumps at well defined instructions. Apart from well defined jumps the code is executed vertically.<\/p>\n\n\n\n
# synchron code\ndef foo():\n print(\"starting here\")\n a = 0\n a += 1\n b = (a + 1) \/ 2\n # this for loop jumps at a well-defined place.\n for x in range(10):\n b += x\n print(\"leaving funtion\")\n return b\n<\/code><\/pre>\n<\/div>\n\n\n\n\nCode goes from top to bottom.
Jumps occur here at the for loop
Code is executed vertically<\/p>\n<\/div>\n<\/div>\n\n\n\n
For sure this nothing new to you but for the next thoughts it could be useful to keep this in mind.<\/p>\n\n\n\n\n\n\n\n
Threading: Parallel but Horizontal<\/h2>\n\n\n\n
Parallel programming or better parallel execution allows code to be run in parallel. So each line can be executed at virtual the same time. Pure time parallelism can only occur when code is executed on more then one processor. If you program is just running on one CPU we are running only virtual parallel and switching from time to time between the tasks.
But the programmer cannot determine at which time this exactly happens. Of course there are pretty good scheduler run by the OS and triggered by hardware interrupts so that the timing of context switches is already optimized. But in many cases the programmer knows better in front when there is a good moment to halt and give other code way (for example doing I\/O with the hard-drive). A very interesting stack overflow answer on how and when task switch happen can be found here<\/a>.<\/p>\n\n\n\n
My mental model now visualize this as horizontal parallelism because the function, method etc. is executed virtually parallel but in best case at the same line of code.<\/p>\n\n\n\n
# parallel code using threading\ndef bar():\n\n # this function will be executed virtual parallel. \n # The context switch and therefore the execution of a line of code \n # is non-deterministic from perspective of the function.\n def foo(result):\n b = 0\n for x in range(10):\n b = b + x\n time.sleep(0.1) # here we cannot tell that we halt here for some time. \n result.append(b)\n\n results = []\n # create 2 threads\n threads = [threading.Thread(target=foo, args=(results,)) for _ in range(2)]\n # start 2 threads \"parallel\"\n [t.start() for t in threads]\n # wait for them\n [t.join() for t in threads]\n print(\"result:\", results)\n # prints result: [45, 45]<\/code><\/pre>\n\n\n\nA typical pattern to use the virtual or actual parallelism is the so called producer\/consumer pattern.<\/p>\n\n\n\n
If threads or processes are executing exactly the same code for example inherited from the same function as others do, we call them very often worker. Worker often communicate through a queue where data comes in, is calculated and send out again by another queue. A typical design pattern is the producer\/consumer model. <\/p>\n\n\n\n
# this is \"sync\" consumer producer pattern were the parallelism\n# will be added by creating more producer oder consumer\n# instead a queue one also could use a mutable buffer like a list but needs then a semaphore.\n# The queue does this all for us.\ndef producer(q:queue.Queue):\n for work in range(100):\n time.sleep(0.001) # extra heavy work\n q.put(work)\n q.put(None) # poison pill\n print(\"producer done\")\n\n\ndef consumer(q: queue.Queue):\n res = 0\n while True:\n item = q.get()\n if item is None:\n break\n else:\n res = res + item\n print(\"consumer result\", res)\n\n\ndef main():\n q = queue.Queue()\n threads = [threading.Thread(target=target, args=(q,)) for target in [producer, consumer]]\n [t.start() for t in threads]\n [t.join() for t in threads]\n print(\"bye\")<\/code><\/pre>\n\n\n\nIn traditional multitasking patterns like threading, horizontal parallelism is done at high costs of memory. Threads need their context to run and the context needs to be switched whenever it is time to switch to a different task. In times where memory is cheap and CPU has large bandwidth to the memory this disadvantage tends to disappear as a matter of performance. This also could be a reason why threading is still one of the most popular multitasking paradigm used. But there are many more pitfalls<\/a> one may consider. But that is not subject here. <\/p>\n\n\n\n\n\n\n\nAsynchronous: Parallel but Vertical<\/h2>\n\n\n\n
In asynchronous or cooperative multitasking the programmer can decide at well defined instruction where code can be executed virtual parallel.
This gives availability to make synchronous code vertical parallel. This means that the code itself will be executed in the order of written lines (from top to bottom). But whenever a well defined transition happens other code could be executed in synchronous way at a different place. This is why I call this vertical parallelism.
To read such code, you can just ignore the syntax await and you easily understand the execution direction (like synchronous code from top to bottom). <\/p>\n\n\n\n
# a simple asynchron function will execute vertical line by line \n# allowing at defined places to halt(wait) and resume for\n# other code to be executed\nasync def foo():\n print(\"starting here\")\n a = 1\n await asyncio.sleep(1) # at this place other code can be executed\n print(\"slept for 1 second\")\n a += 2\n await asyncio.sleep(0.5) # at this place other code can be executed\n print(\"leaving\")\n return a<\/code><\/pre>\n\n\n\nThe above code does exact same as the below sync version of it. <\/p>\n\n\n\n
async def foo():\n print(\"starting here\")\n a = 1\n time.sleep(1) # at this place other code can be executed\n print(\"slept for 1 second\")\n a += 2\n time.sleep(0.5) # at this place other code can be executed\n print(\"leaving\")\n return a<\/code><\/pre>\n\n\n\nIn the sync case the interpreter is deemed to wait for the sleep time. In background they so called global interpreter lock (GIL) is released and the OS can continue switch to another thread. <\/p>\n\n\n\n
Parallel but horizontal and vertical<\/h2>\n\n\n\n
Now combining both direction could give us a very high performant pattern. The basic asynchronous framework, asyncio, in Python has therefore a simple wrapper for coroutines. Asyncios event loop comes with a method called create_task. This method wraps the passed coroutine into a task object (a subclass of asyncios Future). The task execution is then scheduled by the event loop. The programmer should hold reference because asyncio\/python will garbage collect after the coroutine has finished. <\/p>\n\n\n\n
async def request(page:int, client: aiohttp.ClientSession):\n response = await client.get(f\"https:\/\/books.toscrape.com\/catalogue\/page-{page}.html\")\n if response.status < 300:\n body = await response.text()\n print(body[:18].strip())\n\n\nasync def main():\n client = aiohttp.ClientSession()\n loop = asyncio.get_event_loop()\n time_taken = time.time()\n async with client:\n # hold reference here\n pending = [loop.create_task(request(page, client)) for page in range(10)]\n # wait for them and release when all complete (default of wait())\n done, pending = await asyncio.wait(pending, return_when=asyncio.ALL_COMPLETED)\n print(f\"done in {time.time() - time_taken:.2f}\")\n await client.close()\n await asyncio.sleep(0.5)\n\nif __name__ == '__main__':\n asyncio.run(main(), debug=True)\n # prints \n # <!DOCTYPE html>\n # ...\n # done in 1.02<\/code><\/pre>\n\n\n\nAs you can see the request coroutine will first issue a http request and wait for it.
When the we received a response, we will check its status and then wait again to read the buffer from network interface. This so far is vertical and typical for asynchronous style.
But this is not enough, so we issue 10 request coroutines at nearly the same time, doing all the above stuff, but 10 times horizontal parallel. <\/p>\n\n\n\n