Making an async coroutine blocking¶

For @amueller

In [1]:

import asyncio
import threading
from concurrent.futures import ThreadPoolExecutor

Let's say I have a coroutine, but want to use it from a blocking API:

In [2]:

async def some_coroutine(n=1):
    """A coroutine that takes some time"""
    for i in range(n):
        await asyncio.sleep(0.1)
        print(i)
    return f"I ran in thread {threading.get_ident()}"

IPython suports top-level await to run on the current eventloop

In [3]:

await some_coroutine(3)

0
1
2

Out[3]:

'I ran in thread 4449029632'

At the library level, you can pass coroutines to asyncio.run in a different thread to turn any async def function into a blocking one.

In [4]:

def block_coroutine(n):
    with ThreadPoolExecutor(1) as pool:
        return pool.submit(lambda: asyncio.run(some_coroutine(n))).result()

block_coroutine(4)

Out[4]:

'I ran in thread 123145466920960'

This assumes it's self-contained, i.e. doesn't refer to other things running on the same loop, as the loop will be created and destroyed around just this call.

A similar, but more complicated approach can hand off things to a persistent eventloop in another thread, using the threadsafe concurrent.futures to coordinate. This is how IPython Parallel's client works!.