All 15 Concurrency (asyncio/threading/multiprocessing) questions
More in Concurrency (asyncio/threading/multiprocessing)
How do you safely run a blocking, synchronous function from within an asyncio coroutine without freezing the event loop? How do you share data between processes using multiprocessing, given that separate processes don't share memory the way threads do? How would you implement a producer-consumer pattern combining a thread-safe queue.Queue with multiple worker threads? How do you correctly cancel a running asyncio task, and how should the task itself handle the resulting CancelledError? How would you design a robust worker pool that handles both graceful shutdown (finishing in-flight work) and immediate cancellation (dropping everything), using asyncio?
Advertisement
Python › Concurrency (asyncio/threading/multiprocessing)

How would you use concurrent.futures.ProcessPoolExecutor together with asyncio to run CPU-bound work without blocking the event loop?

Advanced

Answer

Combine loop.run_in_executor() with a ProcessPoolExecutor instance (instead of the default thread pool) to offload genuinely CPU-bound work to a separate PROCESS, bypassing the GIL for true parallelism, while still awaiting the result asynchronously from within your asyncio application.
import asyncio
from concurrent.futures import ProcessPoolExecutor

def cpu_heavy(n):
    return sum(i * i for i in range(n))

async def main():
    loop = asyncio.get_running_loop()
    with ProcessPoolExecutor() as pool:
        result = await loop.run_in_executor(pool, cpu_heavy, 10_000_000)
    print(result)
Real-world example Running a CPU-intensive computation (like image processing or numerical analysis) from within an async web server without blocking request handling.

Common follow-ups: Why must arguments passed to a ProcessPoolExecutor's function be picklable, unlike with a ThreadPoolExecutor?

functools & Functional Programming Tools
All 15 Concurrency (asyncio/threading/multiprocessing) questions