Python › Async Generators & Async Context Managers

How do you build an async generator pipeline, chaining one async generator's output into another, like a synchronous generator pipeline?

Intermediate

Answer

Define each stage as its own async generator that takes an async iterable as input and 'async for's over it, yielding transformed values — chaining multiple stages together creates a fully lazy, asynchronous processing pipeline, mirroring how synchronous generators compose.
async def double(source):
    async for item in source:
        yield item * 2

async def only_even(source):
    async for item in source:
        if item % 2 == 0:
            yield item

async def main():
    async for value in only_even(double(numbers_stream())):
        print(value)
Real-world example Building a lazy, asynchronous ETL-style pipeline that transforms and filters streamed data without buffering it all in memory.

Common follow-ups: How does this pipeline pattern avoid loading the entire dataset into memory at once?