Python › Memory Management & Garbage Collection

What is memory fragmentation in the context of Python's object allocator, and how does pymalloc address it for small objects?

Advanced

Answer

CPython's pymalloc allocator manages small objects (under 512 bytes) using pools and arenas sized to reduce fragmentation and avoid frequent calls to the OS allocator (malloc), which is comparatively slow and fragmentation-prone for many small, short-lived allocations typical of Python programs. Large objects bypass pymalloc and go directly to the system allocator.

import sys

# pymalloc handles allocation for small objects like this transparently
small_objects = [object() for _ in range(10000)]  # efficiently pooled

# Large arrays typically go through the system allocator instead
import array
big = array.array('d', [0.0] * 10_000_000)

Real-world example Profiling a Python application's memory allocator behavior with tools like tracemalloc reveals many small, short-lived dict and tuple allocations that pymalloc handles efficiently versus the pattern for very large NumPy arrays which bypass it.

Common follow-ups: How does tracemalloc help profile allocation patterns?;Why do large objects bypass pymalloc?

Debugging & Profiling;Data Types & Structures