Python › Logging
Why is logging generally considered thread-safe, and what should you still be careful about in multiprocessing?
AdvancedAnswer
The logging module uses internal locks around handler emit() calls, making it safe for multiple threads to log through the same handler without corrupting output. However, in multiprocessing, each process has its own memory and locks, so multiple processes writing to the same file handler can interleave or corrupt output -- you typically need a QueueHandler/QueueListener setup or process-specific log files instead.
from logging.handlers import QueueHandler, QueueListener
import logging, multiprocessing
q = multiprocessing.Queue()
qh = QueueHandler(q)
logging.getLogger().addHandler(qh)
# A QueueListener in the main process consumes from q and writes safelyReal-world example A multiprocessing data pipeline routes all worker process logs through a shared Queue to a single QueueListener that writes safely to one file, avoiding interleaved or corrupted log lines.