Python Performance in Depth — waiting or computing

Every performance question in Python is the same question asked twice: is this work waiting, or is this work computing?

Every performance question in Python is the same question asked twice: is this work waiting, or is this work computing?

§ IFrame: The Fourth Category

The Python-track Dev slot has three categories behind it. The 07-27 lesson taught structured concurrency: TaskGroup, the cancellation contract, and the timeout scope. The 07-30 lesson taught packaging. The 08-02 lesson taught typing at an ops library's boundary. Performance is the fourth, and it exists because the 07-27 lesson answered only half of a question it did not admit was halved.

asyncio is a scheduler for work that waits. Two hundred HTTP requests spend their lives blocked on a socket, and one thread walking a ready-queue serves all two hundred beautifully. Change the workload to hashing two hundred files, or parsing two hundred megabytes of JSON, and the same code gets slower than a plain loop. Nothing waits. There is nothing for the scheduler to interleave, and the interleaving machinery is now pure overhead.

The dividing line has a name and the name is the point of the lesson. Ask what the CPU is doing while the task runs. If it is idle, you have an I/O-bound problem, and 07-27's answer stands. If it is busy, you have a CPU-bound problem, and everything below applies.

§ IILanguage Idiom: What the GIL Actually Locks

The Global Interpreter Lock is a mutex around the interpreter state. Ramalho states its scope precisely in Fluent Python (Part IV, p. 766): only one thread at a time may execute Python bytecode, and the interpreter releases the lock about every five milliseconds by default so other threads may run. Two consequences follow, and most confusion about Python threads comes from holding one of them without the other.

The first consequence is the famous one. Four threads running pure-Python arithmetic on a four-core machine run no faster than one, and often slower, because they now pay for lock handoffs on top of the arithmetic.

The second consequence is the one that gets forgotten. The lock is released around blocking calls. Ramalho makes the point directly in the same part of the book (pp. 729-730): every function in the standard library that performs a syscall releases the GIL while it waits, so time.sleep, socket reads, file reads, and subprocess all permit other threads to run. Threads in Python are genuinely useful. They are useful for exactly the workload asyncio also serves, and they cost more memory per unit of concurrency but require no rewriting of synchronous library code. That is not nothing, and for an ops tool that wraps three synchronous SDKs it is often the better trade.

Extension modules obey the same rule. NumPy releases the lock inside its C loops. hashlib releases it around digest computation on large inputs. So "threads cannot use more than one core in Python" is a claim about Python bytecode, not about work done by a Python program. A thread pool over hashlib.sha256 on large files does scale across cores, and a thread pool over a pure-Python checksum does not.

The free-threaded build

PEP 703 removed the lock, and CPython ships that removal as a separate build: python3.13t and later, the free-threaded interpreter. Object reference counting becomes atomic or biased, container operations take fine-grained internal locks, and Python-level threads execute bytecode genuinely in parallel.

Two cautions belong beside that sentence. Single-threaded code on the free-threaded build runs slower than on the default build, because the atomic reference-count operations are not free; the gap has narrowed release over release but has not closed. And a C extension must declare support through Py_mod_gil before the interpreter will keep the lock disabled while it is imported. An extension without that declaration causes the runtime to re-enable the GIL at import time, which means one unaudited dependency in the tree can silently return the whole process to serial bytecode execution.

The practical posture for an ops fleet in 2026 is to treat the free-threaded build as a measurement, not a default. Build the workload both ways, time it, and let the number decide.

§ IIICode Worked Example: Measure, Then Choose

The failure mode this section exists to prevent has a name: the confident guess. An engineer reads that processes beat threads for CPU work, reaches for ProcessPoolExecutor, and ships a tool that is four times slower because the payload it passes to each worker costs more to pickle than the work costs to perform.

Start with the profiler. cProfile answers where the time goes at function granularity, which is the only question worth asking first.

import cProfile, pstats, io

def profile(fn, *args, **kwargs):
    pr = cProfile.Profile()
    pr.enable()
    result = fn(*args, **kwargs)
    pr.disable()
    buf = io.StringIO()
    pstats.Stats(pr, stream=buf).sort_stats("cumulative").print_stats(15)
    return result, buf.getvalue()

Two columns in that output carry the decision. tottime is time spent inside a function excluding its callees, and a large tottime in pure-Python code is the signature of CPU-bound work. A profile whose top entries are all socket reads or _ssl calls is the signature of I/O-bound work, and the answer is 07-27's TaskGroup, not a process pool.

Once the work is known to be CPU-bound, the choice is between three executors, and the deciding variable is the ratio of payload size to compute time.

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import hashlib, pathlib

def digest(path: pathlib.Path) -> tuple[str, str]:
    h = hashlib.sha256()
    with path.open("rb") as fh:
        for block in iter(lambda: fh.read(1 << 20), b""):
            h.update(block)
    return str(path), h.hexdigest()

def run_threads(paths, workers=8):
    with ThreadPoolExecutor(max_workers=workers) as pool:
        return list(pool.map(digest, paths))

def run_processes(paths, workers=8):
    with ProcessPoolExecutor(max_workers=workers) as pool:
        return list(pool.map(digest, paths))

run_threads wins here, and the reason is instructive. The function passes a path across the boundary and returns a hex string; the bytes never cross. Meanwhile hashlib.update releases the GIL on every one-megabyte block, so the threads genuinely occupy multiple cores. Rewrite digest to accept bytes instead of a path and run_processes collapses, because every worker now pays to pickle a megabyte of payload in and out.

Ramalho's prime-checker comparison (pp. 785-786) is the mirror case: is_prime on a large integer is pure-Python arithmetic with a tiny argument and a boolean result. Payload near zero, compute high, no C call to release the lock. Processes win outright, and the multi-core speedup is close to linear.

The rule condenses to one sentence worth carrying. Processes buy parallelism and charge for the crossing; threads are cheap to cross and buy parallelism only where C code releases the lock.

Where neither answer is enough, the third door is the C boundary itself. Moving the inner loop into NumPy, or into a Cython or Rust extension, changes the shape of the problem rather than its scheduling: one Python call now performs a million operations with the lock released, and the GIL debate ends because there is almost no bytecode left to serialize. Lutz notes the family of alternative execution paths in Learning Python (Part I, p. 84) and the modern members of that family, Cython and PyPy among them, remain the honest answer when the profile shows a single hot loop and the loop is arithmetic.

The startup cost nobody profiles

ProcessPoolExecutor starts workers with spawn on macOS and Windows, and increasingly on Linux, which means every worker re-imports the parent's module graph. A tool that imports boto3 at module scope pays roughly a second per worker before any work begins. Eight workers, eight seconds, spent before the first digest. Profile the process, not the function.

§ IVConnection to Today's Ops Lesson

The Ops lesson made concurrency a validated configuration field. This lesson is the argument for why it must be a field rather than a constant. The right number depends on whether the work waits or computes, on whether the hot path releases the lock, on core count, and on payload size, and none of those are knowable at authoring time. The field exists so that an operator with a profile in hand can set it on evidence.

The second connection is sharper and it concerns secrets. The Ops lesson resolves credentials once at startup into parent-process memory. Under fork, workers inherit that memory and the values are simply there. Under spawn, workers start clean, re-import the module graph, and inherit nothing, so a worker that reaches for a resolved secret finds an empty dictionary. The same code passes on Linux and fails on macOS, which is the worst class of bug an ops tool can carry.

The discipline that closes it: pass resolved values explicitly as arguments to the worker function, or set the start method explicitly at the entry point so the behaviour is chosen rather than inherited from the platform.

import multiprocessing as mp

def main() -> int:
    mp.set_start_method("spawn", force=True)
    settings = build_settings(cli_args(), config_path())
    secrets = resolve_secrets(settings)
    return run(settings, secrets)

Setting the method at the entry point rather than at import time is the same rule the 07-30 lesson drew around packaging and the Ops lesson drew around configuration: process-global decisions belong to the application, never to the library.

§ VPrior-Lesson Reach

The 07-27 lesson's TaskGroup and this lesson's ProcessPoolExecutor are not competitors; they compose. The standing pattern for a mixed workload is an async outer layer that fetches, wrapped around loop.run_in_executor with a process pool for the compute step, so waiting and computing each get the machinery built for them. The cancellation contract from 07-27 does not cross the process boundary cleanly, though. Cancelling a future that a worker is already executing does not stop the worker, so a per-task deadline in a process pool needs a timeout inside the worker function as well.

The 08-02 typing lesson gives the executor boundary its contract. A Protocol describing the worker callable documents what must be picklable in and what comes back out, and a type checker catches the closure that captured a client object and cannot cross a process boundary at all.

§ VIClosing

Three moves, in order, and the order is the discipline. Profile first and read tottime to learn whether the work waits or computes. Choose the executor by payload-to-compute ratio, not by folklore. Reach for the C boundary only when the profile shows one hot arithmetic loop and the scheduling answers have been exhausted.

The GIL is not the villain of Python performance. The unmeasured assumption is. A four-times slowdown from a process pool chosen on principle is a self-inflicted wound, and the profiler that would have prevented it ships in the standard library and takes nine lines to use.

Run the profiler on the tool you already trust. The result will be a surprise, and the surprise is the lesson.

Related

• Prior arc: Python's Typing in Depth — Protocols, PEP 695 Generics, and the Ops-Library Boundary
• Language hub: Python
• Grounding tome: Fluent Python, 2nd ed. — Luciano Ramalho (Part IV, Control Flow, pp. 729-730 and p. 766)
• Paired Ops: Configuration and Secrets for Python Ops Tools
• Paired Cert: AWS DOP-C02 Domain 1 — SDLC Automation