Why Python Owns ML

TL;DR

Every serious ML library — PyTorch, TensorFlow, JAX, scikit-learn, Hugging Face Transformers, LangChain, vLLM — exposes a Python API. The actual numerics live in C++, CUDA, Rust, or Triton, but the layer you write is Python. There is no second-place language; the gap between Python and the runner-up (Julia, briefly hopeful, has not closed it) is enormous.

That means “knowing Python” is not optional and not negotiable for an ML engineer. But “knowing Python” in this context is a specific subset: mutability and references (so you don’t accidentally share state across training runs), comprehensions and generators (so your data pipelines don’t OOM), the typing system (so your codebase survives past 5,000 lines), NumPy and Pandas (because you’re moving tensors and dataframes all day), async (because every LLM call is I/O-bound), and the toolchain around all of it.

This topic is the wiki entry for those things. It is not a beginner programming course — if you’ve never written a for loop, start elsewhere. It assumes you can read Python and want the working knowledge that ML engineers have day to day.

The picture in your head

Python is a glue language with a thin layer of syntax sitting on top of C extensions. When you call torch.matmul(a, b), the Python interpreter spends maybe 10 microseconds dispatching the call, then hands the actual work to a CUDA kernel that runs in 200 microseconds. The Python layer’s job is orchestration: build the right tensors, set up the right optimizer, schedule the right training loop. The performance-critical work is not in Python — it’s in the C library Python is calling.

This split is the single most important fact about Python for ML. It explains why “Python is slow” is mostly irrelevant (the inner loop isn’t in Python); it explains why a clumsy Pandas one-liner can be 1000× slower than a different one-liner that does the same thing (one stays in C, one falls back to Python); and it explains why you optimize ML code by staying out of Python, not by writing faster Python.

Why Python and not something else

A few reasons, ordered by how much they actually matter.

The non-reasons: “Python is elegant” (it’s fine, but Ruby is more elegant and nobody does ML in Ruby), “Python is easy” (so is JavaScript). The reason is the libraries, full stop.

What “knowing Python for ML” actually means

A working ML engineer uses these things every week. If any of them feel fuzzy, the corresponding guide in this topic is worth a slow read.

Language fundamentals you will use daily

The numerical stack

You don’t write training loops in pure Python. You write them in NumPy and PyTorch, which means thinking in arrays and broadcasts.

Concurrency

Most ML latency is not CPU-bound. It’s GPU kernels (handled by PyTorch), or I/O — file reads, network calls to LLM APIs, S3 fetches.

Tooling

Code that doesn’t ship is code that doesn’t matter. The shipping layer in Python is its own subject.

A worked example — the Python you actually write in ML

Here is a tiny but honest snippet. It loads a dataset, batches it, runs inference with an LLM client asynchronously, validates the output with Pydantic, and writes results. Every guide above shows up.

import asyncio
import logging
from dataclasses import dataclass
from pathlib import Path

import pandas as pd
from pydantic import BaseModel, Field
from openai import AsyncOpenAI

logger = logging.getLogger(__name__)
client = AsyncOpenAI()


class Classification(BaseModel):
    label: str = Field(pattern=r"^(positive|negative|neutral)$")
    confidence: float = Field(ge=0.0, le=1.0)


@dataclass(frozen=True, slots=True)
class JobConfig:
    input_path: Path
    output_path: Path
    model: str = "gpt-4o-mini"
    concurrency: int = 16


async def classify_one(text: str, model: str) -> Classification:
    resp = await client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": f"Classify sentiment: {text!r}"}],
        response_format={"type": "json_object"},
    )
    return Classification.model_validate_json(resp.choices[0].message.content)


async def run(cfg: JobConfig) -> None:
    df = pd.read_csv(cfg.input_path)
    sem = asyncio.Semaphore(cfg.concurrency)

    async def bounded(text: str) -> Classification:
        async with sem:
            return await classify_one(text, cfg.model)

    results = await asyncio.gather(*(bounded(t) for t in df["text"]))
    out = df.assign(
        label=[r.label for r in results],
        confidence=[r.confidence for r in results],
    )
    out.to_csv(cfg.output_path, index=False)
    logger.info("wrote %d rows to %s", len(out), cfg.output_path)


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    asyncio.run(run(JobConfig(Path("in.csv"), Path("out.csv"))))

Count what’s there: a dataclass for the config, a Pydantic model for validated LLM output, async fan-out with a semaphore for rate limiting, a Pandas DataFrame for I/O, structured logging, type hints throughout. That’s an honest 80 lines of ML engineering Python. None of the libraries are exotic; all of them are in this topic.

Common beginner mistakes — what trips people up

• Treating Python like Java. Long classes for small jobs. dataclasses and module-level functions are usually the right answer for ML pipelines, not OOP hierarchies.
• For-looping over NumPy or Pandas. A Python loop over a million-row Series is 100× slower than the vectorized one-liner. See PY 203.
• Mutable default arguments. def f(x, items=[]): is the most famous Python footgun. The [] is created once and shared across every call. See PY 101.
• Skipping type hints because “we’ll add them later.” You won’t. Add them as you write. mypy or pyright catches real bugs.
• pip install into the system Python. Use a venv or uv. Always.
• Using print for logging. print has no levels, no destination control, no structure. Logs from production are how you debug at 3am. See PY 402.
• Not knowing what is does. is checks identity (same object), not equality. [1, 2] is [1, 2] is False. a = b; a is b is True. Use == for value comparison, is for None.

Python versions and what to use

Use Python 3.11 or newer unless something pins you. 3.11 brought significant interpreter speedups (10-60% on real workloads), 3.12 improved f-strings and error messages, 3.13 made the GIL optional (experimental). The free-threaded build under PEP 703 is the most significant Python release of the decade for compute-heavy ML; production adoption will still take a couple of years.

Avoid Python 3.10 and below for new projects. Hard-avoid Python 2.

How to actually learn this

The content standard for the rest of this topic is “engineer at a whiteboard.” Not “language tour.” Each guide picks a concept that ML engineers use weekly, explains the mental model, shows real code, lists the gotchas, and points at the production patterns. There are no exhaustive language references — docs.python.org already exists.

The order to read in, if you’re new to ML Python:

1. PY 101–103 — language fundamentals you’ll get wrong otherwise.
2. PY 201–203 — NumPy is the lingua franca of tensors.
3. PY 401, 402, 403 — set up your project, your logs, your tests.
4. Then everything else as the need arises.

Resources

• Python docs — docs.python.org — the canonical reference.
• Fluent Python (2nd ed.) — Luciano Ramalho — oreilly.com — the single best book on idiomatic Python. Read cover to cover at some point.
• High Performance Python (2nd ed.) — Gorelick & Ozsvald — oreilly.com — vectorization, profiling, Numba, multiprocessing — done right.
• Real Python — realpython.com — workmanlike tutorials, better than most blogs.
• PEP Index — peps.python.org — the design specs. Read PEP 8, PEP 20, PEP 257, PEP 484, PEP 492, PEP 557 at minimum.
• Talk Python To Me podcast — talkpython.fm — interviews with maintainers; how to stay current.