Mutability, Identity, References

TL;DR

Every Python value is an object on the heap. Variables are names that point to objects, never the objects themselves. a = b does not copy b — it makes the name a point at the same object b already points at. If that object is mutable, modifying it through a is immediately visible through b, because there is only one object.

That single fact — names are references, not boxes — explains the entire class of “I changed one thing and something else broke” bugs. It explains why def f(items=[]): accumulates state across calls. It explains why model_a.weights = model_b.weights doesn’t give you two independent weight tensors. It explains why import copy exists and when you need copy.deepcopy.

Mutable in Python: list, dict, set, bytearray, most class instances, numpy.ndarray, torch.Tensor. Immutable: int, float, str, bytes, tuple, frozenset, None, True, False. The line between them is the line where assignment-aliasing goes from “fine” to “a bug waiting”.

The picture in your head

A variable in Python is a sticky note with a name on it, slapped onto a real object sitting on a shelf. a = [1, 2, 3] puts the list [1, 2, 3] on the shelf and slaps a sticky note labeled a onto it. b = a peels off another sticky note labeled b and slaps it onto the same list. There is one list, two sticky notes.

a.append(4) reaches over to the shelf, grabs the list (using either sticky note — same shelf, same object), and adds 4 to it. Now both a and b see [1, 2, 3, 4], because there is still only one list.

a = [9] does something different. It puts a new list on the shelf, peels off the a sticky note from the old list, and slaps it onto the new one. b is still on the old list. After this, a == [9] and b == [1, 2, 3, 4].

Mutating an object versus rebinding a name are different operations. Conflating them is the bug.

The three operators that matter

Use == for value comparison (x == 5, name == "alice"). Use is only for None, True, False, and sentinels — anything where there is exactly one canonical instance.

>>> a = [1, 2, 3]
>>> b = a
>>> c = [1, 2, 3]
>>> a == b, a is b
(True, True)
>>> a == c, a is c
(True, False)        # equal value, different objects
>>> id(a) == id(b)
True
>>> id(a) == id(c)
False

The classic footgun — mutable default arguments

def append_one(x, items=[]):
    items.append(x)
    return items

>>> append_one(1)
[1]
>>> append_one(2)   # uh oh
[1, 2]
>>> append_one(3)
[1, 2, 3]

The default [] is constructed once, when the def statement runs, and reused on every call. Every invocation that doesn’t pass items mutates the same list.

The fix is the canonical idiom: use None as the sentinel, build a fresh list inside.

def append_one(x, items=None):
    if items is None:
        items = []
    items.append(x)
    return items

Note items is None, not items == None. Identity is right here: there is exactly one None.

This bug compounds in ML. A Trainer.__init__(self, callbacks=[]) with a mutable default means every Trainer instance you don’t explicitly pass callbacks to ends up sharing the same callback list. Add a callback to one trainer, every other trainer that took the default now has it too. Ruthless to debug.

Aliasing vs copying — one object or two?

import copy

a = [[1, 2], [3, 4]]
b = a                  # alias — same outer list, same inner lists
c = a.copy()           # shallow copy — new outer list, same inner lists
d = copy.deepcopy(a)   # deep copy — new outer list, new inner lists

a[0].append(99)
print(b)   # [[1, 2, 99], [3, 4]]   shared
print(c)   # [[1, 2, 99], [3, 4]]   inner list still shared!
print(d)   # [[1, 2], [3, 4]]       fully independent

a.append([5, 6])
print(b)   # [[1, 2, 99], [3, 4], [5, 6]]   still aliased
print(c)   # [[1, 2, 99], [3, 4]]            outer list independent

Shallow vs deep matters constantly in ML. state_dict.copy() returns a new dict but the tensor values are shared with the original — change a parameter through one, see the change through both. To checkpoint weights you actually want, {k: v.clone() for k, v in state_dict.items()} or copy.deepcopy(state_dict).

Mutable vs immutable — the line that matters

>>> x = 10
>>> y = x
>>> x = 11   # x rebinds; y still sees 10
>>> y
10

>>> a = [1, 2]
>>> b = a
>>> a.append(3)   # mutate; b sees the change
>>> b
[1, 2, 3]

For immutables, you can never have a “change visible through another name” bug, because there’s no way to mutate the object — every modification produces a new object. s = "hello"; s += " world" rebinds s to a new string; the old "hello" is unchanged.

For mutables, every obj.method_that_mutates(...) call is a potential spooky-action-at-a-distance bug if anyone else holds a reference.

Function arguments — pass by reference, kind of

Python is “pass by object reference” — also known as “call by sharing”. The function gets a new local name pointing at the same object the caller passed. Mutating the object is visible to the caller. Rebinding the local name is not.

def mutate(x):
    x.append(99)        # caller sees this

def rebind(x):
    x = [0, 0, 0]       # caller does NOT see this; only local name reassigned

a = [1, 2, 3]
mutate(a);  print(a)    # [1, 2, 3, 99]
rebind(a);  print(a)    # [1, 2, 3, 99]  — unchanged by rebind

This is the rule for every mutable Python object passed to a function: NumPy arrays, PyTorch tensors, dictionaries of weights, the lot. If a function “doesn’t return anything”, it’s almost certainly because it’s mutating its input. Read the docstring.

Identity caching — small ints lie about is

>>> a = 256
>>> b = 256
>>> a is b
True
>>> a = 257
>>> b = 257
>>> a is b
False              # CPython caches small ints in [-5, 256]

This is an implementation detail of CPython, not a language guarantee. Never use is for integer comparison. The fact that a is b is True for small integers is a coincidence of caching, not a feature you can rely on.

Same trap for short strings — interned, so "hi" is "hi" is often True — but again, don’t depend on it.

Hashability — the immutable / mutable boundary, again

A dict key must be hashable. Hashable in practice means immutable (with a few subtleties). You cannot use a list as a dict key, you can use a tuple. You cannot put a set in another set, you can put a frozenset.

>>> {[1, 2]: "x"}
TypeError: unhashable type: 'list'

>>> {(1, 2): "x"}
{(1, 2): 'x'}

>>> seen = set()
>>> seen.add(frozenset({1, 2, 3}))   # works
>>> seen.add({1, 2, 3})              # TypeError

This shows up when you want a dict keyed by “the set of features used in this model variant”. Use frozenset. Or convert to tuple(sorted(...)).

Common gotchas

• Default mutable args. Already covered. The single most cited Python footgun.
• Aliased state in dataclasses. A field(default_factory=list) is the dataclass-equivalent fix. Plain field(default=[]) will be rejected by dataclasses precisely because it would be the bug.
• a = b = []. Both names point at the same list. a.append(1) also appears in b.
• [[]] * 3. Creates a list of three references to the same inner list. x = [[]] * 3; x[0].append(1) gives [[1], [1], [1]]. Use [[] for _ in range(3)] to get three independent lists.
• dict.fromkeys(keys, []). Same problem — every value is the same list. Use a comprehension.
• is on integers, strings, or floats. Don’t. Use ==.
• Mutating during iteration. for x in d: del d[x] raises RuntimeError. Iterate over list(d) if you need to mutate.

Where this bites in ML

• Shared optimizer state across runs. A default_factory=list you forgot to use means two TrainingRun instances share their metrics_log. Every metric appended in run 2 also “happened” in run 1.
• state_dict.copy() for checkpointing. Shallow — tensors are aliased. The “checkpoint” mutates with the model. Use {k: v.detach().clone() for k, v in state_dict.items()}.
• Config dataclasses passed into multiple model factories. If the config is mutable and one factory tweaks cfg.dropout mid-build, every other consumer of that config sees the new value.
• Dataloader workers and shared lists. Multiprocessing forks copy the parent’s references; mutate-then-fork patterns differ from mutate-after-fork patterns. The frozen-dataclass habit avoids the whole class.

The defensive habit: prefer immutable types where you can. frozen=True on dataclasses. tuple over list for fixed-size collections. frozenset for membership-test sets that won’t change. Cheap, free, removes a category of bugs.

A worked diagnostic

Two functions, find the bug:

class ModelConfig:
    def __init__(self, layers=[64, 64], dropout=0.1):
        self.layers = layers
        self.dropout = dropout

def make_models():
    a = ModelConfig()
    b = ModelConfig()
    a.layers.append(128)
    return a, b

>>> a, b = make_models()
>>> a.layers, b.layers
([64, 64, 128], [64, 64, 128])

a and b share the exact same layers list because [64, 64] is the default arg, instantiated once when the class body ran. Mutate via a, visible via b.

Fix:

class ModelConfig:
    def __init__(self, layers=None, dropout=0.1):
        self.layers = list(layers) if layers is not None else [64, 64]
        self.dropout = dropout

Or, better, use a frozen dataclass and stop the whole class of bug:

from dataclasses import dataclass, field

@dataclass(frozen=True, slots=True)
class ModelConfig:
    layers: tuple[int, ...] = (64, 64)
    dropout: float = 0.1

Now you literally cannot mutate the layers, and the default is an immutable tuple anyway.

Resources

• Ned Batchelder — Facts and Myths about Python Names and Values — nedbatchelder.com — the talk that finally makes Python assignment make sense.
• Python docs — Data model — docs.python.org — the canonical reference for object identity, mutability, hashing.
• Fluent Python (2nd ed.) — Chapter 6 — oreilly.com — Object References, Mutability, and Recycling.
• Python docs — copy module — docs.python.org — copy.copy vs copy.deepcopy, and how to customize for your own classes.