VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D2: Math, Stats & Optimization · Reading Maths Notation — Functions, Sets, Sums, Products and argmax
Phase 1/Domain 2/Lesson 1

Reading Maths Notation — Functions, Sets, Sums, Products and argmax

15 min read·Foundational·Free lesson
Reading Maths Notation — Functions, Sets, Sums, Products and argmax

You already know: Python loops, lists, dicts and functions (Python track); NumPy arrays (Python internals).

Why this matters for Atlas. ML maths is mostly a handful of symbols used over and over: functions mapping one space to another, sets and membership, sums and products over indices, and argmax/argmin to pick the best option. The loss of a language model, the definition of attention and the training objective of every model in this course are written this way. This lesson translates each symbol into code, so formulas become something you can run and check.


1. Functions and sets

The question. What does f: ℝⁿ → ℝ mean, and why do papers bother writing it?

Start from nothing. A function f: X → Y maps every element of a domain X to one element of a codomain Y. The signature is a type annotation: f: ℝⁿ → ℝ takes a vector of n real numbers and returns one real number (a loss is such a function); softmax: ℝᵏ → Δᵏ maps k scores to a probability vector. Sets are collections without order or duplicates: x ∈ S means x is in S, A ⊆ B means every element of A is in B, A ∪ B and A ∩ B are union and intersection, {x ∈ S : condition} is a filtered set (a set comprehension), and |S| is the number of elements. Common sets: ℕ (naturals), ℤ (integers), ℝ (reals), ℝⁿ (vectors of n reals), [a, b] (the interval).

python
import numpy as np

def softmax(z):                                  # softmax: R^k -> simplex (non-negative, sums to 1)
    e = np.exp(z - z.max())
    return e / e.sum()

vocab = {"the", "cat", "sat", "mat"}
sentence = ["the", "cat", "sat", "on", "the", "mat"]
oov = {w for w in sentence if w not in vocab}    # {w in sentence : w not in V}
p = softmax(np.array([2.0, 1.0, 0.1]))
print("out-of-vocabulary set:", oov, "| size |S| =", len(set(sentence)))
print("softmax output:", np.round(p, 3), "sum =", p.sum().round(6), "all >= 0:", bool((p >= 0).all()))

The set comprehension finds {'on'} as the only word outside the vocabulary, and the sentence has 5 distinct words (|S| counts "the" once). The softmax output [0.659 0.242 0.099] sums to 1 with every entry non-negative: it really does land in the probability simplex Δ³, as its signature promised.

The signature you'll read most often: tensor shapes. In ML papers and code, the most common "function signature" is a shape. X ∈ ℝ^{B×T×d} says X is a batch of B sequences, each T tokens long, each token a d-dimensional vector. Matrix products only make sense when inner dimensions agree, so shapes act like types: a mismatch is caught before any maths happens.

python
import numpy as np

B, T, d = 2, 5, 8                                    # batch, sequence length, model width
rng = np.random.default_rng(0)
X = rng.standard_normal((B, T, d))                   # X ∈ R^{B×T×d}
W_q = rng.standard_normal((d, d))                    # W_Q ∈ R^{d×d}
Q = X @ W_q                                          # (B,T,d)·(d,d) -> (B,T,d)
scores = Q @ Q.transpose(0, 2, 1)                    # (B,T,d)·(B,d,T) -> (B,T,T)
print("X", X.shape, "-> Q", Q.shape, "-> scores", scores.shape)

W_bad = rng.standard_normal((d + 1, d))              # a weight declared as R^{9×8} by mistake
try:
    X @ W_bad
except ValueError:
    print(f"shape error: inner dimensions {X.shape[-1]} and {W_bad.shape[0]} must match")
notation mapped to meaning and code (loss: ℝⁿ → ℝ, softmax: ℝᵏ → Δᵏ, a set comprehension, |S| = len(S)), beside an attention shape chain X ∈ ℝ^{B×T×d} → Q = X W_Q ∈ ℝ^{B×T×d} → QQᵀ ∈ ℝ^{B×T×T}, and a 9 × 8 weight that fails because the inner dimensions 8 and 9 don't match.
Diagram— notation mapped to meaning and code (loss: ℝⁿ → ℝ, softmax: ℝᵏ → Δᵏ, a set comprehension, |S| = len(S)), beside an attention shape chain X ∈ ℝ^{B×T×d} → Q = X W_Q ∈ ℝ^{B×T×d} → QQᵀ ∈ ℝ^{B×T×T}, and a 9 × 8 weight that fails because the inner dimensions 8 and 9 don't match.

The shapes flow as the notation says: (2, 5, 8) → (2, 5, 8) → (2, 5, 5), one T×T score matrix per sequence. The mis-declared weight fails immediately: inner dimensions 8 and 9 must match.

In production. Writing shapes next to every tensor, in comments, docstrings or type annotations (libraries such as jaxtyping let you annotate Float[Tensor, "batch seq dim"]), is one of the most effective habits in ML engineering. Most model bugs are shape bugs that happen to run: a sum over the wrong axis or a silent broadcast. A signature on paper catches them before they become a wrong number in a training curve.

From production

A paper defines the attention scores S = QKᵀ / √d with Q, K ∈ ℝ^{T×d}. Your implementation produces S with shape (d, d).

What did you most likely do?

  1. Computed QᵀK instead of QKᵀ: (d×T)(T×d) gives d×d, while the paper's QKᵀ is (T×d)(d×T) = T×T, one score per pair of tokens
  2. Nothing; S should be d × d
  3. Forgot to divide by √d
  4. Used the wrong random seed
Show the answer

A. Computed QᵀK instead of QKᵀ: (d×T)(T×d) gives d×d, while the paper's QKᵀ is (T×d)(d×T) = T×T, one score per pair of tokens — Reading the signature tells you S must be T×T (token-to-token). A d×d result means the transpose is on the wrong factor. Checking shapes against the notation catches this before any training run.

2. Σ and ∏: sums and products over indices

The question. How do you read Σᵢ₌₁ⁿ and ∏ᵢ?

Start from nothing. Σᵢ₌₁ⁿ xᵢ means "for i from 1 to n, add up xᵢ": a loop with an accumulator, or x.sum(). ∏ᵢ means the same with multiplication. Indices can range over sets (Σ_{x∈S}) or conditions (Σ_{i: yᵢ=1}). Two rules make most derivations easy: constants factor out (Σ c·xᵢ = c·Σ xᵢ), and the log of a product is a sum of logs: log ∏ pᵢ = Σ log pᵢ. That last identity is why language models minimize a sum of negative log-probabilities: the probability of a sentence is a product of per-token probabilities, which underflows to zero in floating point for long texts, while its log is a well-behaved sum.

python
import numpy as np

rng = np.random.default_rng(0)
token_probs = rng.uniform(0.05, 0.6, 2000)            # p(token_t | previous tokens) for a 2,000-token document
product = np.prod(token_probs)
log_sum = np.sum(np.log(token_probs))
print(f"product of probabilities: {product}  (underflowed to zero)")
print(f"sum of log-probabilities: {log_sum:.1f}  -> mean negative log-likelihood {-log_sum / len(token_probs):.3f} nats/token")

# the same sum written three ways
x = rng.standard_normal(5)
loop = 0.0
for i in range(len(x)):
    loop += 3 * x[i]
print(np.isclose(loop, 3 * x.sum()), np.isclose(loop, np.sum(3 * x)))

The product of 2,000 probabilities prints 0.0: it fell below the smallest positive float64 (about 5 × 10⁻³²⁴) partway through the document. The sum of their logs is an ordinary number, -2582.0, which gives a mean negative log-likelihood of 1.291 nats per token. The last line prints True True: the loop, the factored form and the vectorised form agree, which is "constants factor out" checked numerically.

log10 of the running product of per-token probabilities falling about 0.56 decades per token, crossing the smallest float64 (≈ 5e-324) so the product is exactly 0.0 from token 605 onward, while Σ log pₜ stays an ordinary −2,582.0.
Chart— log10 of the running product of per-token probabilities falling about 0.56 decades per token, crossing the smallest float64 (≈ 5e-324) so the product is exactly 0.0 from token 605 onward, while Σ log pₜ stays an ordinary −2,582.0.

The companion trick: log-sum-exp. Logs fix products, but softmax and its log need a sum of exponentials, and exponentials overflow. Large models routinely produce logits in the hundreds. The identity log Σ exp(zᵢ) = m + log Σ exp(zᵢ − m), with m = max zᵢ, is exact and never overflows:

python
import numpy as np

z = np.array([1000.0, 999.0, 998.0])                 # large logits are normal for big models
with np.errstate(over="ignore", invalid="ignore"):
    naive = np.log(np.sum(np.exp(z)))                # exp(1000) overflows to inf
m = z.max()
stable = m + np.log(np.sum(np.exp(z - m)))           # log Σ exp(z) = m + log Σ exp(z − m)
print("naive log-sum-exp:", naive)
print("stable log-sum-exp:", round(stable, 4))
print("log-softmax:", np.round(z - stable, 4))

The naive version returns inf. The shifted version returns 1000.4076, and log-softmax [-0.4076 -1.4076 -2.4076] follows by subtraction. Subtracting the maximum changes nothing mathematically, since it cancels between numerator and denominator, but it keeps every exponent at or below 0.

In production. Every deep learning framework implements softmax, log-softmax and cross-entropy with this shift internally (torch.logsumexp, torch.nn.functional.cross_entropy working directly on logits). That's why you pass logits to the loss rather than probabilities you computed yourself: computing log(softmax(z)) in two separate steps can produce log(0) = -inf and NaN gradients that a fused, stable implementation avoids.

From production

A training run's loss becomes NaN after a few thousand steps. The code computes probs = torch.softmax(logits, -1) and then loss = -torch.log(probs[range(n), y]).mean().

What's the most likely cause and fix?

  1. The learning rate is too low
  2. The batch size is odd
  3. Some probabilities underflow to exactly 0, so log gives -inf and the gradients become NaN; pass the logits to F.cross_entropy (or use log_softmax), which applies the log-sum-exp trick internally
  4. Softmax should be applied twice
Show the answer

C. Some probabilities underflow to exactly 0, so log gives -inf and the gradients become NaN; pass the logits to F.cross_entropy (or use log_softmax), which applies the log-sum-exp trick internally — Separate softmax-then-log loses the stability of the fused computation. log_softmax and cross_entropy work in log space, using the max-shift identity, so tiny probabilities never become log(0).

3. argmax, argmin, max and min

The question. What's the difference between max and argmax?

Start from nothing. max_x f(x) is the best value; argmax_x f(x) is where it's achieved (the input). Training is written as θ* = argmin_θ L(θ): we want the parameters, not the loss value. Greedy decoding is argmax_t p(t | context): we want the token, not its probability. In code: np.argmax returns an index; ties are broken by the first occurrence, which is a convention you should know when results look arbitrary.

python
import numpy as np

vocab = ["Paris", "London", "Rome", "Berlin"]
logits = np.array([3.1, 1.2, 0.4, 3.1])
probs = np.exp(logits) / np.exp(logits).sum()
print(f"max probability {probs.max():.3f}; argmax index {np.argmax(probs)} -> '{vocab[np.argmax(probs)]}' (tie: first wins)")

thetas = np.linspace(-3, 3, 601)
loss = (thetas - 1.25) ** 2 + 0.5
print(f"min loss {loss.min():.3f} at theta* = argmin = {thetas[np.argmin(loss)]:.2f}")

The best probability is 0.451, but argmax returns index 0 ("Paris"): the tie with "Berlin", which has the same logit, goes to the first occurrence. The loss curve's minimum value is 0.500, reached at θ* = 1.25. max and argmax answer different questions about the same curve.

Softmax is a "soft" argmax. argmax is a hard choice with no useful gradient. Nudge the inputs slightly and the output either doesn't change or jumps. Softmax with a temperature T interpolates: high T spreads probability out, and T → 0 approaches the one-hot argmax.

python
import numpy as np

logits = np.array([2.0, 1.5, 0.3])
for T in (1.0, 0.5, 0.1, 0.01):
    p = np.exp(logits / T - (logits / T).max())
    p /= p.sum()
    print(f"temperature {T:<5} softmax {np.round(p, 3)}")
print("argmax:", np.eye(3)[np.argmax(logits)])
softmax probabilities over three tokens (logits 2.0, 1.5, 0.3) at temperatures 1.0 (0.56, 0.34, 0.10), 0.5 (0.71, 0.26, 0.02) and 0.1 (0.99, 0.01, 0.00), approaching the one-hot argmax (1, 0, 0).
Chart— softmax probabilities over three tokens (logits 2.0, 1.5, 0.3) at temperatures 1.0 (0.56, 0.34, 0.10), 0.5 (0.71, 0.26, 0.02) and 0.1 (0.99, 0.01, 0.00), approaching the one-hot argmax (1, 0, 0).

At T = 1 the distribution is [0.559 0.339 0.102]. At 0.5 it sharpens to [0.714 0.263 0.024], at 0.1 it's [0.993 0.007 0.], and at 0.01 it's indistinguishable from argmax's one-hot [1. 0. 0.].

In production. This is exactly the temperature parameter in LLM APIs. Decoding divides the model's logits by T before sampling, so T = 0 (or "greedy") means argmax, and higher values sample more diverse tokens. Training always uses the smooth softmax, because gradients must flow. Inference chooses between argmax-like and sampling behaviour per use case: low temperature for extraction and code, higher for brainstorming.

Quick check

A paper writes θ* = argmin_θ Σᵢ ℓ(f_θ(xᵢ), yᵢ). What is θ*?

  1. The sum of all parameters
  2. The index of the worst training example
  3. The smallest possible loss value
  4. The parameter values that make the total loss over the training examples as small as possible
Show the answer

D. The parameter values that make the total loss over the training examples as small as possible — argmin returns the argument (the parameters) at which the sum of per-example losses is minimized, not the minimum value itself.

4. Reading a real formula

The question. Can you read the cross-entropy loss of a classifier and implement it?

Start from nothing. L(θ) = −(1/N) Σᵢ₌₁ᴺ Σₖ₌₁ᴷ yᵢₖ log p_θ(k | xᵢ). Read it outside in: average over N examples (1/N Σᵢ); inside, sum over K classes (Σₖ); yᵢₖ is 1 for the true class and 0 otherwise (one-hot), so the inner sum picks out the log-probability of the true class; the minus sign turns "maximize log-probability" into "minimize loss". With that reading, the code is two lines.

python
import numpy as np

rng = np.random.default_rng(1)
N, K = 4, 3
logits = rng.standard_normal((N, K))
y = np.array([0, 2, 1, 2])
p = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
Y = np.eye(K)[y]                                                  # one-hot y_ik
loss_formula = -(1 / N) * np.sum(Y * np.log(p))                   # literal translation of the double sum
loss_indexed = -np.mean(np.log(p[np.arange(N), y]))               # what the one-hot sum simplifies to
print(f"{loss_formula:.6f} {loss_indexed:.6f}")

Both versions print 0.978593: the literal double sum and the simplified "pick the true class's log-probability" agree, which confirms the reading.

Now the formula at the centre of every LLM. The attention operation from "Attention Is All You Need" (Vaswani et al., 2017) is written Attention(Q, K, V) = softmax(QKᵀ / √d) V. Read it the same way, outside in: a softmax of something, times V. The something is QKᵀ (all query–key dot products), scaled by 1/√d.

python
import numpy as np

rng = np.random.default_rng(3)
T, d = 4, 8
Q, K, V = (rng.standard_normal((T, d)) for _ in range(3))

def softmax(S):
    E = np.exp(S - S.max(axis=-1, keepdims=True))
    return E / E.sum(axis=-1, keepdims=True)

A = softmax(Q @ K.T / np.sqrt(d))                    # Attention(Q,K,V) = softmax(QKᵀ / √d) V
out = A @ V
print("weights shape", A.shape, "| each row sums to 1:", np.allclose(A.sum(axis=1), 1))
print("output shape", out.shape)
print("row 0 weights:", np.round(A[0], 3))
the attention formula as four steps with shapes for T = 4, d = 8: QKᵀ (4×8)(8×4) → 4×4, divided by √d, row-wise softmax (rows sum to 1), times V (4×4)(4×8) → 4×8; below, row 0's weights 0.105, 0.008, 0.017, 0.870.
Diagram— the attention formula as four steps with shapes for T = 4, d = 8: QKᵀ (4×8)(8×4) → 4×4, divided by √d, row-wise softmax (rows sum to 1), times V (4×4)(4×8) → 4×8; below, row 0's weights 0.105, 0.008, 0.017, 0.870.

The weights matrix is (4, 4) with every row summing to 1 (True), and the output is (4, 8), one new vector per token. Token 0 puts 0.87 of its attention on token 3 and very little elsewhere: each output row is a weighted average of the value vectors.

In production. Reading papers this way, by writing the shapes and a few lines of NumPy, is how engineers check that an implementation matches its paper before trusting it. Library code adds masking (causal attention sets future positions to −∞ before the softmax), multiple heads and fused kernels. The skeleton is still these three lines (Transformer architecture, in the LLM path).

Quick check

In softmax(QKᵀ / √d) V, along which axis is the softmax taken, and why?

  1. Over the batch, so batches sum to 1
  2. Over the feature dimension d
  3. Over the last axis (the keys) of the T×T score matrix, so each query's weights over all tokens sum to 1
  4. It doesn't matter which axis
Show the answer

C. Over the last axis (the keys) of the T×T score matrix, so each query's weights over all tokens sum to 1 — Each row is one query's scores against every key. Normalising across that row turns the scores into weights for a weighted average of the value vectors. The wrong axis would mix queries, and the attention would be meaningless.


Check yourself

  1. What does f: ℝⁿ → ℝ tell you about f?
  2. Translate {x ∈ S : x > 0} and |S| into Python.
  3. Why do models minimize a sum of log-probabilities instead of a product of probabilities?
  4. What's the difference between max_x f(x) and argmax_x f(x)?
  5. In the cross-entropy formula, what does the one-hot yᵢₖ do?
  6. Why do frameworks compute log-softmax as z − (m + log Σ exp(z − m)) rather than log(softmax(z))?
Answers
  1. It takes a vector of n real numbers and returns one real number.
  2. {x for x in S if x > 0} and len(S).
  3. Products of many probabilities underflow to zero; logs turn them into sums that stay well-scaled (and are easier to differentiate).
  4. The first is the best value; the second is the input that achieves it.
  5. It selects the log-probability of the true class, so the inner sum has a single non-zero term.
  6. exp of large logits overflows and tiny probabilities underflow to 0 (giving log 0 = −∞). Subtracting the max keeps every exponent ≤ 0 and keeps the computation in log space.

Exercise

Implement, from their formulas, (a) the mean squared error (1/N) Σᵢ (ŷᵢ − yᵢ)², (b) the log-likelihood of a dataset under a Bernoulli model with parameter q, Σᵢ [yᵢ log q + (1 − yᵢ) log(1 − q)], and (c) q* = argmax_q of that log-likelihood over a grid. Check that q* equals the fraction of ones in the data.

Solution
python
import numpy as np

rng = np.random.default_rng(2)
y_hat, y_true = rng.standard_normal(10), rng.standard_normal(10)
mse = (1 / len(y_true)) * np.sum((y_hat - y_true) ** 2)
print(f"MSE {mse:.4f}")

y = (rng.random(200) < 0.3).astype(float)
def log_likelihood(q):
    return np.sum(y * np.log(q) + (1 - y) * np.log(1 - q))
grid = np.linspace(0.01, 0.99, 981)
q_star = grid[np.argmax([log_likelihood(q) for q in grid])]
print(f"argmax q = {q_star:.3f}, fraction of ones = {y.mean():.3f}")

The grid search lands on the fraction of ones in the data, which is the maximum-likelihood estimate you'd get by setting the derivative to zero (the Probability lesson derives it). Translating notation directly into code like this is the fastest way to check you've read a formula correctly.


This unlocks:

  • Linear algebra — vectors and matrices, written in this notation.
  • Probability and statistics — distributions, likelihoods and expectations as sums.

Sources: Goodfellow, Bengio & Courville, Deep Learning, Notation section (2016); Deisenroth, Faisal & Ong, Mathematics for Machine Learning (2020), ch. 1–2; Strang, Introduction to Linear Algebra.