Appearance
12.2.2 — Matrices as Machines
By March, Cartwise's fraud team had noticed that fraud is not one thing. There are at least three distinct patterns:
- Big order, new account: somebody opens an account and immediately spends a lot.
- Cross-border: the card was issued in one country and the goods ship to another.
- Card testing: a tiny purchase, often from a mismatched country, made only to check whether a stolen card number works before using it for something big.
Each pattern can be scored with its own dot product from 12.2.1: its own weight vector, dotted with the order's feature vector, plus its own bias. So every order now needs three dot products. And the checkout handles about a thousand orders a minute at peak, so that is three thousand dot products a minute, and a real model would look at hundreds of patterns, not three.
You could write a loop. But there is a much better way to think about it, and it is the way every ML library, every graphics card and every neural network actually works: stack the weight vectors into a grid, and treat the grid as a single machine that turns one vector into another. That grid is a matrix.
1. What a matrix is
A matrix is a rectangular grid of numbers arranged in rows and columns. Here are the three fraud patterns stacked as the rows of a matrix called W (for weights). Each column corresponds to one feature of the order: amount in thousands of rupees, whether the account is new (1 or 0), and whether the countries differ (1 or 0).
W = \begin{bmatrix} 0.4 & 3.0 & 0.2 \\ 0.0 & 1.0 & 2.0 \\ -0.2 & 0.0 & 1.0 \end{bmatrix} \begin{matrix} \leftarrow \text{big order, new account} \\ \leftarrow \text{cross-border} \\ \leftarrow \text{card testing} \end{matrix}
The first row is exactly the weight vector from 12.2.1. The second row puts most of its weight on the countries differing. The third row has a negative weight on the amount, so a smaller order scores higher, and a positive weight on a country mismatch: tiny orders across borders, which is what card testing looks like.
The shape of a matrix is its number of rows followed by its number of columns, written with a times sign. W is a 3 \times 3 matrix, read aloud as "three by three". A matrix with 2 rows and 5 columns is "2 by 5". The order is always rows first, and you will be checking shapes constantly once you write ML code, so it is worth making that a reflex now. A single entry is named by its row and column: W_{1,2} (read "W one two") is the entry in row 1, column 2, which here is 3.0.
2. A matrix times a vector: many dot products at once
Multiplying a matrix by a vector means taking the dot product of each row of the matrix with the vector. Each row gives one number, and those numbers form a new vector.
Take the order from 12.2.1: ₹12,000, new account, same country, so \mathbf{x} = (12, 1, 0).
Follow the colours. The blue row is the "big order, new account" pattern: 0.4 \times 12 + 3.0 \times 1 + 0.2 \times 0 = 7.8. The green row is cross-border: 0 + 1.0 + 0 = 1.0. The amber row is card testing: -2.4 + 0 + 0 = -2.4, strongly negative because this is a large order, the opposite of a test purchase. Then each pattern adds its own bias from \mathbf{b} = (-6, -1, 0), giving final scores of (1.8,\ 0.0,\ -2.4). The order looks like the first pattern, a little like the second, and not at all like the third.
Written compactly, the whole calculation is:
\mathbf{y} = W\mathbf{x} + \mathbf{b}
Read it as "y equals W times x, plus b". That one line is the most important formula in deep learning. It is exactly what one layer of a neural network computes, which Chapter 12.5.1 will build. A layer is a matrix of weights and a vector of biases, and "running the layer" means doing this multiplication.
A matrix is a machine that changes the size of a vector
Notice what went in and what came out. A 3-number vector went in, and a 3-number vector came out, because W has 3 columns (one per input feature) and 3 rows (one per pattern). But the two counts do not need to match. If Cartwise added a fourth pattern, W would gain a fourth row and become 4 \times 3, and the output would have 4 numbers. If it added two more features, W would gain two columns.
So the rule is: a matrix with shape m \times n takes in a vector of n numbers and gives back a vector of m numbers. The number of columns must match the input. The number of rows decides the output. That is why it is useful to think of a matrix as a machine rather than as a table. It has an input slot of a fixed size, and an output slot of a fixed size, and the numbers inside decide what it does to whatever passes through.
A neural network is several of these machines connected end to end. The first might turn a 300-number description of an order into 128 numbers, the next turns those 128 into 64, and the last turns 64 into 1, the fraud probability. Each machine reshapes the information into a form the next one can use.
3. What a matrix does geometrically
There is one more way to see a matrix, and it builds the intuition you will need for embeddings in Chapter 12.6. Take a 2 \times 2 matrix, so that it turns 2-number vectors into 2-number vectors, and watch what it does to the plane.
A = \begin{bmatrix} 2 & 1 \\ 0 & 1 \end{bmatrix}
Feed it the two simplest possible vectors: (1, 0), the arrow one step to the right, and (0, 1), the arrow one step up.
- A(1, 0) = (2 \times 1 + 1 \times 0,\ 0 \times 1 + 1 \times 0) = (2, 0)
- A(0, 1) = (2 \times 0 + 1 \times 1,\ 0 \times 0 + 1 \times 1) = (1, 1)
Look at those two answers and compare them with the columns of A. The first column of A is (2, 0), and the second is (1, 1). The columns of a matrix are exactly where the "one step right" and "one step up" arrows end up. Once you know where those two land, you know where everything lands, because every other vector is just some amount of "right" plus some amount of "up".
In the "before" panel, the purple square is the region between the two step arrows. In the "after" panel, the blue arrow has been stretched to length 2, the green arrow has been tilted over to (1, 1), and the square they enclose has turned into a slanted parallelogram. Every point in the plane has moved in the same coordinated way: stretched to the right and slanted. Different matrices do different things: some rotate, some stretch, some squash the plane flat onto a line.
Why should an AI engineer care about a slanted square? Because this is what a neural network layer is doing to data. Its matrix moves every input point to a new position. Training chooses the numbers in the matrix so that, after a few layers of moving things around, the points that should be classified the same way end up close together and the ones that should be separated end up far apart. When fraud and genuine orders are hopelessly mixed in the raw features, the layers keep reshaping the space until a straight line can separate them. That is the geometric heart of deep learning. Volume II's chapter on matrices goes further if you want the full theory.
4. A matrix times a matrix: a whole batch at once
Scoring one order at a time wastes the hardware. In practice you score many orders at once, called a batch. Put each order in as one row of a matrix X:
X = \begin{bmatrix} 12 & 1 & 0 \\ 2 & 0 & 1 \\ 0.5 & 0 & 1 \end{bmatrix} \begin{matrix} \leftarrow \text{order 1: 12 thousand, new account} \\ \leftarrow \text{order 2: 2 thousand, cross-border} \\ \leftarrow \text{order 3: 500, cross-border} \end{matrix}
Now we want every order scored against every pattern: 3 orders times 3 patterns is 9 dot products. Multiplying two matrices does exactly that. Entry (row i, column j) of the result is the dot product of row i of the first matrix with column j of the second.
There is one wrinkle. Our orders are rows of X and our patterns are rows of W, but matrix multiplication pairs rows of the first with columns of the second. So we need to flip W so that its rows become columns. Flipping a matrix across its diagonal like this is called the transpose, written W^\top (read "W transpose"). Row 1 becomes column 1, row 2 becomes column 2, and so on. Then:
\text{scores} = X W^\top + \mathbf{b}
Here is the whole computation in NumPy:
python
import numpy as np
W = np.array([[ 0.4, 3.0, 0.2], # big order, new account
[ 0.0, 1.0, 2.0], # cross-border
[-0.2, 0.0, 1.0]]) # card testing
b = np.array([-6.0, -1.0, 0.0])
x = np.array([12, 1, 0])
print(W @ x + b) # (1) → [ 1.8 0. -2.4]
X = np.array([[12, 1, 0],
[ 2, 0, 1],
[0.5, 0, 1]])
print(X.shape, W.T.shape) # (2) → (3, 3) (3, 3)
print(X @ W.T + b) # (3)
# → [[ 1.8 0. -2.4]
# [-5. 1. 0.6]
# [-5.6 1. 0.9]]Line (1) is the single-order calculation from section 2: matrix times vector, plus bias, giving the same (1.8, 0, -2.4). Line (2) prints the shapes, the habit that will save you more debugging time than any other in ML code. .T is NumPy's transpose. Line (3) scores all three orders against all three patterns in one operation.
Read the result row by row. Order 1 is the same as before. Order 2, the ₹2,000 cross-border order, scores −5.0 on "big order" (it is neither big nor from a new account), 1.0 on cross-border, and 0.6 on card testing. To check one entry by hand, the cross-border score for order 2 is 2 \times 0.0 + 0 \times 1.0 + 1 \times 2.0 - 1 = 1.0. Order 3, the tiny ₹500 cross-border order, has the highest card-testing score of all three, 0.9, because the card-testing row rewards small amounts and mismatched countries. The bias vector \mathbf{b} has 3 numbers and the result has 3 columns, and NumPy adds the same bias to every row automatically. That automatic stretching of a smaller array across a bigger one is called broadcasting.
The shape rule, and the error you will see a hundred times
Matrix multiplication only works when the inner sizes match:
(m \times n) \times (n \times p) \to (m \times p)
Read it as: an m-by-n matrix times an n-by-p matrix gives an m-by-p matrix. The two $n$s in the middle must be equal, because each dot product pairs a row of length n with a column of length n. Those middle numbers then disappear, and the outer numbers give the shape of the result.
Our scoring was (3 \text{ orders} \times 3 \text{ features}) \times (3 \text{ features} \times 3 \text{ patterns}) = (3 \text{ orders} \times 3 \text{ patterns}). If Cartwise had a batch of 1,000 orders, X would be 1000 \times 3, and the result 1000 \times 3: one row of scores per order.
When the inner sizes do not match, NumPy refuses, and the message looks like this:
text
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0,
with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2 is different from 3)It looks intimidating, but it is always the same complaint: the number of columns on the left does not equal the number of rows on the right. Nine times out of ten, the fix is a missing or extra transpose. Print both shapes, write them next to each other, and check that the middle numbers agree.
5. Tensors: when two directions are not enough
Real ML data often needs more than rows and columns. A batch of 32 support emails, each cut into 512 word pieces, each piece described by 768 numbers, is a three-way grid with shape 32 \times 512 \times 768. A grid of numbers with any number of directions is called a tensor: a vector is a 1-direction tensor, a matrix is a 2-direction tensor, and the email batch is a 3-direction tensor. Each direction is called an axis. This is where the library name PyTorch's torch.Tensor and the name TensorFlow come from.
You do not need any new maths for tensors. The operations are the same multiplications, applied separately along the extra axes. The email batch is 32 separate 512 \times 768 matrices, processed at the same moment. What you do need is the shape habit: in ML code, writing the expected shape as a comment next to each line, like # (32, 512, 768), is how experienced engineers keep their sanity.
6. Why graphics chips run AI
Count the work in one layer of a mid-sized LLM. A matrix of shape 4096 \times 4096 times a vector of 4,096 numbers is 4,096 dot products, each with 4,096 multiplications and additions. That is about 16.8 million multiply-and-add steps for one layer, for one word. A model has dozens of such matrices per layer and dozens of layers.
The good news is that every one of those dot products is independent of the others. Row 1's dot product does not need to wait for row 2's. So the work can be split across thousands of small calculators running at the same moment. A typical processor (CPU) has a few dozen powerful cores, designed to run complicated code one step after another very quickly. A graphics processor (GPU) has thousands of simpler cores, originally designed to compute the colour of millions of screen pixels at the same moment, which is also a giant pile of independent multiply-adds. That is why the hardware built for video games became the hardware for AI, and Chapter 12.5.7 and 12.12.3 are about using it well.
Recall
- A matrix is a grid of numbers. Its shape is rows × columns, always rows first.
- Matrix times vector = the dot product of every row with the vector. An m \times n matrix turns n numbers into m numbers, and \mathbf{y} = W\mathbf{x} + \mathbf{b} is exactly one neural network layer.
- Geometrically, a matrix moves every point in space; its columns show where the step arrows land. Layers of a network reshape space until classes can be separated.
- Matrix times matrix scores a whole batch at once. The transpose W^\top swaps rows and columns. Shapes must obey (m\times n)(n\times p)\to(m\times p), and a mismatch error almost always means a missing transpose. Broadcasting adds a bias to every row automatically.
- A tensor is a grid with any number of axes. All the dot products in a layer are independent, which is why GPUs, with thousands of simple cores, run AI so well.
Self-test: A layer takes 300 features in and produces 128 numbers out. What is the shape of its weight matrix, and of its bias vector? You multiply a batch of shape (64, 128) by a matrix of shape (64, 10) and get an error. What is the fix, and what is the shape of the result? Where does the vector (0, 1) land under the matrix A above?
Next: 12.2.3 Derivatives, Gradients and the Chain Rule — we can now compute a model's output; to train it, we need to know which way to nudge each weight to make the output less wrong, and that is what a derivative tells you.