A matrix is a 2D grid of numbers — rows and columns. In ML, your entire dataset is a matrix: each row a sample, each column a feature. A model's prediction is what happens when that matrix is combined with a weight vector through matrix-vector multiplication.
Your Dataset, As a Matrix
This \(3 \times 2\) matrix (\(m \times n\): \(m\) rows/samples, \(n\) columns/features) represents 3 houses, each with square-footage and bedroom-count as features — exactly what X.shape reports in scikit-learn.
Matrix-Vector Multiplication — How a Linear Model Actually Predicts
Each output value is the dot product of one row of A with the vector x.
Formula
\(A_{ij}\) is the entry in row \(i\), column \(j\) of matrix \(A\). Each output entry is literally a dot product between one row of \(A\) and the vector \(\vec{x}\) — this is exactly how a linear layer computes predictions for every sample in one matrix operation, instead of looping row by row.
import numpy as np
A = np.array([[1, 2],
[3, 4]])
x = np.array([5, 6])
result = A @ x # matrix-vector multiplication (@ is the matmul operator)
print(result) # [17 39]Why This Matters for ML
- Predicting for an entire dataset at once —
predictions = X @ weights— computes every sample's prediction in a single vectorized operation, instead of a slow Python loop - A neural network layer is just repeated matrix multiplication followed by an activation function
- PCA relies entirely on matrix operations — the covariance matrix and its eigenvectors
Common Mistakes
- Trying to multiply matrices with incompatible shapes — the number of columns in the first matrix must equal the number of rows in the second (or the length of the vector).
- Assuming matrix multiplication is commutative — in general, \(AB \neq BA\).
Interview Relevance
Q: "Why is it faster to predict with matrix multiplication than looping through rows in Python?" NumPy's matrix multiplication runs in optimized, compiled C/BLAS code and processes the whole computation in one call, avoiding Python's per-iteration interpreter overhead — the same vectorization principle from NumPy for ML.
Practice Question
Given \(A = \begin{bmatrix}2 & 0\\0 & 3\end{bmatrix}\) and \(\vec{x}=[4, 5]\), compute \(A\vec{x}\) by hand, then verify with NumPy.
