Matrices

Coordinate representation of linear maps; why matrix multiplication is defined the way it is.

about 8 min

Start from a problem

Problem

A linear map is determined by the images of a basis. But to evaluate a map, store it, and compose two maps, we need a notation that turns these operations into mechanical arithmetic.

Matrices are that notation. The "row times column" rule for multiplication looks odd; this section shows it is the only natural choice.

Observe

In the previous experiment you dragged $T(\mathbf e_1)$ and $T(\mathbf e_2)$ , and the matrix $A$ below changed with them. Watch how: the coordinates of $T(\mathbf e_1)$ are the first column, those of $T(\mathbf e_2)$ the second. A matrix is nothing but the two image vectors written side by side.

Interactive experimentLinear transformation

A=\begin{pmatrix}\textcolor{#c2452d}{1}&\textcolor{#1f7a4d}{0}\\\textcolor{#c2452d}{0}&\textcolor{#1f7a4d}{1}\end{pmatrix}

Determinant (signed area) 1

Eigenvalues 1, 1

Show eigenvectors

Conjecture

Since a linear map is determined by the images of a basis, writing those images as columns of a table records the map completely. "Matrix times vector" should be "evaluate the map", and "matrix times matrix" should be "compose the maps". If so, the odd-looking definition of matrix multiplication stops being odd.

Definitions

Definition 4.1Matrix of a linear map

Let $T:\mathbb R^n\to\mathbb R^m$ be linear and $\mathbf e_1,\dots,\mathbf e_n$ the standard basis. The matrix of $T$ is the $m\times n$ array $A$ whose $j$ -th column is the coordinate vector of $T(\mathbf e_j)$ : $A=\bigl[\,T(\mathbf e_1)\ \ T(\mathbf e_2)\ \cdots\ T(\mathbf e_n)\,\bigr].$

Definition 4.2Matrix times vector

For $A=[a_{ij}]$ and $x=(x_1,\dots,x_n)$ , $Ax$ is the linear combination of the columns of $A$ : $Ax=x_1A_{\cdot1}+x_2A_{\cdot2}+\cdots+x_nA_{\cdot n},\qquad\text{i.e.}\ (Ax)_i=\sum_{j=1}^na_{ij}x_j.$

Proposition 4.1

If $A$ is the matrix of $T$ , then $T(x)=Ax$ for all $x$ .

Proof

$x=\sum x_j\mathbf e_j$ , so by linearity $T(x)=\sum x_jT(\mathbf e_j)=\sum x_jA_{\cdot j}=Ax$ .

Why matrix multiplication is defined this way

Definition 4.3Matrix product

For $A$ of size $m\times n$ and $B$ of size $n\times p$ , $AB$ is the $m\times p$ matrix whose $j$ -th column is $A(B_{\cdot j})$ , i.e. $(AB)_{ij}=\sum_{k=1}^na_{ik}b_{kj}.$

Theorem 4.2Matrix product = composition

If $S:\mathbb R^p\to\mathbb R^n$ has matrix $B$ and $T:\mathbb R^n\to\mathbb R^m$ has matrix $A$ , then $T\circ S$ has matrix $AB$ .

Proof

$T\circ S$ is linear (check both properties). Its $j$ -th column is $(T\circ S)(\mathbf e_j)=T(S(\mathbf e_j))=T(B_{\cdot j})=A(B_{\cdot j})$ , the $j$ -th column of $AB$ .

That is the whole origin of "row times column". Immediate consequences:

Associativity $(AB)C=A(BC)$ , because composition of maps is associative. No sums need expanding.
Non-commutativity $AB\ne BA$ : rotating then stretching differs from stretching then rotating. Try it in the experiment.
The identity matrix $I$ is the identity map, $AI=IA=A$ .

The determinant

In the experiment the unit square becomes a parallelogram whose area depends on the matrix.

Definition 4.42×2 determinant

$\det\begin{pmatrix}a&b\\c&d\end{pmatrix}=ad-bc.$

Theorem 4.3The determinant is signed area

The signed area of the parallelogram spanned by $T(\mathbf e_1)=(a,c)$ and $T(\mathbf e_2)=(b,d)$ is $ad-bc$ . It is negative exactly when $T$ reverses orientation (the turn from $\mathbf e_1$ to $\mathbf e_2$ changes from counter-clockwise to clockwise).

Proof

Let $u=(a,c)$ , $v=(b,d)$ , and $u^\perp=(-c,a)$ the counter-clockwise quarter-turn of $u$ . The area is $|u|$ times the signed projection of $v$ onto $u^\perp$ : $|u|\cdot\dfrac{v\cdot u^\perp}{|u^\perp|}=v\cdot u^\perp=-bc+ad$ . The projection is positive when $v$ lies on the counter-clockwise side of $u$ .

Theorem 4.4Multiplicativity

$\det(AB)=\det A\cdot\det B$ .

Proof

Geometrically: $B$ turns the unit square into a parallelogram of area $\det B$ , and $A$ multiplies the area of any region by $\det A$ (it sends every small grid cell to the same parallelogram, and any region's area is a limit of sums of cells). So $AB$ multiplies area by $\det A\det B$ ; orientation signs multiply likewise.

Proposition 4.5Invertibility

$A$ is invertible iff $\det A\ne0$ , and then $A^{-1}=\dfrac1{ad-bc}\begin{pmatrix}d&-b\\-c&a\end{pmatrix}$ .

$\det A=0$ is the "Collapse" preset: the plane is flattened onto a line (or a point), information is lost, nothing can be undone.

Common mistake

$AB\ne BA$ . Matrix multiplication is composition: $AB$ means apply $B$ first, then $A$ (acting on a vector to the right, $B$ touches it first). Writing the factors in the wrong order is the most common mistake.

Application

ApplicationRotation matrices

Counter-clockwise rotation by $\theta$ : $\mathbf e_1\mapsto(\cos\theta,\sin\theta)$ , $\mathbf e_2\mapsto(-\sin\theta,\cos\theta)$ ; as columns, $R_\theta=\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix},\qquad\det R_\theta=1.$ Expanding $R_\alpha R_\beta=R_{\alpha+\beta}$ yields the angle-addition formulas: matrix multiplication turns them into the triviality "rotate by $\beta$ , then by $\alpha$ ".

Remark

One sentence suffices: the columns of a matrix are where the basis vectors go. Every definition and theorem about matrices can be re-derived from that sentence and linearity. The next section, Eigenvalues, asks: is there a vector whose direction the map leaves unchanged?

Exercises

Compute

\det\begin{pmatrix}2&1\\3&4\end{pmatrix}

The matrix (in the standard basis) of the counter-clockwise rotation by

90^\circ