The Matrix Inverse

In the previous section, we saw how the identity matrix $I$ acts like the number 1 in matrix multiplication: multiplying a matrix $A$ by $I$ (where the dimensions are compatible) leaves $A$ unchanged. That is, $AI = A$ and $IA = A$. This property is essential when thinking about how to "undo" matrix multiplication, much like we use division or reciprocals to undo multiplication with regular numbers.

Consider a simple algebraic equation like $5x = 10$. To find $x$, you would divide both sides by 5, or equivalently, multiply both sides by the reciprocal of 5, which is $1/5$ or $5^{-1}$. $(5^{-1}) \times 5x = (5^{-1}) \times 10$ $1 \times x = 2$ $x = 2$ Multiplying by the reciprocal $5^{-1}$ effectively isolates $x$ by turning the coefficient $5$ into $1$.

Can we find a similar concept for matrices? If we have the matrix equation $Ax = b$, can we find a matrix that acts like the "reciprocal" or "inverse" of $A$? If such a matrix exists, we could multiply both sides of the equation by it to potentially isolate the vector $x$.

This brings us to the definition of the matrix inverse. For a square matrix $A$ (meaning it has the same number of rows and columns, say $n \times n$), its inverse, denoted as $A^{-1}$, is another $n \times n$ matrix such that when multiplied by $A$ (in either order), the result is the $n \times n$ identity matrix $I$.

Formally, if $A$ is an $n \times n$ square matrix, its inverse $A^{-1}$ is an $n \times n$ matrix satisfying: $AA^{-1} = I \quad \text{and} \quad A^{-1}A = I$

Properties of the Matrix Inverse

1. Must be Square: Only square matrices can have an inverse. If a matrix is not square (e.g., $3 \times 2$), it doesn't have an inverse in this sense. Think about why: for both $AA^{-1}$ and $A^{-1}A$ to be defined and equal to the same square identity matrix, $A$ and $A^{-1}$ must both be square with the same dimensions.
2. Existence is Not Guaranteed: Not every square matrix has an inverse. Square matrices that do have an inverse are called invertible or non-singular. Square matrices that do not have an inverse are called non-invertible or singular. We'll explore the conditions for invertibility in the next section.
3. Uniqueness: If a matrix $A$ does have an inverse, that inverse ($A^{-1}$) is unique. There's only one matrix that satisfies the definition.
4. Inverse of the Inverse: If $A$ is invertible, then its inverse $A^{-1}$ is also invertible, and its inverse is the original matrix $A$. $(A^{-1})^{-1} = A$
5. Inverse of a Product: If $A$ and $B$ are both invertible square matrices of the same size, then their product $AB$ is also invertible. The inverse of the product is the product of their inverses in reverse order: $(AB)^{-1} = B^{-1}A^{-1}$ The order reversal is important because matrix multiplication is generally not commutative ($AB \neq BA$).

Why is the Inverse Useful for $Ax = b$?

The idea of the matrix inverse gives us a way to solve the system $Ax = b$. If $A$ is invertible, we can multiply both sides of the equation on the left by $A^{-1}$: $A^{-1}(Ax) = A^{-1}b$ Using the associative property of matrix multiplication: $(A^{-1}A)x = A^{-1}b$ Since $A^{-1}A = I$ (the identity matrix): $Ix = A^{-1}b$ And because the identity matrix $I$ times any vector $x$ is just $x$: $x = A^{-1}b$ This looks like a direct way to find the solution vector $x$: just find the inverse of $A$ and multiply it by the vector $b$.

While this is mathematically correct and provides a clear algebraic solution path, calculating the inverse $A^{-1}$ explicitly just to compute $x = A^{-1}b$ is often not the most computationally efficient or numerically stable method for solving systems of equations in practice, especially for large matrices. Libraries like NumPy typically use more effective algorithms (which we'll see shortly). However, understanding the inverse is fundamental to grasping the properties of linear systems and matrix operations.

Before we jump into calculating inverses with NumPy, we need to understand when a matrix actually has an inverse. What makes a square matrix invertible or singular? That's what we'll cover next.