9.5. Fundamental Matrix Subspaces¶
There are four possible fundamental vector subspaces of an
matrix
. They are called the column
space, the null space, the row space, and the left null space. The
subspaces are found from rank determined regions of the SVD factors. The
regions are identified in figure Fig. 9.8 as
,
,
, and
.

Fig. 9.8 The columns of and rows of
from
the SVD constitute the four fundamental matrix subspaces of the
matrix. Which subspace a row or column is part of depends on if it is
multiplied by a nonzero or zero singular value.
-
Column Space
: The first
columns of
are the basis vectors for the vector space spanned by the columns of
.
-
Null Space
: Rows
to
of
are the transpose of the basis vectors for the space spanned by the column vectors
such that
. Note that square, full-rank matrices do not have a null space. MATLAB’s
null
function returns these vectors.
-
Row Space
: The first
rows of
are the basis vectors for the space spanned by the rows of
.
-
Left Null Space
: Columns
to
of
are the transpose of the basis vectors for the space spanned by the row vectors
such that
. Note that square, full-rank matrices do not have a left null space.
9.5.1. Column Space¶
The column space of a matrix , which we denote as
, is the vector space spanned by all
independent columns of
. If
is a square
full rank matrix, then it is all of the columns of
. For
singular or under-determined matrices the column space is the set of
columns with nonzero pivots during elimination. The vectors in the
column space can also be referred to as basis vectors. Of course, an
orthogonal set of basis vectors is preferred and can be found using the
algorithms mentioned in Finding Orthogonal Basis Vectors.
Rank and Column Space Dimension
The rank of the matrix is the same as the dimension of the column space, which is the number of columns in the column space.
When there is an exact solution to the equation , we say that
is “in the column space of
” because it is a linear combination of the independent columns
of
. The vector
tells us the coefficients for the
linear combination of the columns that are needed to get to
.
Here is a singular matrix example, which shows a case where the column
space is a subset of the columns of .
>> A(:,3) = -2*A(:,1) - A(:,2)
A =
-1 4 -2
3 0 -6
6 2 -14
>> rank(A)
ans =
2
>> rref(A)
ans =
1 0 -2
0 1 -1
0 0 0
>> [U, ~, ~] = svd(A);
>> rank([U(:,1:2) A(:,1:2)])
ans =
2
Notice from the output of rref
that the first two columns are pivot
columns. Thus we can see, as was designed from constructing the singular
matrix, that the first two columns span the column space.
Then from the SVD, the first two columns of provides an
orthogonal basis for the column space. As we did in Exact Solution or Approximation?,
we verified that the first two columns of
and
are in the same column space because the rank of the
augmented matrix of the four columns is still two. The subspace here is
a two dimensional plane that is slanted relative to the
coordinate system.
9.5.2. Null Space¶
The null space (right null space) of a matrix , which we
denote as
, is the vector space spanned by
all column vectors
that satisfy the matrix equation
. Square, singular matrices
and under-determined matrices have a
null space. The number of vectors in the null space is the number of
dependent columns (
size(A, 2) - rank(A)
). If is a
square full rank matrix, then the null space is an empty set.
The null
function returns the normalized basis vectors of a matrix’s
null space. MATLAB uses the SVD to find the null space from the last
columns of
corresponding to the singular
values equal to zero.
The null space vectors can also be found by elimination from the reduced row echelon form of the matrix as was done in Reduced Row Echelon Form.
Here is an example of a singular matrix.
% Find a random, singular matrix
>> A(:,3) = 4*A(:,1) + 2*A(:,2)
A =
1 7 18
1 5 14
6 -2 20
>> rref(A)
ans =
1 0 4
0 1 2
0 0 0
The third column from the RREF shows the dependent relationship of the
columns of . The third column is four of the first column
plus two of the second column.
If we consider the SVD of a singular system as , then we can divide the
,
, and
matrices into blocks related to
nonzero and zero valued singular values.
The last columns of
is the null space
because
.
>> x = [4; 2; -1];
>> A*x
ans =
0
0
0
>> x/norm(x)
ans =
0.8729
0.4364
-0.2182
>> [~,~,V] = svd(A);
>> V(:,3) % = null(A)
ans =
0.8729
0.4364
-0.2182
The null space of an under-determined matrix is at least ()
columns. Remember from Under-determined Systems that under-determined
systems have an infinite set of solutions. The RREF expresses the
solution in terms of the general and particular solutions, while the
null space from the SVD is the least squares solution. Although the
solutions are different, they are both correct.
>> A = randi(20, 3, 5) - 8;
>> [~, S, V] = svd(A);
>> nullA = V(:, end-1:end);
>> A*nullA
ans =
1.0e-14 *
-0.0583 0.0992
0.0444 -0.0666
-0.0722 -0.1568
9.5.3. Row Space¶
The row space of a matrix , which we denote as
, is the column space of
. It is the first
columns of
from the SVD.
9.5.4. Left Null Space¶
The left null space of a matrix are the set of row
vectors,
, that satisfy the relationship
. It is also the right null space of
. The left null space of
may be found
from the SVD as the transpose of the last (
) columns of
. It is more common in MATLAB to compute it as the
transpose of the right null space of
,
Y = null(A’)’
. The left null space exists for singular and
over-determined matrices.
9.5.5. Orthogonality of Spaces¶
The four fundamental vector subspaces form interesting orthogonality relationships. Vectors in the null space are orthogonal to vectors in the row space. Similarly, vectors in the column space are orthogonal to vectors in the left null space.
We will use a under-determined matrix to illustrate
orthogonality between the null space and row space.
>> A = randi(15, 3, 5) - 6;
>> nullSpace = null(A);
>> size(nullSpace)
ans =
5 3
>> rank(A)
ans =
3
% The row Space is all columns of A^T.
>> rowSpace = A';
% Check for orthogonality with a dot product.
>> nullSpace'*rowSpace
ans =
1.0e-14 *
0 0 0
0 -0.0444 0.1110
% The left null space is empty because the column space vectors
% are full rank.
>> left_null = null(A')
left_null =
3x0 empty double matrix
Here is a matrix of rank 1 to demonstrate
orthogonality of the column space and the left null space.
>> a = randi(5, 3, 1);
>> a2 = a.*2;
>> a3 = a.*3;
>> A = [a a2 a3];
>> rank(A)
ans =
1
>> colSpace = A(:,1);
>> left_null = null(A');
>> colSpace'*left_null
ans =
1.0e-15 *
-0.1943 0.4441