6.11. Over-determined Systems and Vector ProjectionsΒΆ

The over-determined matrix equation of the form \mathbf{A}\,\bm{x} = \bm{b} has more equations than unknown variables (m > n). A common situation where an over-determined system occurs is in the results of an experiment. The experiment may be repeated many times as a control variable is adjusted. Thus a researcher may have many more equations relating the inputs to the outputs than unknown variables.

It is required that \text{rank}(\mathbf{A}) = n to find a solution. A unique solution exist when all row equations are consistent, which we formally describe as when \bm{b} is in the column space (span) of \bf{A}. But otherwise, we can only approximate the solution.

We can test if \bm{b} is in the column space of \bf{A} by comparing the rank of the augmented matrix of both \bf{A} and \bm{b} to the rank of \bf{A}, \text{rank}([\mathbf{A}\;\bm{b}]) = \text{rank}(\bf{A}). If the rank of the augmented matrix and \bf{A} are the same, then \bm{b} is in the column space of \bf{A}.

When \bm{b} is not in the column space of \bf{A}, the only solution available is an approximation. In the following example, we use the rank test and see that \bm{b} is in the column space of \bf{A}. Then after a change to the \bf{A} matrix, we see from the rank test that \bm{b} is no longer in the column space of \bf{A}

>> A
A =
     5     3
     6    -3
    -2    -1
     6     2
>> b
b =
     7
    15
    -3
    10
% Rank test:
% b is in column space
>> rank([A b])
ans =
     2
>> rank(A)
ans =
     2

% Change A
>> A(4,:) = [5 1];
A =
     5     3
     6    -3
    -2    -1
     5     1
% Rank test:
% b is not in column space
>> rank([A b])
ans =
     3
>> rank(A)
ans =
     2

We find the approximation solution by projection, which is our next topic.