6.11. Over-determined Systems and Vector Projections¶
The over-determined matrix equation of the form \(\mathbf{A}\,\bm{x} = \bm{b}\) has more equations than unknown variables (\(m > n\)). A common situation where an over-determined system occurs is in the result of an experiment. The experiment may be repeated many times as a control variable is adjusted. Thus, a researcher may have many more equations relating the inputs to the outputs than unknown variables.
It is required that \(\text{rank}(\mathbf{A}) = n\) to find a solution. A unique solution exists when all row equations are consistent, which we formally describe as when \(\bm{b}\) is in the column space (span) of \(\bf{A}\). But otherwise, we can only approximate the solution.
We can test if \(\bm{b}\) is in the column space of \(\bf{A}\) by comparing the rank of the augmented matrix of both \(\bf{A}\) and \(\bm{b}\) to the rank of \(\bf{A}\), \(\text{rank}([\mathbf{A}\;\bm{b}]) = \text{rank}(\bf{A})\). If the rank of the augmented matrix and \(\bf{A}\) are the same, then \(\bm{b}\) is in the column space of \(\bf{A}\).
When \(\bm{b}\) is not in the column space of \(\bf{A}\), the only solution available is an approximation. In the following example, we use the rank test and see that \(\bm{b}\) is in the column space of \(\bf{A}\). Then, after a change to the \(\bf{A}\) matrix, we see from the rank test that \(\bm{b}\) is no longer in the column space of \(\bf{A}\)
>> A
A =
5 3
6 -3
-2 -1
6 2
>> b
b =
7
15
-3
10
% Rank test:
% b is in the column space of A
>> rank([A b])
ans =
2
>> rank(A)
ans =
2
% Change A
>> A(4,:) = [5 1];
A =
5 3
6 -3
-2 -1
5 1
% Rank test:
% b is not in the column space of A
>> rank([A b])
ans =
3
>> rank(A)
ans =
2
We can find the approximation solution by projection, or from the QR decomposition.
