.. _SVD:

Singular Value Decomposition (SVD)
==================================

We now consider what is perhaps the most important matrix factorization
in linear algebra. We introduced the SVD in :ref:`SVDintro` because we
needed it to solve systems of linear equations, especially rectangular
systems. This chapter presents a more complete picture of the SVD and
its applications.

Three properties summarize the importance of the SVD.

#. The SVD can factor any matrix, even singular and rectangular
   matrices.

#. The SVD solves many linear algebra problems and has applications in
   many science and engineering fields, including artificial
   intelligence, data analysis, and image processing. The linear algebra
   applications of the SVD include finding the rank of a matrix, finding
   the inverse of a square matrix (:ref:`SVDintro`), and the
   pseudo-inverse of rectangular matrices
   (:ref:`SVDpinv`, :ref:`underPinv`) [1]_, 
   finding the condition number of a
   matrix, finding orthogonal basis vectors for the column space of a
   matrix, finding the null space, row space, and the left null space of
   a matrix, and low-rank matrix approximations.

#. The algorithm to find the SVD is now fast and numerically accurate.
   That was not always the case. We discuss the development of the
   modern SVD algorithm and present a simple implementation in
   :ref:`SVDcalc`.

The fast factoring algorithm and the vast application domain of the
decomposition have made the SVD a key algorithm in numerical analysis 
software. It has replaced some elimination-based algorithms in the
internal workings of linear algebra software.

.. note::
   Our discussion of the SVD in this chapter focuses primarily on its
   interpretation and application.

.. topic:: The history of the SVD

   The official introduction of the SVD came in 1873 when Eugenio
   Beltrami published a paper describing it. Unaware of Beltrami’s
   paper, Camille Jordan independently published a paper describing it a
   year later. Jordan’s paper is more complete and accurate in the
   details presented. Beltrami and Jordan built on previous works on the
   eigenvalues and eigenvectors of symmetric matrices. Augustin-Louis
   Cauchy established the properties of eigenvalues and eigenvectors of
   symmetric matrices in 1827. Then, in 1846, Carl Jacobi presented the
   diagonalization factorization of symmetric matrices. Stewart gives a
   nice summary of the early history of SVD and related
   topics [STEWART93]_.

   Cleve Moler, co-founder of MathWorks, observed in a blog post that
   the SVD was of little practical value until
   1970 [MOLER06]_. Even when computers became
   available, the algorithm was too slow to be useful. The SVD
   algorithms developed between 1965 and 1990 are faster than the
   classic SVD algorithm and provide more numerically accurate
   results [YETURU20]_ and [LAMBERS10]_.

.. index:: Singular Value Decomposition

.. index:: SVD

**Contents**

.. toctree::
   :maxdepth: 2

   svdGeometry
   svdVectorChange
   SVDrank
   conditionNum
   subspaces
   dimRed
   otherSVDapps

.. rubric:: Footnote:

.. [1]
   Computing the inverse and pseudo-inverse is covered in
   :ref:`linAlg`, so it is not part of this chapter.