2.1. Python for Linear Algebra Applications

Python is a very good language for general problem solving. It is a fair environment for numerical computing. Python alone is not well suited for numerical computation and does not include linear algebra functions. Extra modules are used to fill that gap. As compared to environments, like MATLAB, that were designed from the beginning for numerical computation, especially linear algebra, the general purpose nature of Python does constrain the expressiveness of the modules. Yet two modules (numpy and scipy) provide the tools to solve any linear algebra problem.

Numerical computing requires fast access to arrays of numbers. The standard array-like storage container in Python is the list, which extends the functionality found with conventional arrays, but a list is not an array. A list can contain many different data types and has advanced features, which are convenient in general computing applications, but make them too slow for linear algebra applications.

The numpy module contains an array storage facility for arrays of one or more dimensions. Applications such data and signal processing use one dimensional arrays. A vector could also use a one dimensional array, but two dimensional arrays are generally preferred for both vectors and matrices. A tensor can need more dimensions than two, but we will not work with tensors.

2.1.1. Declaring Vectors and Matrices

Here is one a dimensional array in the iPython shell. Notice that with iPython, we can end a statement with a semicolon and then enter the name of a variable to display its value immediately.

In [76]: import numpy as np

In [77]: d = np.array([1, 2, 3]); d
Out[77]: array([1, 2, 3])

In [78]: d.shape
Out[78]: (3,)

The shape property for d shows only one dimension and several linear algebra functions will not work as expected on d. Let’s make another vector in one row.

In [79]: v = np.array([[1, 2, 3]]); v
Out[79]: array([[1, 2, 3]])

In [80]: v.shape
Out[80]: (1, 3)

In [81]: print(v)
[[1 2 3]]

In [82]: v.T
Out[82]:
array([[1],
    [2],
    [3]])

Vector v is a row vector with one row and three columns. We can convert it to a column vector with the transpose operator (.T). Although some application make use of one dimensional arrays, more predictable results will be found with a two dimensional array, even for vectors.

Let’s now make a matrix with manually entered values.

In [83]: A = np.array([[1, 2, 3],
    ...: [2, 4, 5],
    ...: [3, 5, 2]])

In [84]: print(A)
[[1 2 3]
 [2 4 5]
 [3 5 2]]

In [85]: A.shape
Out[85]: (3, 3)

In [86]: m,n = A.shape # m x n matrix

Other functions from numpy generate matrices holdings ones, zeros, and the identity matrix.

In [87]: B = 2 * np.ones((3, 2), dtype=float); B
Out[87]:
array([[2., 2.],
    [2., 2.],
    [2., 2.]])

In [88]: C = np.zeros((2, 2)); C
Out[88]:
array([[0., 0.],
    [0., 0.]])

In [89]: I = np.eye(3); I
Out[89]:
array([[1., 0., 0.],
    [0., 1., 0.],
    [0., 0., 1.]])

We will use mostly numpy for basic matrix and vector operations, but will use scipy for more advanced algorithms.

2.1.2. Indexing

Vectors are indexed in the same way as lists in Python. Matrix indexing is [row, column]. A colon (:) means all rows or all columns.

In [90]: A = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]); A
Out[90]:
array([[0, 1, 2],
       [3, 4, 5],
       [6, 7, 8]])

In [91]: A[2,2]
Out[91]: np.int64(8)

In [92]: A[:,2]          # column 2
Out[92]: array([2, 5, 8])

In [93]: A[1,:]          # row 1
Out[93]: array([3, 4, 5])

A vector extracted from a matrix is one dimensional, but it may be reshaped if needed.

In [94]: u = A[:,1]; u
Out[94]: array([1, 4, 7])

In [95]: u.shape
Out[95]: (3,)

In [96]: u = u.reshape(3,1); u
Out[96]:
array([[1],
       [4],
       [7]])

In [97]: u.shape
Out[97]: (3, 1)