1 The determinant

The determinant of a square matrix (a square table of numbers) is a very important object. We want to understand and relate some of its uses.

A matrix \(d \times d\) matrix \(M\) is a table of the form

\begin{equation*} \begin{pmatrix} M_{1,1} & \dots & M_{1,d} \\ \vdots & \ddots & \vdots \\ M_{d,1} & \ldots & M_{d,d} \end{pmatrix} \end{equation*}

where \(M_{1,1}\), \(M_{1,2}\), \(M_{2,1}\) etc. are numbers. The first index is the row index, the second index is the column index.

A Hessian of a function \(f\colon \R^2\to \R\) is a \(2\times 2\) matrix.

Q: Why is a Hessian of a function \(f\colon \R^3 \to \R\) a \(3\times 3\) matrix? What are its entries?

Q: Why is a Hessian of a function \(f\colon \R^d \to \R\) a \(d\times d\) matrix? What are its entries?

One can compute the determinant of a \(d\times d\) matrix that we denote by \(\det(M)\) or \(|M|\).

Q: Take a look and review the formulae for the determinant of a \(2\times 2\) and \(3\times 3\) matrix that can be found in the book or at the beginning of https://en.wikipedia.org/wiki/Determinant.

An interesting property is that the determinant of a matrix is the same if you flip row and columns.

Q: Check that this is true in the case of \(2\times 2\) matrices.

1.1 Areas and volumes

Let \(\vec{u}=[u_x;u_y]\) and \(\vec{v}=[v_x;v_y]\) be two vectors in \(\mathbb{R^3}\). We can create a matrix out of these two vectors as follows:

\begin{equation*} M=\begin{pmatrix} u_x & v_x \\ u_y &v_y \end{pmatrix} \end{equation*}

Q: Show that the area of the parallelogram with sides \(\vec{u}\) and \(\vec{v}\) is the determinant of the matrix (up to sign). Hint: compare with the cross product formula

We can also make the matrix as follows

\begin{equation*} M=\begin{pmatrix} u_x & u_y \\ v_x & v_y \end{pmatrix} \end{equation*}

The difference is sometimes referred to as interpering \(\vec{u}\) and \(\vec{v}\) as "column vectors" (former) or "row vectors" (latter)

Q: Show that the area of the parallelogram with sides \(\vec{u}\) and \(\vec{v}\) is the determinant of the matrix above (up to sign). Hint: compare with the cross product formula

Let \(\vec{u}=[u_x;u_y;u_z]\), \(\vec{v}=[v_x;v_y;v_z]\), \(\vec{w}=[w_x;w_y;w_z]\) be three vectors in \(\mathbb{R}^3\). We can create a matrix out of these two vectors as follows:

\begin{equation*} M=\begin{pmatrix} u_x & v_x & w_x \\ u_y &v_y& w_y \\ u_z &v_z& w_z \end{pmatrix} \end{equation*}

Q: Check that the volume of the parallelogram with sides \(\vec{u}\), \(\vec{v}\), \(\vec{w}\) is, up to sign, the determinant of the above matrix. Refer to the cross product and the triple product rule from the book.

Remark:

The determinant of a \(d\times d\) matrix is (up to sign) the d-dimensional volume of the hyper-parallelipiped with \(d\) sides that are the \(d\)-dimensional column vectors of the matrix.

1.2 Solving systems of linear equations in \(2D\)

We are interested in solving a system of linear equations.

\begin{equation*} \begin{aligned}[t] & a x +b y = S \\ & c x +d y = T \end{aligned} \end{equation*}

where \(a,b,c,d,S,T\) are FIXED numbers and \(x,y\) are variables. It is a well known fact that a solution to this problem exists and is unique if

\begin{equation*} \det \begin{pmatrix} a &b \\ c &d \end{pmatrix}\neq 0 \end{equation*}

We want to try to give a geometric interpretation of this fact using our intuition about areas and volumes

Q: What geometric object does the equation

\begin{equation*} ax+by =S \end{equation*}

represent. What is role do the numbers \(a,b\) play? What role does the number \(S\) play?

Q: What geometric object does the equation

\begin{equation*} cx+dy =T \end{equation*}

represent. What is role do the numbers \(c,d\) play? What role does the number \(T\) play?

Q: Taking a system of equations corresponds to which operation on the corresponding geometric objects:

  • intersecting them
  • taking their union

Q: From geometry, we know that two lines in \(\R^2\) have a unique intersection if they are not parallel. How can this condition be expressed as a condition on the normal directions to the lines?

Q: How can this condition be expressed in terms of the area of the parallelogram with sides given by the normal directions?

Q: Relate

\begin{equation*} \det \begin{pmatrix} a &b \\ c &d \end{pmatrix}\neq 0 \end{equation*}

to always being able to solve

\begin{equation*} \begin{aligned}[t] & a x +b y = S \\ & c x +d y = T \end{aligned} \end{equation*}

1.3 Solving systems of linear equations in \(3D\)

We are interested in solving a system of linear equations.

\begin{equation*} \begin{aligned}[t] & a_{1,1} x +a_{1,2} y + a_{1,3} z= b_1 \\ & a_{2,1} x +a_{2,2} y + a_{2,3} z= b_1 \\ & a_{3,1} x +a_{3,2} y + a_{3,3} z= b_1 \end{aligned} \end{equation*}

where \(a_{1,1},\ldots,a_{3,3}\) and \(b_1,b_2,b_3\) are FIXED numbers and \(x,y,z\) are variables. It is a well known fact that a solution to this problem exists and is unique if

\begin{equation*} \det \begin{pmatrix} a_{1,1} &a_{1,2}& a_{1,3} \\ a_{2,1} & a_{2,2}&a_{2,3} \\ a_{3,1} &a_{3,2} &a_{3,3} \end{pmatrix}\neq 0 \end{equation*}

We want to try to give a geometric interpretation of this fact using our intuition about areas and volumes

Q: What geometric object does the equation

\begin{equation*} a_{1,1} x +a_{1,2} y + a_{1,3} z= b_1 \end{equation*}

represent. What is role do the numbers \( a_{1,1}, a_{1,2}, a_{1,3}\) play? What role does the number \(b_1\) play?

Q: Answer the same question for the other 2 equations

Q: Taking a system of equations corresponds to which operation on the corresponding geometric objects:

  • intersecting them
  • taking their union

Q: If

\begin{equation*} \det \begin{pmatrix} a_{1,1} &a_{1,2}& a_{1,3} \\ a_{2,1} & a_{2,2}&a_{2,3} \\ a_{3,1} &a_{3,2} &a_{3,3} \end{pmatrix}\neq 0 \end{equation*}

why are the vectors \(\vec{a}_1 = [a_{1,1} ; a_{1,2}; a_{1,3} ]\) and \(\vec{a}_2 = [a_{2,1} ; a_{2,2}; a_{2,3} ]\) not parallel? Think about the volume.

Q: Given the consideration of the previous question what geometric object is determined by

\begin{equation*} \begin{aligned}[t] & a_{1,1} x +a_{1,2} y + a_{1,3} z= b_1 \\ & a_{2,1} x +a_{2,2} y + a_{2,3} z= b_1? \end{aligned} \end{equation*}

Q: Now we need to add the third equation into the system. Why are the vectors \(\vec{a}_1 = [a_{1,1} ; a_{1,2}; a_{1,3} ]\), \(\vec{a}_2 = [a_{2,1} ; a_{2,2}; a_{2,3} ]\), and \(\vec{a}_3 = [a_{3,1} ; a_{3,2}; a_{3,3} ]\) not lying in the same plane?

Q: Using facts from geometry (intersection of plane and line in space) deduce that there is only one solution to the system

\begin{equation*} \begin{aligned}[t] & a_{1,1} x +a_{1,2} y + a_{1,3} z= b_1 \\ & a_{2,1} x +a_{2,2} y + a_{2,3} z= b_1 \\ & a_{3,1} x +a_{3,2} y + a_{3,3} z= b_1 \end{aligned} \end{equation*}

1.4 Solving systems of linear equations in higher dimensions

What facts would you need and how would you replicate the reasoning above for systems of equations in higher dimensions?