Diagonalization of complex matrices

Section 4.4 Diagonalization of complex matrices

Recall that when we first defined vector spaces, we mentioned that a vector space can be defined over any field

F .

To keep things simple, we’ve mostly assumed

F = R .

But most of the theorems and proofs we’ve encountered go through unchanged if we work over a general field. (This is not quite true: over a finite field things can get more complicated. For example, if

F = Z_{2} = {0, 1},

then we get weird results like

v + v = 0,

since

1 + 1 = 0 .

)

🔗

In fact, if we replace

R

C,

about the only thing we’d have to go back and change is the definition of the dot product. The reason for this is that although the complex numbers seem computationally more complicated, (which might mostly be because you don’t use them often enough) they follow the exact same algebraic rules as the real numbers. In other words, the arithmetic might be different, but the algebra is the same. There is one key difference between the two fields: over the complex numbers, every polynomial can be factored. This is important if you’re interested in finding eigenvalues.

🔗

This section is written based on the assumption that complex numbers were covered in a previous course. If this was not the case, or to review this material, see Appendix A before proceeding.

🔗

Subsection 4.4.1 Complex vectors

🔗

A complex vector space is simply a vector space where the scalars are elements of

C

rather than

R .

Examples include polynomials with complex coefficients, complex-valued functions, and

C^{n},

which is defined exactly how you think it should be. In fact, one way to obtain

C^{n}

is to start with the exact same standard basis we use for

R^{n},

and then take linear combinations using complex scalars.

🔗

We’ll write elements of

C^{n}

z = (z_{1}, z_{2}, \dots, z_{n}) .

The complex conjugate of

z

is given by

\bar{z} = ({\overset{―}{z}}_{1}, {\overset{―}{z}}_{2}, \dots, {\overset{―}{z}}_{n}) .

🔗

The standard inner product on

C^{n}

looks a lot like the dot product on

R^{n},

with one important difference: we apply a complex conjugate to the second vector.

🔗

Definition 4.4.1.

🔗

The standard inner product on

C^{n}

is defined as follows: given

z = (z_{1}, z_{2}, \dots, z_{n})

and

w = (w_{1}, w_{2}, \dots, w_{n}),

⟨ z, w ⟩ = z \cdot \bar{w} = z_{1} {\bar{w}}_{1} + z_{2} {\bar{w}}_{2} + \dots + z_{n} {\bar{w}}_{n} .

🔗

z, w

are real, this is just the usual dot product. The reason for using the complex conjugate is to ensure that we still have a positive-definite inner product on

C^{n} :

⟨ z, z ⟩ = z_{1} {\overset{―}{z}}_{1} + z_{2} {\overset{―}{z}}_{2} + \dots + z_{n} {\overset{―}{z}}_{n} = | z_{1} |^{2} + | z_{2} |^{2} + \dots + | z_{n} |^{2},

🔗

which shows that

⟨ z, z ⟩ \geq 0,

and

⟨ z, z ⟩ = 0

if and only if

z = 0 .

🔗

Exercise 4.4.2.

🔗

Compute the dot product of

z = (2 - i, 3 i, 4 + 2 i)

and

w = (3 i, 4 - 5 i, - 2 + 2 i) .

🔗

This isn’t hard to do by hand, but it’s useful to know how to ask the computer to do it, too. Unfortunately, the dot product in SymPy does not include the complex conjugate. One likely reason for this is that while most mathematicians take the complex conjugate of the second vector, some mathematicians, and most physicists, put the conjugate on the first vector. So they may have decided to remain agnostic about this choice. We can manually apply the conjugate, using Z.dot(W.H). (The .H operation is the hermitian conjugate; see Definition 4.4.6 below.)


    
        
xxxxxxxxxx
 
1
from sympy import Matrix,init_printing
2
init_printing()
3
Z = Matrix(3,1,[2-I,3*I,4+2*I])
4
W = Matrix(3,1,[3*I,4-5*I,-2+2*I])
5
Z, W, Z.dot(W.H)

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Again, you might want to wrap that last term in simplify() (in which case you’ll get

- 22 - 6 i

for the dot product). Above, we saw that the complex inner product is designed to be positive definite, like the real inner product. The remaining properties of the complex inner product are given as follows.

🔗

Theorem 4.4.3.

🔗

For any vectors

z_{1}, z_{2}, z_{3}

and any complex number

α,

$⟨ z_{1} + z_{2}, z_{3} ⟩ = ⟨ z_{1}, z_{3} ⟩ + ⟨ z_{2}, z_{3} ⟩$ and $⟨ z_{1}, z_{2} + z_{3} ⟩ = ⟨ z_{1}, z_{2} ⟩ + ⟨ z_{1}, z_{3} ⟩ .$
$⟨ α z_{1}, z_{2} ⟩ = α ⟨ z_{1}, z_{2} ⟩$ and $⟨ z_{1}, α z_{2} ⟩ = \bar{α} ⟨ z_{1}, z_{2} ⟩ .$
$⟨ z_{2}, z_{1} ⟩ = \overset{―}{⟨ z_{1}, z_{2} ⟩}$
$⟨ z_{1}, z_{1} ⟩ \geq 0,$ and $⟨ z_{1}, z_{1} ⟩ = 0$ if and only if $z_{1} = 0 .$

Proof.

Using the distributive properties of matrix multiplication and the transpose,

$\begin{aligned} ⟨ z_{1} + z_{2}, z_{3} ⟩ & = (z_{1} + z_{2})^{T} \bar{z_{3}} \\ = (z_{1}^{T} + z_{2}^{T}) \bar{z_{3}} \\ = z_{1}^{T} \bar{z_{3}} + z_{2}^{T} \bar{z_{3}} \\ = ⟨ z_{1}, z_{3} ⟩ + ⟨ z_{2}, z_{3} ⟩ . \end{aligned}$

The proof is similar when addition is in the second component. (But not identical -- you’ll need the fact that the complex conjugate is distributive, rather than the transpose.)
These again follow from writing the inner product as a matrix product.

$⟨ α z_{1}, z_{2} ⟩ = (α z_{1})^{T} \bar{z_{2}} = α (z_{1}^{T} \bar{z_{2}}) = α ⟨ z_{1}, z_{2} ⟩,$

and

$⟨ z_{1}, α z_{2} ⟩ = z_{1}^{T} \overset{―}{α z_{2}} = z_{1}^{T} (\bar{α} \bar{z_{2}}) = \bar{α} (z_{1}^{T} z_{2}) = α ⟨ z_{1}, z_{2} ⟩ .$
Note that for any vectors $z, w,$ $z^{T} w$ is a number, and therefore equal to its own transpose. Thus, we have $z^{T} w = (z^{T} w)^{T} = w^{T} z,$ and

$\overset{―}{⟨ z_{1}, z_{2} ⟩} = \overset{―}{z_{1}^{T} \bar{z_{2}}} = \overset{―}{{\bar{z_{2}}}^{T} z_{1}} = z_{2}^{T} \overset{―}{z_{1}} = ⟨ z_{2}, z_{1} ⟩ .$
This was already demonstrated above.

🔗

Definition 4.4.4.

🔗

The norm of a vector

z = (z_{1}, z_{2}, \dots, z_{n})

C^{n}

is given by

‖ z ‖ = \sqrt{⟨ z, z ⟩} = \sqrt{| z_{1} |^{2} + | z_{2} |^{2} + \dots + | z_{n} |^{2}} .

🔗

Note that much like the real norm, the complex norm satisfies

‖ α z ‖ = | α | ‖ z ‖

for any (complex) scalar

α .

🔗

Exercise 4.4.5.

🔗

The norm of a complex vector is always a real number.

True.
Since the norm is computed using the modulus, which is always real and non-negative, the norm will be a real number as well. If you ever get a complex number for your norm, you’ve probably forgotten the complex conjugate somewhere.
False.
Since the norm is computed using the modulus, which is always real and non-negative, the norm will be a real number as well. If you ever get a complex number for your norm, you’ve probably forgotten the complex conjugate somewhere.

🔗

Subsection 4.4.2 Complex matrices

🔗

Linear transformations are defined in exactly the same way, and a complex matrix is simply a matrix whose entries are complex numbers. There are two important operations defined on complex matrices: the conjugate, and the conjugate transpose (also known as the hermitian transpose).

🔗

Definition 4.4.6.

🔗

The conjugate of a matrix

A = [a_{i j}] \in M_{m n} (C)

is the matrix

\bar{A} = [{\bar{a}}_{i j}] .

The conjugate transpose of

A

is the matrix

A^{H}

defined by

A^{H} = (\bar{A})^{T} = \overset{―}{(A^{T})} .

🔗

Note that many textbooks use the notation

A^{†}

for the conjugate transpose.

🔗

Definition 4.4.7.

🔗

n \times n

matrix

A \in M_{n n} (C)

is called hermitian if

A^{H} = A,

and unitary if

A^{H} = A^{- 1} .

(A matrix is skew-hermitian if

A^{H} = - A .

)

🔗

Hermitian and unitary matrices (or more accurately, linear operators) are very important in quantum mechanics. Indeed, hermitian matrices represent “observable” quantities, in part because their eigenvalues are real, as we’ll soon see. For us, hermitian and unitary matrices can simply be viewed as the complex counterparts of symmetric and orthogonal matrices, respectively. In fact, a real symmetric matrix is hermitian, since the conjugate has no effect on it, and similarly, a real orthogonal matrix is technically unitary. As with orthogonal matrices, a unitary matrix can also be characterized by the property that its rows and columns both form orthonormal bases.

🔗

Exercise 4.4.8.

🔗

Show that the matrix

A = [\begin{matrix} 4 & 1 - i & - 2 + 3 i \\ 1 + i & 5 & 7 i \\ - 2 - 3 i & - 7 i & - 4 \end{matrix}]

is hermitian, and that the matrix

B = \frac{1}{2} [\begin{matrix} 1 + i & \sqrt{2} \\ 1 - i & \sqrt{2} i \end{matrix}]

is unitary.

🔗

When using SymPy, the hermitian conjugate of a matrix A is executed using A.H. (There appears to also be an equivalent operation named Dagger coming from sympy.physics.quantum, but I’ve had more success with .H.) The complex unit is entered as I. So for the exercise above, we can do the following.


    
        
xxxxxxxxxx
 
1
A = Matrix(3,3,[4,1-I,-2+3*I,1+I,5,7*I,-2-3*I,-7*I,-4])
2
A == A.H

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

The last line verifies that

A = A^{H} .

We could also replace it with A,A.H to explicitly see the two matrices side by side. Now, let’s confirm that

B

is unitary.


    
        
xxxxxxxxxx
 
1
B = Matrix(2,2,[1/2+1/2*I, sqrt(2)/2,1/2-1/2*I,(sqrt(2)/2)*I])
2
B,B*B.H

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Hmm... That doesn’t look like the identity on the right. Maybe try replacing B*B.H with simplify(B*B.H). (You will want to add from sympy import simplify at the top of the cell.) Or you could try B.H, B**-1 to compare results. Actually, what’s interesting is that in a Sage cell, B.H == B**-1 yields False, but B.H == simplify(B**-1) yields True!

🔗

As mentioned above, hermitian matrices are the complex analogue of symmetric matrices. Recall that a key property of a symmetric matrix is its symmetry with respect to the dot product. For a symmetric matrix

A,

we had

x \cdot (A y) = (A x) \cdot y .

Hermtian matrices exhibit the same behaviour with respect to the complex inner product.

🔗

Theorem 4.4.9.

🔗

n \times n

complex matrix

A

is hermitian if and only if

⟨ A z, w ⟩ = ⟨ z, A w ⟩

🔗

for any

z, w \in C^{n}

Proof.

Note that the property

A^{H} = A

is equivalent to

A^{T} = \bar{A} .

This gives us

⟨ A z, w ⟩ = (A z)^{T} \bar{w} = (z^{T} A^{T}) \bar{w} = (z^{T} \bar{A}) \bar{w} = z^{T} (\overset{―}{A w}) = ⟨ z, w ⟩ .

Conversely, suppose

⟨ A z, w ⟩ = ⟨ z, A w ⟩

for all

z, w \in C^{n},

and let

{e_{1}, e_{2}, \dots, e_{n}}

denote the standard basis for

C^{n} .

Then

a_{j i} = ⟨ A e_{i}, e_{j} ⟩ = ⟨ e_{i}, A e_{j} ⟩ = \overset{―}{a_{i j}},

which shows that

A^{T} = \bar{A} .

🔗

Next, we’ve noted that one advantage of doing linear algebra over

C

is that every polynomial can be completely factored, including the characteristic polynomial. This means that we can always find eigenvalues for a matrix. When that matrix is hermitian, we get a surprising result.

🔗

Theorem 4.4.10.

🔗

For any hermitian matrix

A,

The eigenvalues of $A$ are real.
Eigenvectors corresponding to distinct eigenvalues are orthogonal.

Proof.

Suppose $A z = λ z$ for some $λ \in C$ and $z \neq 0 .$ Then

$λ ⟨ z, z ⟩ = ⟨ λ z, z ⟩ = ⟨ A z, z ⟩ = ⟨ z, A z ⟩ = ⟨ z, λ z ⟩ = \bar{λ} ⟨ z, z ⟩ .$

Thus, $(λ - \bar{λ}) ‖ z ‖^{2} = 0,$ and since $‖ z ‖ \neq 0,$ we must have $\bar{λ} = λ,$ which means $λ \in R .$
Similarly, suppose $λ_{1}, λ_{2}$ are eigenvalues of $A,$ with corresponding eigenvectors $z, w .$ Then

$λ_{1} ⟨ z, w ⟩ = ⟨ λ_{1} z, w ⟩ = ⟨ A z, w ⟩ = ⟨ z, A w ⟩ = ⟨ z, λ_{2} w ⟩ = \bar{λ_{2}} ⟨ z, w ⟩ .$

This gives us $(λ_{1} - \bar{λ_{2}}) ⟨ z, w ⟩ = 0 .$ And since we already know $λ_{2}$ must be real, and $λ_{1} \neq λ_{2},$ we must have $⟨ z, w ⟩ = 0 .$

🔗

In light of Theorem 4.4.10, we realize that diagonalization of hermitian matrices will follow the same script as for symmetric matrices. Indeed, Gram-Schmidt Orthonormalization Algorithm applies equally well in

C^{n},

as long as we replace the dot product with the complex inner product. This suggests the following.

🔗

Theorem 4.4.11. Spectral Theorem.

🔗

A

is an

n \times n

hermitian matrix, then there exists an orthonormal basis of

C^{n}

consisting of eigenvectors of

A .

Moreover, the matrix

U

whose columns consist of those eigenvectors is unitary, and the matrix

U^{H} A U

is diagonal.

🔗

Exercise 4.4.12.

🔗

Confirm that the matrix

A = [\begin{matrix} 4 & 3 - i \\ 3 + i & 1 \end{matrix}]

is hermitian. Then, find the eigenvalues of

A,

and a unitary matrix

U

such that

U^{H} A U

is diagonal.

🔗

To do the above exercise using SymPy, we first define

A

and ask for the eigenvectors.


    
        
xxxxxxxxxx
 
1
A = Matrix(2,2,[4,3-I,3+I,1])
2
A.eigenvects()

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

We can now manually determine the matrix

U,

as we did above, and input it:


    
        
xxxxxxxxxx
 
1
U = Matrix([[(3-I)/sqrt(35),(3-I)/sqrt(14)],
2
            [-5/sqrt(35),2/sqrt(14)]])

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

To confirm it’s unitary, add the line U*U.H to the above, and confirm that you get the identity matrix as output. You might need to use simplify(U*U.H) if the result is not clear. Now, to confirm that

U^{H} A U

really is diagonal, go back to the cell above, and enter it. Try (U.H)*A*U, just to remind yourself that adding the simplify command is often a good idea.

🔗

If you want to cut down on the manual labour involved, we can make use of some of the other tools SymPy provides. In the next cell, we’re going to assign the output of A.eigenvects() to a list. The only trouble is that the output of the eigenvector command is a list of lists. Each list item is a list (eigenvalue, multiplicity, [eigenvectors]).


    
        
xxxxxxxxxx
 
1
L = A.eigenvects()
2
L

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Try the above modifications, in sequence. First, replacing the second line by L[0] will give the first list item, which is another list:

(- 1, 1, [[\begin{matrix} - \frac{3}{5} + \frac{i}{5} \end{matrix}]]) .

🔗

We want the third item in the list, so try (L[0])[2]. But note the extra set of brackets! There could (in theory) be more than one eigenvector, so this is a list with one item. To finally get the vector out, try ((L[0])[2])[0]. (There is probably a better way to do this. Someone who is more fluent in Python is welcome to advise.)

🔗

Now that we know how to extract the eigenvectors, we can normalize them, and join them to make a matrix. The norm of a vector is simnply v.norm(), and to join column vectors u1 and u2 to make a matrix, we can use the command u1.row_join(u2). We already defined the matrix A and list L above, but here is the whole routine in one cell, in case you didn’t run all the cells above.


    
        
xxxxxxxxxx
 
1
from sympy import Matrix, init_printing, simplify
2
init_printing()
3
A = Matrix(2,2,[4,3-I,3+I,1])
4
L = A.eigenvects()
5
v = ((L[0])[2])[0]
6
w = ((L[1])[2])[0]
7
u1 = (1/v.norm())*v
8
u2 = (1/w.norm())*w
9
U = u1.row_join(u2)
10
u1, u2, U, simplify(U.H*A*U)

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Believe me, you want the simplify command on that last matrix.

🔗

While Theorem 4.4.11 guarantees that any hermitian matrix can be “unitarily diagonalized”, there are also non-hermitian matrices for which this can be done as well. A classic example of this is the rotation matrix

[\begin{matrix} 0 & 1 \\ - 1 & 0 \end{matrix}] .

This is a real matrix with complex eigenvalues

\pm i,

and while it is neither symmetric nor hermitian, it can be orthogonally diagonalized. This should be contrasted with the real spectral theorem, where any matrix that can be orthogonally diagonalized is necessarily symmetric.

🔗

This suggests that perhaps hermitian matrices are not quite the correct class of matrix for which the spectral theorem should be stated. Indeed, it turns out there is a somewhat more general class of matrix: the normal matrices.

🔗

Definition 4.4.13.

🔗

n \times n

matrix

A

is normal if

A^{H} A = A A^{H} .

🔗

Exercise 4.4.14.

$[\begin{matrix} 3 & 1 - 3 i \\ 1 + 3 i & - 4 \end{matrix}]$
This matrix is hermitian, and we know that every hermitian matrix is normal.
$[\begin{matrix} 1 & 3 \\ 0 & 2 \end{matrix}]$
This matrix is not normal; this can be confirmed by direct computation, or by noting that it cannot be diagonalized.
$\frac{1}{\sqrt{2}} [\begin{matrix} 1 & 1 \\ i & - i \end{matrix}]$
This matrix is unitary, and every unitary matrix is normal.
$[\begin{matrix} i & 2 i \\ 2 i & 3 i \end{matrix}]$
This matrix is neither hermitian nor unitary, but it is normal, which can be verified by direct computation.

🔗

It turns out that a matrix

A

is normal if and only if

A = U D U^{H}

for some unitary matrix

U

and diagonal matrix

D .

A further generalization is known as Schur’s Theorem.

🔗

Theorem 4.4.15.

🔗

For any complex

n \times n

matrix

A,

there exists a unitary matrix

U

such that

U^{H} A U = T

is upper-triangular, and such that the diagonal entries of

T

are the eigenvalues of

A .

🔗

Using Schur’s Theorem, we can obtain a famous result, known as the Cayley-Hamilton Theorem, for the case of complex matrices. (It is true for real matrices as well, but we don’t yet have the tools to prove it.) The Cayley-Hamilton Theorem states that substituting any matrix into its characteristic polynomial results in the zero matrix. To understand this result, we should first explain how to define a polynomial of a matrix.

🔗

Given a polynomial

p (x) = a_{0} + a_{1} x + \dots + a_{n} x_{n},

we define

p (A)

p (A) = a_{0} I + a_{1} A + \dots + a_{n} A^{n} .

🔗

(Note the presence of the identity matrix in the first term, since it does not make sense to add a scalar to a matrix.) Note further that since

(P^{- 1} A P)^{n} = P^{- 1} A^{n} P

for any invertible matrix

P

and positive integer

n,

we have

p (U^{H} A U) = U^{H} p (A) U

for any polynomial

p

and unitary matrix

U .

🔗

Theorem 4.4.16.

🔗

Let

A

be an

n \times n

complex matrix, and let

c_{A} (x)

denote the characteristic polynomial of

A .

Then we have

c_{A} (A) = 0 .

Proof.

By Theorem 4.4.15, there exists a unitary matrix

U

such that

A = U T U^{H},

where

T

is upper triangular, and has the eigenvalues of

A

as diagonal entries. Since

c_{A} (A) = c_{A} (U T U^{H}) = U c_{A} (T) U^{H},

and

c_{A} (x) = c_{T} (x)

(since

A

and

T

are similar) it suffices to show that

c_{A} (A) = 0

when

A

is upper-triangular. (If you like, we are showing that

C_{T} (T) = 0,

and deducing that

c_{A} (A) = 0 .

) But if

A

is upper-triangular, so is

x I_{A},

and therefore,

det (x I - A)

is just the product of the diagonal entries. That is,

c_{A} (x) = (x - λ_{1}) (x - λ_{2}) \dots (x - λ_{n}),

c_{A} (A) = (A - λ_{1} I) (A - λ_{2} I) \dots (A - λ_{n} I) .

Since the first column of

A

{[\begin{matrix} λ_{1} & 0 & \dots & 0 \end{matrix}]}^{T},

the first column of

A - λ_{1} I

is identically zero. The second column of

A - λ_{2} I

similarly has the form

[\begin{matrix} k & 0 & \dots & 0 \end{matrix}]

for some number

k .

It follows that the first two columns of

(A - λ_{1} I) (A - λ_{2} I)

are identically zero. Since only the first two entries in the third column of

(A - λ_{3} I)

can be nonzero, we find that the first three columns of

(A - λ_{1} I) (A - λ_{2} I) (A - λ_{3} I)

are zero, and so on.

🔗

Exercises 4.4.3 Exercises

🔗

1.

🔗

Suppose

A

is a

3 \times 3

matrix with real entries that has a complex eigenvalue

- 6 - 2 i

with corresponding eigenvector

[\begin{matrix} 2 + 3 i \\ 1 \\ 8 i \end{matrix}] .

Find another eigenvalue and eigenvector for

A .

🔗

2.

🔗

Give an example of a [`2 times 2 `] matrix with no real eigenvalues.

🔗

3.

🔗

Find all the eigenvalues (real and complex) of the matrix

M = [\begin{array}{ccc} - 3 & - 6 & 2 \\ 5 & 2 & 1 \\ 5 & 3 & 0 \end{array}] .

🔗

4.

🔗

Find all the eigenvalues (real and complex) of the matrix

M = [\begin{array}{cccc} 2 & 0 & 0 & 5 \\ 0 & - 5 & 3 & 3 \\ 0 & - 4 & 2 & - 1 \\ 0 & - 3 & 3 & 1 \end{array}] .

🔗

5.

🔗

Let

M = [\begin{array}{cc} 3 & - 9 \\ 9 & 3 \end{array}] .

Find formulas for the entries of

M^{n},

where

n

is a positive integer. (Your formulas should not contain complex numbers.)

🔗

6.

🔗

Let

M = [\begin{array}{ccc} 5 & - 3 & 10 \\ 3 & 5 & - 5 \\ 0 & 0 & 1 \end{array}] .

🔗

Find formulas for the entries of

M^{n},

where

n

is a positive integer. (Your formulas should not contain complex numbers.)

🔗

7.

🔗

Let

M = [\begin{array}{ccc} 3 & - 1 & - 3 \\ - 3 & 3 & 1 \\ 3 & - 2 & - 1 \end{array}] .

Find

c_{1},

c_{2},

and

c_{3}

such that

M^{3} + c_{1} M^{2} + c_{2} M + c_{3} I_{3} = 0,

where

I_{3}

is the

3 \times 3

identity matrix.

You have attempted 1 of 11 activities on this page.

Prev Top Next