Section 3.3 Orthogonal Projection
In Exercise 3.1.18, we saw that the Fourier expansion theorem gives us an efficient way of testing whether or not a vector belongs to the span of an orthogonal set. When the answer is “no”, the quantity we compute while testing turns out to be very useful: it gives the orthogonal projection of that vector onto the span of our orthogonal set. This turns out to be exactly the ingredient needed to solve certain minimum distance problems.
We hinted above that the calculations we’ve been doing have a lot to do with projection. Since any single nonzero vector forms an orthogonal basis for its span, the projection
can be viewed as the orthogonal projection of the vector not onto the vector but onto the subspace This is, after all, how we viewed projections in elementary linear algebra: we drop the perpendicular from the tip of onto the line in the direction of
Now that we know how to define an orthogonal basis for a subspace, we can define orthogonal projection onto subspaces of dimension greater than one.
Note that is indeed an element of since it’s a linear combination of its basis vectors. In the case of the trivial subspace we define orthogonal projection of any vector to be since really, what other choice do we have? (This case isn’t really of any interest, we just like being thorough.)
Let’s see how this might be put to use in a classic problem: finding the distance from a point to a plane.
Example 3.3.2.
Solution 1. Using projection onto a normal vector
In an elementary linear algebra (or calculus) course, we would solve this problem as follows. First, we would need two vectors parallel to the plane. If lies in the plane, then so and
so and are parallel to the plane. We then compute the normal vector
and compute the projection of the position vector for the point onto This gives the vector
Now, this vector is parallel to so it’s perpendicular to the plane. Subtracting it from gives a vector parallel to the plane, and this is the position vector for the point we seek.
so the closest point is We weren’t asked for it, but note that if we wanted the distance from the point to the plane, this is given by
Solution 2. Using orthogonal projection
Let’s solve the same problem using orthogonal projection. First, we have to deal with the fact that the vectors and are probably not orthogonal. To get around this, we replace with
We now set
Lo and behold, we get the same answer as before.
The only problem with Definition 3.3.1 is that it appears to depend on the choice of orthogonal basis. To see that it doesn’t, we need one more definition.
Definition 3.3.3.
The term “complement” comes from terminology we mentioned early on, but didn’t spend much time on. Theorem 1.8.10 told us that for any subspace of a vector space it is possible to construct another subspace of such that The subspace is known as a complement of A complement is not unique, but the orthogonal complement is. As you might guess from the name, is also a subspace of
Exercise 3.3.4.
Hint.
The trusty Subspace Test is your friend here. Just be careful to work correctly with the definition of
Theorem 3.3.5. Projection Theorem.
Strategy.
For the first part, review the Orthogonal Lemma, and convince yourself that this says the same thing. The second part is the hard part, and it requires a trick: we can write as and then notice that What can we say using the first part, and the Pythagorean theorem?
Proof.
By Definition 3.3.1, is a linear combination of elements in so The fact that follows directly from the Orthogonal Lemma.
Choose any with and write
Since and we know that these two vectors are orthogonal, and therefore,
by the Pythagorean Theorem.
Exercise 3.3.6.
Show that Use this fact to show that Definition 3.3.1 does not depend on the choice orthogonal basis.
Hint.
Finally, we note one more useful fact. The process of sending a vector to its orthogonal projection defines an operator on and yes, it’s linear.
Theorem 3.3.7.
Strategy.
The fact that is linear follows from properties of the dot product, and some careful checking. We know that by definition of the projection, and you can show that acts as the identity on using the Fourier expansion theorem.
If then by definition of (Recall that it is defined using dot products with vectors in ) If use the Projection Theorem, to show that
Remark 3.3.8.
It follows from this result and the Dimension Theorem that
and since is indeed a complement of in the sense introduced in Theorem 1.8.10. It’s also fairly easy to see that directly. If and is a basis for then we have
and for an unknown this is simply a homogeneous system of equations with variables. Moreover, they are independent equations, since the form a basis. We thus expect free parameters in the general solution.
Theorem 3.3.9.
Exercise 3.3.10.
True.
- A subspace can have many complements, but only one orthgonal complement. For example, a complement to the
axis in is given by any other line through the origin, but only the axis is orthogonal. False.
- A subspace can have many complements, but only one orthgonal complement. For example, a complement to the
axis in is given by any other line through the origin, but only the axis is orthogonal.
Theorem 3.3.11.
Theorem 3.3.11 tells us that we can find a basis for by solving the homogeneous system Make sure you can see why this is true!
Example 3.3.12.
Solution.
First, we note that for a general element of we have
so is a basis for (We have just shown that this set spans it is independent since the first two vectors are not parallel, and the third vector cannot be in the span of the first two, since its third entry is nonzero.) As in Theorem 3.3.11, we set
To find a basis for we simply need to find the nullspace of which we do below.
xxxxxxxxxx
from sympy import Matrix, init_printing
init_printing()
A = Matrix(5,3,[1,-1,3,2,1,0,0,0,3,4,-1,3,1,0,-4])
B=A.T
B.nullspace()
Exercises Exercises
1.
Hint.
Given can you write it as a sum of an element of and an element of
2.
Hint.
Show that and then use Remark 3.3.8 to show that the two spaces must have the same dimension.
3.
Hint.
One inclusion is easier than the other. Use Theorem 1.8.6 and Remark 3.3.8 to show that the dimensions must be equal.
4.
5.
6.
7.
8.
9.
You have attempted 1 of 13 activities on this page.