Skip to main content

Section 3.3 Orthogonal Projection

In Exercise 3.1.18, we saw that the Fourier expansion theorem gives us an efficient way of testing whether or not a vector belongs to the span of an orthogonal set. When the answer is “no”, the quantity we compute while testing turns out to be very useful: it gives the orthogonal projection of that vector onto the span of our orthogonal set. This turns out to be exactly the ingredient needed to solve certain minimum distance problems.
We hinted above that the calculations we’ve been doing have a lot to do with projection. Since any single nonzero vector forms an orthogonal basis for its span, the projection
projuv=(uvu2)u
can be viewed as the orthogonal projection of the vector v, not onto the vector u, but onto the subspace span{u}. This is, after all, how we viewed projections in elementary linear algebra: we drop the perpendicular from the tip of v onto the line in the direction of u.
Now that we know how to define an orthogonal basis for a subspace, we can define orthogonal projection onto subspaces of dimension greater than one.

Definition 3.3.1.

Let U be a subspace of Rn with orthogonal basis {u1,,uk}. For any vector vRn, we define the orthogonal projection of v onto U by
projUv=i=1k(uivui2)ui.
Note that projUv is indeed an element of U, since it’s a linear combination of its basis vectors. In the case of the trivial subspace U={0}, we define orthogonal projection of any vector to be 0, since really, what other choice do we have? (This case isn’t really of any interest, we just like being thorough.)
Let’s see how this might be put to use in a classic problem: finding the distance from a point to a plane.

Example 3.3.2.

Find the distance from the point (3,1,2) to the plane P defined by x2y+4z=0.
Solution 1. Using projection onto a normal vector
In an elementary linear algebra (or calculus) course, we would solve this problem as follows. First, we would need two vectors parallel to the plane. If [xyz] lies in the plane, then x2y+4z=0, so x=2y4z, and
[xyz]=[2y4zyz]=y[210]+z[401],
so u=[210] and v[401] are parallel to the plane. We then compute the normal vector
n=u×v=[124],
and compute the projection of the position vector p=[3,1,2] for the point P=(3,1,2) onto n. This gives the vector
x=(pnn2)n=721[124]=[1/32/34/3].
Now, this vector is parallel to n, so it’s perpendicular to the plane. Subtracting it from p gives a vector parallel to the plane, and this is the position vector for the point we seek.
q=px=[312][1/32/34/3]=[10/31/32/3]
so the closest point is Q=(103,13,23). We weren’t asked for it, but note that if we wanted the distance from the point P to the plane, this is given by x=1321.
Solution 2. Using orthogonal projection
Let’s solve the same problem using orthogonal projection. First, we have to deal with the fact that the vectors u and v are probably not orthogonal. To get around this, we replace v with
w=v(vuu2)u=[401]+85[210]=[4/58/51].
We now set
q=(puu2)u(pww2)w=75[210]+14105[485]=[10/31/32/3].
Lo and behold, we get the same answer as before.
The only problem with Definition 3.3.1 is that it appears to depend on the choice of orthogonal basis. To see that it doesn’t, we need one more definition.

Definition 3.3.3.

For any subspace U of Rn, we define the orthogonal complement of U, denoted U, by
U={xRn|xy=0 for all yU}.
The term “complement” comes from terminology we mentioned early on, but didn’t spend much time on. Theorem 1.8.10 told us that for any subspace U of a vector space V, it is possible to construct another subspace W of V such that V=UW. The subspace W is known as a complement of U. A complement is not unique, but the orthogonal complement is. As you might guess from the name, U is also a subspace of Rn.

Exercise 3.3.4.

Show that U is a subspace of Rn.
Hint.
The trusty Subspace Test is your friend here. Just be careful to work correctly with the definition of U.

Strategy.

For the first part, review the Orthogonal Lemma, and convince yourself that this says the same thing. The second part is the hard part, and it requires a trick: we can write xu as (xp)+(pu), and then notice that puU. What can we say using the first part, and the Pythagorean theorem?

Proof.

By Definition 3.3.1, p is a linear combination of elements in U, so pU. The fact that xpU follows directly from the Orthogonal Lemma.
Choose any uU with up, and write
xu=(xp)+(pu).
Since puU and xpU, we know that these two vectors are orthogonal, and therefore,
xu2=xp2+pu2>xp2,

Exercise 3.3.6.

Show that UU={0}. Use this fact to show that Definition 3.3.1 does not depend on the choice orthogonal basis.
Hint.
Suppose we find vectors p and p using basis B and B. Note that ppU, but also that
pp=(px)(px)
Finally, we note one more useful fact. The process of sending a vector to its orthogonal projection defines an operator on Rn, and yes, it’s linear.

Strategy.

The fact that PU is linear follows from properties of the dot product, and some careful checking. We know that imPUU by definition of the projection, and you can show that PU acts as the identity on U using the Fourier expansion theorem.
If xU, then PU(x)=0 by definition of PU. (Recall that it is defined using dot products with vectors in U.) If xkerPU, use the Projection Theorem, to show that xU.

Remark 3.3.8.

It follows from this result and the Dimension Theorem that
dimU+dimU=n,
and since UU={0}, U is indeed a complement of U in the sense introduced in Theorem 1.8.10. It’s also fairly easy to see that dimU+dimU=n directly. If wU, and {u1,,uk} is a basis for U, then we have
wu1=0,,wuk=0,
and for an unknown w, this is simply a homogeneous system of k equations with n variables. Moreover, they are independent equations, since the ui form a basis. We thus expect nk free parameters in the general solution.

Exercise 3.3.10.

    Given subspaces U,W of Rn with UW={0}, if UW=Rn, then W=U.
  • True.

  • A subspace can have many complements, but only one orthgonal complement. For example, a complement to the x axis in R2 is given by any other line through the origin, but only the y axis is orthogonal.
  • False.

  • A subspace can have many complements, but only one orthgonal complement. For example, a complement to the x axis in R2 is given by any other line through the origin, but only the y axis is orthogonal.
Theorem 3.3.11 tells us that we can find a basis for U by solving the homogeneous system ATx=0. Make sure you can see why this is true!

Example 3.3.12.

Let U={(ab+3c,2a+b,3c,4ab+3c,a4c)|a,b,cR}R5. Determine a basis for U.
Solution.
First, we note that for a general element of U, we have
(ab+3c,2a+b,3c,4ab+3c,a4c)=a(1,2,0,4,1)+b(1,1,0,1,0)+c(3,0,3,3,4),
so {(1,2,0,4,1),(1,1,0,1,0),(3,0,3,3,4)} is a basis for U. (We have just shown that this set spans U; it is independent since the first two vectors are not parallel, and the third vector cannot be in the span of the first two, since its third entry is nonzero.) As in Theorem 3.3.11, we set A=[113210003413104].
To find a basis for U, we simply need to find the nullspace of AT, which we do below.

Exercises Exercises

1.

Prove that for any subspace URn, projU+projU is the identity operator on Rn.
Hint.
Given xRn, can you write it as a sum of an element of U and an element of U?

2.

Prove that for any subspace URn, (U)=U.
Hint.
Show that U(U), and then use Remark 3.3.8 to show that the two spaces must have the same dimension.

3.

Let U and W be subspaces of Rn. Prove that (U+W)=UW.
Hint.
One inclusion is easier than the other. Use Theorem 1.8.6 and Remark 3.3.8 to show that the dimensions must be equal.

4.

Given v=[538353], find the coordinates for v in the subspace W spanned by u1=[661] and u2=[2412].
Note that u1 and u2 are orthogonal.
v= u1+ u2

5.

Let W be the set of all vectors [xyx+y] with x and y real. Find a vector whose span is W.

6.

Let u=[2731] and v=[1443], and let W the subspace of R4 spanned by u and v. Find a basis of W, the orthogonal complement of W in R4.

7.

Let y=[536], u1=[462], and u2=[2210].
Compute the distance d from y to the plane in R3 spanned by u1 and u2.

8.

Given v=[710210], find the closest point to v in the subspace W spanned by [3241] and [25120].

9.

Find the orthogonal projection of v=[9219] onto the subspace W of R3 spanned by [162] and [6621].
You have attempted 1 of 13 activities on this page.