When we encountered various types of matrices in Chapter 5, it became apparent that a particular kind of matrix, the diagonal matrix, was much easier to use in computations. For example, if , then can be found, but its computation is tedious. If then
Even when presented with a non-diagonal matrix, we will see that it is sometimes possible to do a bit of work to be able to work with a diagonal matrix. This process is called diagonalization.
In a variety of applications it is beneficial to be able to diagonalize a matrix. In this section we will investigate what this means and consider a few applications. In order to understand when the diagonalization process can be performed, it is necessary to develop several of the underlying concepts of linear algebra.
By now, you realize that mathematicians tend to generalize. Once we have found a “good thing,” something that is useful, we apply it to as many different concepts as possible. In doing so, we frequently find that the “different concepts” are not really different but only look different. Four sentences in four different languages might look dissimilar, but when they are translated into a common language, they might very well express the exact same idea.
Early in the development of mathematics, the concept of a vector led to a variety of applications in physics and engineering. We can certainly picture vectors, or “arrows,” in the plane and even in the three-dimensional space. Does it make sense to talk about vectors in four-dimensional space, in ten-dimensional space, or in any other mathematical situation? If so, what is the essence of a vector? Is it its shape or the rules it follows? The shape in two- or three-space is just a picture, or geometric interpretation, of a vector. The essence is the rules, or properties, we wish vectors to follow so we can manipulate them algebraically. What follows is a definition of what is called a vector space. It is a list of all the essential properties of vectors, and it is the basic definition of the branch of mathematics called linear algebra.
Let be any nonempty set of objects. Define on an operation, called addition, for any two elements , and denote this operation by . Let scalar multiplication be defined for a real number and any element and denote this operation by . The set together with operations of addition and scalar multiplication is called a vector space over if the following hold for all , and :
There exists a vector , such that for all .
For each vector , there exists a unique vector , such that .
In a vector space it is common to call the elements of vectors and those from scalars. Vector spaces over the real numbers are also called real vector spaces.
Example12.3.2.A Vector Space of Matrices.
Let and let the operations of addition and scalar multiplication be the usual operations of addition and scalar multiplication on matrices. Then together with these operations is a real vector space. The reader is strongly encouraged to verify the definition for this example before proceeding further (see Exercise 3 of this section). Note we can call the elements of vectors even though they are not arrows.
Example12.3.3.The Vector Space .
Let . If we define addition and scalar multiplication the natural way, that is, as we would on matrices, then is a vector space over . See Exercise 12.3.3.4 of this section.
In this example, we have the “bonus” that we can illustrate the algebraic concept geometrically. In mathematics, a “geometric bonus” does not always occur and is not necessary for the development or application of the concept. However, geometric illustrations are quite useful in helping us understand concepts and should be utilized whenever available.
Sum of two vectors in
Figure12.3.4.Sum of two vectors in
Let’s consider some illustrations of the vector space . Let and . We illustrate the vector as a directed line segment, or “arrow,” from the point to the point. The vectors and are as shown in Figure 12.3.4 together with . The vector is a vector in the same direction as , but with twice its length.
In many situations a vector space is given and we would like to describe the whole vector space by the smallest number of essential reference vectors. An example of this is the description of , the -plane, via the and axes. Again our concepts must be algebraic in nature so we are not restricted solely to geometric considerations.
A vector in vector space (over ) is a linear combination of the vectors ,, if there exist scalars in such that
Example12.3.7.A Basic Example.
The vector in is a linear combination of the vectors and since .
Example12.3.8.A little less obvious example.
Prove that the vector is a linear combination of the vectors (3, 1) and (1, 4).
By the definition we must show that there exist scalars and such that:
This system has the solution ,.
Hence, if we replace and both by 1, then the two vectors (3, 1) and (1, 4) produce, or generate, the vector (4,5). Of course, if we replace and by different scalars, we can generate more vectors from . If, for example, and , then
Will the vectors and generate any vector we choose in ? To see if this is so, we let be an arbitrary vector in and see if we can always find scalars and such that . This is equivalent to solving the following system of equations:
Let be a set of vectors in a vector space over . This set is said to generate, or span, if, for any given vector , we can always find scalars ,, such that . A set that generates a vector space is called a generating set.
We know that the standard coordinate system, axis and axis, were introduced in basic algebra in order to describe all points in the -plane algebraically. It is also quite clear that to describe any point in the plane we need exactly two axes.
We can set up a new coordinate system in the following way. Draw the vector and an axis from the origin through (3, 1) and label it the axis. Also draw the vector and an axis from the origin through to be labeled the axis. Draw the coordinate grid for the axis, that is, lines parallel, and let the unit lengths of this “new” plane be the lengths of the respective vectors, and , so that we obtain Figure 12.3.10.
From Example 12.3.8 and Figure 12.3.10, we see that any vector on the plane can be described using the standard -axes or our new -axes. Hence the position which had the name in reference to the standard axes has the name with respect to the axes, or, in the phraseology of linear algebra, the coordinates of the point with respect to the axes are .
Example12.3.11.One point, Two position descriptions.
From Example 12.3.8 we found that if we choose and , then the two vectors and generate the vector . Another geometric interpretation of this problem is that the coordinates of the position with respect to the axes of Figure 12.3.10 is . In other words, a position in the plane has the name in reference to the -axes and the same position has the name in reference to the axes.
From the above, it is clear that we can use different axes to describe points or vectors in the plane. No matter what choice we use, we want to be able to describe each position in a unique manner. This is not the case in Figure 12.3.12. Any point in the plane could be described via the axes, the axes or the axes. Therefore, in this case, a single point would have three different names, a very confusing situation.
A set of vectors from a real vector space is linearly independent if the only solution to the equation is . Otherwise the set is called a linearly dependent set.
If is a basis for a vector space V over , then any vector can be uniquely expressed as a linear combination of the 's.
Proof.
Assume that is a basis for over . We must prove two facts:
each vector can be expressed as a linear combination of the 's, and
each such expression is unique.
Part 1 is trivial since a basis, by its definition, must generate all of .
The proof of part 2 is a bit more difficult. We follow the standard approach for any uniqueness facts. Let be any vector in and assume that there are two different ways of expressing , namely
and
where at least one is different from the corresponding . Then equating these two linear combinations we get
so that
Now a crucial observation: since the form a linearly independent set, the only solution to the previous equation is that each of the coefficients must equal zero, so for . Hence , for all . This contradicts our assumption that at least one is different from the corresponding , so each vector can be expressed in one and only one way.
This theorem, together with the previous examples, gives us a clear insight into the significance of linear independence, namely uniqueness in representing any vector.
Example12.3.16.Another basis for .
Prove that is a basis for over and explain what this means geometrically.
First we show that the vectors and generate all of . We can do this by imitating Example 12.3.8 and leave it to the reader (see Exercise 12.3.3.10 of this section). Secondly, we must prove that the set is linearly independent.
Let and be scalars such that . We must prove that the only solution to the equation is that and must both equal zero. The above equation becomes which gives us the system
The augmented matrix of this system reduces in such way that the only solution is the trivial one of all zeros:
To explain the results geometrically, note through Exercise 12, part a, that the coordinates of each vector can be determined uniquely using the vectors (1,1) and (-1, 1). The concept of dimension is quite obvious for those vector spaces that have an immediate geometric interpretation. For example, the dimension of is two and that of is three. How can we define the concept of dimension algebraically so that the resulting definition correlates with that of and ? First we need a theorem, which we will state without proof.
Verify that is a vector space over . What is its dimension?
Is a vector space over ? If so, what is its dimension?
Answer.
The dimension of is 6 and yes, is also a vector space of dimension . One basis for is where is the matrix with entries all equal to zero except for in row , column where the entry is 1.
Let ; that is, is the set of all polynomials in having real coefficients with degree less than or equal to three. Verify that is a vector space over . What is its dimension?
Prove that the sets in Exercise 9, parts e and f, form bases of the respective vector spaces.
Answer.
The set is linearly independent: let and be scalars such that , then and which has as its only solutions. The set generates all of : let be an arbitrary vector in . We want to show that we can always find scalars and such that . This is equivalent to finding scalars such that and . This system has a unique solution , and . Therefore, the set generates .
Are the vector spaces , and isomorphic to each other? Discuss with reference to previous parts of this exercise.
Answer.
The answer to the last part is that the three vector spaces are all isomorphic to one another. Once you have completed part (a) of this exercise, the following translation rules will give you the answer to parts (b) and (c),