Skip to main content
Logo image

Applied Discrete Structures

Section 12.3 An Introduction to Vector Spaces

Subsection 12.3.1 Motivation for the study of vector spaces

When we encountered various types of matrices in Chapter 5, it became apparent that a particular kind of matrix, the diagonal matrix, was much easier to use in computations. For example, if A=(2123), then A5 can be found, but its computation is tedious. If D=(1004) then
D5=(1004)5=(150045)=(1001024)
Even when presented with a non-diagonal matrix, we will see that it is sometimes possible to do a bit of work to be able to work with a diagonal matrix. This process is called diagonalization.
In a variety of applications it is beneficial to be able to diagonalize a matrix. In this section we will investigate what this means and consider a few applications. In order to understand when the diagonalization process can be performed, it is necessary to develop several of the underlying concepts of linear algebra.

Subsection 12.3.2 Vector Spaces

By now, you realize that mathematicians tend to generalize. Once we have found a “good thing,” something that is useful, we apply it to as many different concepts as possible. In doing so, we frequently find that the “different concepts” are not really different but only look different. Four sentences in four different languages might look dissimilar, but when they are translated into a common language, they might very well express the exact same idea.
Early in the development of mathematics, the concept of a vector led to a variety of applications in physics and engineering. We can certainly picture vectors, or “arrows,” in the xy plane and even in the three-dimensional space. Does it make sense to talk about vectors in four-dimensional space, in ten-dimensional space, or in any other mathematical situation? If so, what is the essence of a vector? Is it its shape or the rules it follows? The shape in two- or three-space is just a picture, or geometric interpretation, of a vector. The essence is the rules, or properties, we wish vectors to follow so we can manipulate them algebraically. What follows is a definition of what is called a vector space. It is a list of all the essential properties of vectors, and it is the basic definition of the branch of mathematics called linear algebra.

Definition 12.3.1. Vector Space.

Let V be any nonempty set of objects. Define on V an operation, called addition, for any two elements x,yV, and denote this operation by x+y. Let scalar multiplication be defined for a real number aR and any element xV and denote this operation by ax. The set V together with operations of addition and scalar multiplication is called a vector space over R if the following hold for all x,y,zV , and a,bR:
  • x+y=y+x
  • (x+y)+z=x+(y+z)
  • There exists a vector 0V, such that x+0=x for all xV.
  • For each vector xV, there exists a unique vector xV, such that x+x=0.
These are the main properties associated with the operation of addition. They can be summarized by saying that [V;+] is an abelian group.
The next four properties are associated with the operation of scalar multiplication and how it relates to vector addition.
  • a(x+y)=ax+ay
  • (a+b)x=ax+bx
  • a(bx)=(ab)x
  • 1x=x.
In a vector space it is common to call the elements of V vectors and those from R scalars. Vector spaces over the real numbers are also called real vector spaces.

Example 12.3.2. A Vector Space of Matrices.

Let V=M2×3(R) and let the operations of addition and scalar multiplication be the usual operations of addition and scalar multiplication on matrices. Then V together with these operations is a real vector space. The reader is strongly encouraged to verify the definition for this example before proceeding further (see Exercise 3 of this section). Note we can call the elements of M2×3(R) vectors even though they are not arrows.

Example 12.3.3. The Vector Space R2.

Let R2={(a1,a2)a1,a2R}. If we define addition and scalar multiplication the natural way, that is, as we would on 1×2 matrices, then R2 is a vector space over R. See Exercise 12.3.3.4 of this section.
In this example, we have the “bonus” that we can illustrate the algebraic concept geometrically. In mathematics, a “geometric bonus” does not always occur and is not necessary for the development or application of the concept. However, geometric illustrations are quite useful in helping us understand concepts and should be utilized whenever available.
described in detail following the image
Sum of two vectors in R2
Figure 12.3.4. Sum of two vectors in R2
Let’s consider some illustrations of the vector space R2. Let x=(1,4) and y=(3,1). We illustrate the vector (a1,a2) as a directed line segment, or “arrow,” from the point (0,0) to the point(a1,a2). The vectors x and y are as shown in Figure 12.3.4 together with x+y=(1,4)+(3,1)=(4,5). The vector 2x=2(1,4)=(2,8) is a vector in the same direction as x, but with twice its length.

Note 12.3.5.

  1. The common convention is to use that boldface letters toward the end of the alphabet for vectors, while letters early in the alphabet are scalars.
  2. A common alternate notation for vectors is to place an arrow about a variable to indicate that it is a vector such as this: x.
  3. The vector (a1,a2,,an)Rn is referred to as an n-tuple.
  4. For those familiar with vector calculus, we are expressing the vector x=a1i^+a2j^+a3k^R3 as (a1,a2,a3). This allows us to discuss vectors in Rn in much simpler notation.
In many situations a vector space V is given and we would like to describe the whole vector space by the smallest number of essential reference vectors. An example of this is the description of R2, the xy-plane, via the x and y axes. Again our concepts must be algebraic in nature so we are not restricted solely to geometric considerations.

Definition 12.3.6. Linear Combination.

A vector yy in vector space V (over R) is a linear combination of the vectors x1, x2,, xn if there exist scalars a1,a2,,an in R such that y=a1x1+a2x2++anxn

Example 12.3.7. A Basic Example.

The vector (2,3) in R2 is a linear combination of the vectors (1,0) and (0,1) since (2,3)=2(1,0)+3(0,1).

Example 12.3.8. A little less obvious example.

Prove that the vector (4,5) is a linear combination of the vectors (3, 1) and (1, 4).
By the definition we must show that there exist scalars a1 and a2 such that:
(4,5)=a1(3,1)+a2(1,4)=(3a1+a2,a1+4a2)3a1+a2=4a1+4a2=5
This system has the solution a1=1, a2=1.
Hence, if we replace a1 and a2 both by 1, then the two vectors (3, 1) and (1, 4) produce, or generate, the vector (4,5). Of course, if we replace a1 and a2 by different scalars, we can generate more vectors from R2. If, for example, a1=3 and a2=2, then
a1(3,1)+a2(1,4)=3(3,1)+(2)(1,4)=(9,3)+(2,8)=(7,5)
Will the vectors (3,1) and (1,4) generate any vector we choose in R2? To see if this is so, we let (b1,b2) be an arbitrary vector in R2 and see if we can always find scalars a1 and a2 such that a1(3,1)+a2(1,4)=(b1,b2). This is equivalent to solving the following system of equations:
3a1+a2=b1a1+4a2=b2
which always has solutions for a1 and a2 , regardless of the values of the real numbers b1 and b2. Why? We formalize this situation in a definition:

Definition 12.3.9. Generation of a Vector Space.

Let {x1,x2,,xn} be a set of vectors in a vector space V over R. This set is said to generate, or span, V if, for any given vector yV, we can always find scalars a1, a2,, an such that y=a1x1+a2x2++anxn. A set that generates a vector space is called a generating set.
We now give a geometric interpretation of the previous examples.
We know that the standard coordinate system, x axis and y axis, were introduced in basic algebra in order to describe all points in the xy-plane algebraically. It is also quite clear that to describe any point in the plane we need exactly two axes.
We can set up a new coordinate system in the following way. Draw the vector (3,1) and an axis from the origin through (3, 1) and label it the x axis. Also draw the vector (1,4) and an axis from the origin through (1,4) to be labeled the y axis. Draw the coordinate grid for the axis, that is, lines parallel, and let the unit lengths of this “new” plane be the lengths of the respective vectors, (3,1) and (1,4), so that we obtain Figure 12.3.10.
From Example 12.3.8 and Figure 12.3.10, we see that any vector on the plane can be described using the standard xy-axes or our new xy-axes. Hence the position which had the name (3,1) in reference to the standard axes has the name (1,0) with respect to the xy axes, or, in the phraseology of linear algebra, the coordinates of the point (1,4) with respect to the xy axes are (1,0).
Two sets of axes for the plane
Figure 12.3.10. Two sets of axes for the plane

Example 12.3.11. One point, Two position descriptions.

From Example 12.3.8 we found that if we choose a1=1 and a2=1, then the two vectors (3,1) and (1,4) generate the vector (4,5). Another geometric interpretation of this problem is that the coordinates of the position (4,5) with respect to the xy axes of Figure 12.3.10 is (1,1). In other words, a position in the plane has the name (4,5) in reference to the xy-axes and the same position has the name (1,1) in reference to the xy axes.
From the above, it is clear that we can use different axes to describe points or vectors in the plane. No matter what choice we use, we want to be able to describe each position in a unique manner. This is not the case in Figure 12.3.12. Any point in the plane could be described via the xy axes, the xz axes or the yz axes. Therefore, in this case, a single point would have three different names, a very confusing situation.
Three axes on a plane
Figure 12.3.12. Three axes on a plane
We formalize the our observations in the previous examples in two definitions and a theorem.

Definition 12.3.13. Linear Independence/Linear Dependence.

A set of vectors {x1,x2,,xn} from a real vector space V is linearly independent if the only solution to the equation a1x1+a2x2++anxn=0 is a1=a2==an=0. Otherwise the set is called a linearly dependent set.

Definition 12.3.14. Basis.

A set of vectors B={x1,x2,,xn} is a basis for a vector space V if:
  1. B generates V, and
  2. B is linearly independent.

Proof.

Assume that {x1,x2,,xn} is a basis for V over R. We must prove two facts:
  1. each vector yV can be expressed as a linear combination of the xi's, and
  2. each such expression is unique.
Part 1 is trivial since a basis, by its definition, must generate all of V.
The proof of part 2 is a bit more difficult. We follow the standard approach for any uniqueness facts. Let y be any vector in V and assume that there are two different ways of expressing y, namely
y=a1x1+a2x2++anxn
and
y=b1x1+b2x2++bnxn
where at least one ai is different from the corresponding bi. Then equating these two linear combinations we get
a1x1+a2x2++anxn=b1x1+b2x2++bnxn
so that
(a1b1)x1+(a2b2)x2++(anbn)xn=0
Now a crucial observation: since the xis form a linearly independent set, the only solution to the previous equation is that each of the coefficients must equal zero, so aibi=0 for i=1,2,,n. Hence ai=bi, for all i. This contradicts our assumption that at least one ai is different from the corresponding bi, so each vector yV can be expressed in one and only one way.
This theorem, together with the previous examples, gives us a clear insight into the significance of linear independence, namely uniqueness in representing any vector.

Example 12.3.16. Another basis for R2.

Prove that {(1,1),(1,1)} is a basis for R2 over R and explain what this means geometrically.
First we show that the vectors (1,1) and (1,1) generate all of R2. We can do this by imitating Example 12.3.8 and leave it to the reader (see Exercise 12.3.3.10 of this section). Secondly, we must prove that the set is linearly independent.
Let a1 and a2 be scalars such that a1(1,1)+a2(1,1)=(0,0). We must prove that the only solution to the equation is that a1 and a2 must both equal zero. The above equation becomes (a1a2,a1+a2)=(0,0) which gives us the system
a1a2=0a1+a2=0
The augmented matrix of this system reduces in such way that the only solution is the trivial one of all zeros:
(110110)(100010)  a1=a2=0
Therefore, the set is linearly independent.
To explain the results geometrically, note through Exercise 12, part a, that the coordinates of each vector yR2 can be determined uniquely using the vectors (1,1) and (-1, 1). The concept of dimension is quite obvious for those vector spaces that have an immediate geometric interpretation. For example, the dimension of R2 is two and that of R3 is three. How can we define the concept of dimension algebraically so that the resulting definition correlates with that of R2 and R3? First we need a theorem, which we will state without proof.

Definition 12.3.18. Dimension of a Vector Space.

Let V be a vector space over R with basis {x1,x2,,xn}. Then the dimension of V is n. We use the notation dimV=n to indicate that V is n-dimensional.

Exercises 12.3.3 Exercises

1.

If a=2, b=3, A=(101234), B=(223458), and C=(100322) verify that all properties of the definition of a vector space are true for M2×3(R) with these values.

2.

Let a=3, b=4, x==(1,3), y=(2,3),and z=(1,0). Verify that all properties of the definition of a vector space are true for R2 for these values.

3.

  1. Verify that M2×3(R) is a vector space over R. What is its dimension?
  2. Is Mm×n(R) a vector space over R? If so, what is its dimension?
Answer.
The dimension of M2×3(R) is 6 and yes, Mm×n(R) is also a vector space of dimension mn. One basis for Mm×n(R) is {Aij1im,1jn} where Aij is the m×n matrix with entries all equal to zero except for in row i, column j where the entry is 1.

4.

  1. Verify that R2 is a vector space over R.
  2. Is Rn a vector space over R for every positive integer n?

5.

Let P3={a0+a1x+a2x2+a3x3a0,a1,a2,a3R}; that is, P3 is the set of all polynomials in x having real coefficients with degree less than or equal to three. Verify that P3 is a vector space over R. What is its dimension?

6.

For each of the following, express the vector yy as a linear combination of the vectors x1 and x2.
  1. y=(5,6), x1=(1,0), and x2=(0,1)
  2. y=(2,1), x1=(2,1), and x2=(1,1)
  3. y=(3,4), x1=(1,1), and x2=(1,1)

7.

Express the vector (1233)M2×2(R), as a linear combination of (1111), (1521), (0111) and (0001)
Answer.
If the matrices are named B, A1, A2 , A3, and A4 , then
B=83A1+53A2+53A3+233A4

8.

Express the vector x34x2+3P3 as a linear combination of the vectors 1, x, x2 , and x3.

9.

  1. Show that the set {x1,x2} generates R2 for each of the parts in Exercise 6 of this section.
  2. Show that {x1,x2,x3} generates R2 where x1=(1,1), x2=(3,4), and x3=(1,5).
  3. Create a set of four or more vectors that generates R2.
  4. What is the smallest number of vectors needed to generate R2? Rn?
  5. Show that the set
    {A1,A2,A3,A4}={(1000),(0100),(0010),(0001)}
    generates M2×2(R)
  6. Show that {1,x,x2,x3} generates P3.
Answer.
  1. If x1=(1,0), x2=(0,1), and y=(b1,b2), then y=b1x1+b2x2. If x1=(3,2), x2=(2,1), and y=(b1,b2), then y=(b1+2b2)x1+(2b13b2)x2.
  2. If y=(b1,b2) is any vector in R2 , then y=(3b1+4b2)x1+(b1+b2)x2+(0)x3
  3. One solution is to add any vector(s) to x1, x2, and x3 of part b.
  4. 2, n
  5. (xyzw)=xA1+yA2+zA3+wA4
  6. a0+a1x+a2x2+a3x3=a0(1)+a1(x)+a2(x2)+a3(x3).

11.

  1. Prove that {(4,1),(1,3)} is a basis for R2 over R.
  2. Prove that {(1,0),(3,4)} is a basis for R2 over R.
  3. Prove that {(1,0,1),(2,1,1),(1,3,1)} is a basis for R3 over R.
  4. Prove that the sets in Exercise 9, parts e and f, form bases of the respective vector spaces.
Answer.
  1. The set is linearly independent: let a and b be scalars such that a(4,1)+b(1,3)=(0,0), then 4a+b=0 and a+3b=0 which has a=b=0 as its only solutions. The set generates all of R2: let (a,b) be an arbitrary vector in R2 . We want to show that we can always find scalars β1 and β2 such that β1(4,1)+β2(1,3)=(a,b). This is equivalent to finding scalars such that 4β1+β2=a and β1+3β2=b. This system has a unique solution β1=3ab11, and β2=4ba11. Therefore, the set generates R2.

12.

  1. Determine the coordinates of the points or vectors (3,4), (1,1), and (1,1) with respect to the basis {(1,1),(1,1)} of R2. Interpret your results geometrically.
  2. Determine the coordinates of the points or vector (3,5,6) with respect to the basis {(1,0,0),(0,1,0),(0,0,1)}. Explain why this basis is called the standard basis for R3.

13.

  1. Let y1=(1,3,5,9), y2=(5,7,6,3), and c=2. Find y1+y2 and cy1.
  2. Let f1(x)=1+3x+5x2+9x3 , f2(x)=5+7x+6x2+3x3 and c=2. Find f1(x)+f2(x) and cf1(x).
  3. Let A=(1359), B=(5763), and c=2. Find A+B and cA.
  4. Are the vector spaces R4 , P3 and M2×2(R) isomorphic to each other? Discuss with reference to previous parts of this exercise.
Answer.
The answer to the last part is that the three vector spaces are all isomorphic to one another. Once you have completed part (a) of this exercise, the following translation rules will give you the answer to parts (b) and (c),
(a,b,c,d)(abcd)a+bx+cx2+dx2
You have attempted of activities on this page.