FCLA Types of Solution Sets

Section TSS Types of Solution Sets

We will now be more careful about analyzing the reduced row-echelon form derived from the augmented matrix of a system of linear equations. In particular, we will see how to systematically handle the situation when we have infinitely many solutions to a system, and we will prove that every system of linear equations has either zero, one or infinitely many solutions. With these tools, we will be able to routinely solve any linear system.

🔗

Subsection CS Consistent Systems

The computer scientist Donald Knuth said, “Science is what we understand well enough to explain to a computer. Art is everything else.” In this section we will remove solving systems of equations from the realm of art, and into the realm of science. We begin with a definition.

🔗

Definition CS. Consistent System.

A system of linear equations is consistent if it has at least one solution. Otherwise, the system is called inconsistent.

🔗

We will want to first recognize when a system is inconsistent or consistent, and in the case of consistent systems we will be able to further refine the types of solutions possible. We will do this by analyzing the reduced row-echelon form of a matrix, using the value of

r,

and the sets of column indices,

D

and

F,

first defined back in Definition RREF.

🔗

Use of the notation for the elements of

D

and

F

can be a bit confusing, since we have subscripted variables that are in turn equal to integers used to index the matrix. However, many questions about matrices and systems of equations can be answered once we know

r,

D

and

F .

The choice of the letters

D

and

F

refer to our upcoming definition of dependent and free variables (Definition IDV). An example will help us begin to get comfortable with this aspect of reduced row-echelon form.

🔗

Example RREFN. Reduced row-echelon form notation.

For the

5 \times 9

matrix

\begin{aligned} B & = [\begin{array}{c} 1 & 5 & 0 & 0 & 2 & 8 & 0 & 5 & - 1 \\ 0 & 0 & 1 & 0 & 4 & 7 & 0 & 2 & 0 \\ 0 & 0 & 0 & 1 & 3 & 9 & 0 & 3 & - 6 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 4 & 2 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array}] \end{aligned}

in reduced row-echelon form we have

\begin{aligned} r & = 4 \\ d_{1} & = 1 & d_{2} & = 3 & d_{3} & = 4 & d_{4} & = 7 \\ f_{1} & = 2 & f_{2} & = 5 & f_{3} & = 6 & f_{4} & = 8 & f_{5} & = 9 . \end{aligned}

Notice that the sets

\begin{aligned} D & = {d_{1}, d_{2}, d_{3}, d_{4}} = {1, 3, 4, 7} & F = {f_{1}, f_{2}, f_{3}, f_{4}, f_{5}} = {2, 5, 6, 8, 9} \end{aligned}

have nothing in common and together account for all of the columns of

B

(we say it is a partition of the set of column indices).

The number

r

is the single most important piece of information we can get from the reduced row-echelon form of a matrix. It is defined as the number of nonzero rows, but since each nonzero row has a leading 1, it is also the number of leading 1’s present. For each leading 1, we have a pivot column, so

r

is also the number of pivot columns. Repeating ourselves,

r

is the number of nonzero rows, the number of leading 1’s and the number of pivot columns. Across different situations, each of these interpretations of the meaning of

r

will be useful, though it may be most helpful to think in terms of pivot columns.

🔗

Before proving some theorems about the possibilities for solution sets to systems of equations, let us analyze one particular system with an infinite solution set very carefully as an example. We will use this technique frequently, and shortly we will refine it slightly.

🔗

Archetypes I and J are both fairly large for doing computations by hand (though not impossibly large). Their properties are very similar, so we will frequently analyze the situation in Archetype I, and leave you the joy of analyzing Archetype J yourself. So work through Archetype I with the text, by hand and/or with a computer, and then tackle Archetype J yourself (and check your results with those listed). Notice too that the archetypes describing systems of equations each lists the values of

r,

D

and

F .

Here we go…

🔗

Example ISSI. Describing infinite solution sets, Archetype I.

Archetype I is the system of

m = 4

equations in

n = 7

variables.

\begin{aligned} x_{1} + 4 x_{2} - x_{4} + 7 x_{6} - 9 x_{7} & = 3 \\ 2 x_{1} + 8 x_{2} - x_{3} + 3 x_{4} + 9 x_{5} - 13 x_{6} + 7 x_{7} & = 9 \\ 2 x_{3} - 3 x_{4} - 4 x_{5} + 12 x_{6} - 8 x_{7} & = 1 \\ - x_{1} - 4 x_{2} + 2 x_{3} + 4 x_{4} + 8 x_{5} - 31 x_{6} + 37 x_{7} & = 4 \end{aligned}

This system has a

4 \times 8

augmented matrix that is row-equivalent to the following matrix (check this!), and which is in reduced row-echelon form (the existence of this matrix is guaranteed by Theorem REMEF and its uniqueness is guaranteed by Theorem RREFU).

\begin{array}{r} [\begin{array}{c} 1 & 4 & 0 & 0 & 2 & 1 & - 3 & 4 \\ 0 & 0 & 1 & 0 & 1 & - 3 & 5 & 2 \\ 0 & 0 & 0 & 1 & 2 & - 6 & 6 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array}] \end{array}

So we find that

r = 3

and

\begin{aligned} D & = {d_{1}, d_{2}, d_{3}} = {1, 3, 4} & F & = {f_{1}, f_{2}, f_{3}, f_{4}, f_{5}} = {2, 5, 6, 7, 8} \end{aligned}

Let

i

denote any one of the

r = 3

nonzero rows. Then the index

d_{i}

is a pivot column. It will be easy in this case to use the equation represented by row

i

to write an expression for the variable

x_{d_{i}} .

It will be a linear function of the variables

x_{f_{1}}, x_{f_{2}}, x_{f_{3}}, x_{f_{4}}

(notice that

f_{5} = 8

does not reference a variable, but does tell us the final column is not a pivot column). We will now construct these three expressions. Notice that using subscripts upon subscripts takes some getting used to.

\begin{aligned} (i = 1) & x_{d_{1}} & = x_{1} = 4 - 4 x_{2} - 2 x_{5} - x_{6} + 3 x_{7} \\ (i = 2) & x_{d_{2}} & = x_{3} = 2 - x_{5} + 3 x_{6} - 5 x_{7} \\ (i = 3) & x_{d_{3}} & = x_{4} = 1 - 2 x_{5} + 6 x_{6} - 6 x_{7} \end{aligned}

Each element of the set

F = {f_{1}, f_{2}, f_{3}, f_{4}, f_{5}} = {2, 5, 6, 7, 8}

is the index of a variable, except for

f_{5} = 8 .

We refer to

x_{f_{1}} = x_{2},

x_{f_{2}} = x_{5},

x_{f_{3}} = x_{6}

and

x_{f_{4}} = x_{7}

as free (or independent) variables since they are allowed to assume any possible combination of values that we can imagine and we can continue on to build a solution to the system by solving individual equations for the values of the other (dependent) variables.

Each element of the set

D = {d_{1}, d_{2}, d_{3}} = {1, 3, 4}

is the index of a variable. We refer to the variables

x_{d_{1}} = x_{1},

x_{d_{2}} = x_{3}

and

x_{d_{3}} = x_{4}

as “dependent” variables since they depend on the independent variables. More precisely, for each possible choice of values for the independent variables we get exactly one set of values for the dependent variables that combine to form a solution of the system.

To express the solutions as a set, we write

{[\begin{matrix} 4 - 4 x_{2} - 2 x_{5} - x_{6} + 3 x_{7} \\ x_{2} \\ 2 - x_{5} + 3 x_{6} - 5 x_{7} \\ 1 - 2 x_{5} + 6 x_{6} - 6 x_{7} \\ x_{5} \\ x_{6} \\ x_{7} \end{matrix}] | x_{2}, x_{5}, x_{6}, x_{7} \in C}

The condition that

x_{2}, x_{5}, x_{6}, x_{7} \in C

is how we specify that the variables

x_{2}, x_{5}, x_{6}, x_{7}

are “free” to assume any possible values.

This systematic approach to solving a system of equations will allow us to create a precise description of the solution set for any consistent system once we have found the reduced row-echelon form of the augmented matrix. It will work just as well when the set of free variables is empty and we get just a single solution. And we could program a computer to do it! Now have a whack at Archetype J (Exercise TSS.C10), mimicking the discussion in this example. We’ll still be here when you get back.

Using the reduced row-echelon form of the augmented matrix of a system of equations to determine the nature of the solution set of the system is a very key idea. So let us look at one more example like the last one. But first a definition, and then the example. We mix our metaphors a bit when we call variables free versus dependent. Maybe we should call dependent variables “enslaved”?

🔗

Definition IDV. Independent and Dependent Variables.

Suppose

A

is the augmented matrix of a consistent system of linear equations and

B

is a row-equivalent matrix in reduced row-echelon form. Suppose

j

is the index of a pivot column of

B .

Then the variable

x_{j}

is dependent. A variable that is not dependent is called independent or free.

🔗

If you studied this definition carefully, you might wonder what to do if the system has

n

variables and column

n + 1

is a pivot column? We will see shortly, by Theorem RCLS, that this never happens for a consistent system.

🔗

Example FDV. Free and dependent variables.

Consider the system of five equations in five variables,

\begin{aligned} x_{1} - x_{2} - 2 x_{3} + x_{4} + 11 x_{5} & = 13 \\ x_{1} - x_{2} + x_{3} + x_{4} + 5 x_{5} & = 16 \\ 2 x_{1} - 2 x_{2} + x_{4} + 10 x_{5} & = 21 \\ 2 x_{1} - 2 x_{2} - x_{3} + 3 x_{4} + 20 x_{5} & = 38 \\ 2 x_{1} - 2 x_{2} + x_{3} + x_{4} + 8 x_{5} & = 22 \end{aligned}

whose augmented matrix row-reduces to

[\begin{matrix} 1 & - 1 & 0 & 0 & 3 & 6 \\ 0 & 0 & 1 & 0 & - 2 & 1 \\ 0 & 0 & 0 & 1 & 4 & 9 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{matrix}] .

Columns 1, 3 and 4 are pivot columns, so

D = {1, 3, 4} .

From this we know that the variables

x_{1},

x_{3}

and

x_{4}

will be dependent variables, and each of the

r = 3

nonzero rows of the row-reduced matrix will yield an expression for one of these three variables. The set

F

is all the remaining column indices,

F = {2, 5, 6} .

The column index

6

F

means that the final column is not a pivot column, and thus the system is consistent (Theorem RCLS). The remaining indices in

F

indicate free variables, so

x_{2}

and

x_{5}

(the remaining variables) are our free variables. The resulting three equations that describe our solution set are then

\begin{aligned} (x_{d_{1}} = x_{1}) & x_{1} & = 6 + x_{2} - 3 x_{5} \\ (x_{d_{2}} = x_{3}) & x_{3} & = 1 + 2 x_{5} \\ (x_{d_{3}} = x_{4}) & x_{4} & = 9 - 4 x_{5} . \end{aligned}

Make sure you understand where these three equations came from, and notice how the location of the pivot columns determined the variables on the left-hand side of each equation. We can compactly describe the solution set as,

S = {[\begin{matrix} 6 + x_{2} - 3 x_{5} \\ x_{2} \\ 1 + 2 x_{5} \\ 9 - 4 x_{5} \\ x_{5} \end{matrix}] | x_{2}, x_{5} \in C}

Notice how we express the freedom for

x_{2}

and

x_{5} :

x_{2}, x_{5} \in C .

Sage FDV. Free and Dependent Variables.

Sage has the matrix method .pivot() to quickly and easily identify the pivot columns of the reduced row-echelon form of a matrix. Notice that we do not have to row-reduce the matrix first, we just ask which columns of a matrix

A

would be the pivot columns of the matrix

B

that is row-equivalent to

A

and in reduced row-echelon form. By Definition IDV, the indices of the pivot columns for an augmented matrix of a system of equations are the indices of the dependent variables. And the remainder are free variables. But be careful, Sage numbers columns starting from zero and mathematicians typically number variables starting from one.

Let us reprise Example ISSI.

xxxxxxxxxx
 
coeff = matrix(QQ, [[ 1,  4, 0, -1,  0,   7, -9],
                    [ 2,  8,-1,  3,  9, -13,  7],
                    [ 0,  0, 2, -3, -4,  12, -8],
                    [-1, -4, 2,  4,  8, -31, 37]])
const = vector(QQ, [3, 9, 1, 4])
aug = coeff.augment(const)
dependent = aug.pivots()
dependent

So, incrementing each column index by 1 gives us the same set

D

of indices for the dependent variables. To get the free variables, we can use the following code. Study it and then read the explanation following.

xxxxxxxxxx
 
free = [index for index in range(7) if not index in dependent]
free

This is a Python programming construction known as a list comprehension but in this setting I prefer to call it “set builder notation.” Let us dissect the command in pieces. The brackets ([,]) create a new list. The items in the list will be values of the variable index. range(7) is another list, integers starting at 0 and stopping just before 7. (While perhaps a bit odd, this works very well when we consistently start counting at zero.) So range(7) is the list [0,1,2,3,4,5,6]. Think of these as candidate values for index, which are generated by for index in range(7). Then we test each candidate, and keep it in the new list if it is not in the list dependent.

This is entirely analogous to the following mathematics:

\begin{aligned} F & = {f | 1 \leq f \leq 7, f \notin D} \end{aligned}

where

F

is free,

f

is index, and

D

is dependent, and we make the 0/1 counting adjustments. This ability to construct sets in Sage with notation so closely mirroring the mathematics is a powerful feature worth mastering. We will use it repeatedly.

It was a good exercise to use a list comprehension to form the list of columns that are not pivot columns. However, Sage has us covered.

xxxxxxxxxx
 
free_and_easy = coeff.nonpivots()
free_and_easy

Can you use this new matrix method to make a simpler version of the consistent() function we designed above?

Sets are an important part of algebra, and we have seen a few already. Being comfortable with sets is important for understanding and writing proofs. If you have not already, pay a visit now to Section SET.

🔗

We can now use the values of

m,

n,

r,

and the independent and dependent variables to categorize the solution sets for linear systems through a sequence of theorems.

🔗

Through the following sequence of proofs, you will want to consult three proof techniques. See Proof Technique E, Proof Technique N, Proof Technique CP.

🔗

First we have an important theorem that explores the distinction between consistent and inconsistent linear systems.

🔗

Theorem RCLS. Recognizing Consistency of a Linear System.

Suppose

A

is the augmented matrix of a system of linear equations with

n

variables. Suppose also that

B

is a row-equivalent matrix in reduced row-echelon form with

r

nonzero rows. Then the system of equations is inconsistent if and only if column

n + 1

B

is a pivot column.

🔗

Proof.

(⇐)

The first half of the proof begins with the assumption that column

n + 1

B

is a pivot column. Then the leading 1 of row

r

is located in column

n + 1

B

and so row

r

B

begins with

n

consecutive zeros, finishing with the leading 1. This is a representation of the equation

0 = 1,

which is false. Since this equation is false for any collection of values we might choose for the variables, there are no solutions for the system of equations, and the system is inconsistent.

(⇒)

For the second half of the proof, we wish to show that if we assume the system is inconsistent, then column

n + 1

B

is a pivot column. But instead of proving this directly, we will form the logically equivalent statement that is the contrapositive, and prove that instead (see Proof Technique CP). Turning the implication around, and negating each portion, we arrive at the logically equivalent statement: if column

n + 1

B

is not a pivot column, then the system of equations is consistent.

If column

n + 1

B

is not a pivot column, the leading 1 for row

r

is located somewhere in columns 1 through

n .

Then every preceding row’s leading 1 is also located in columns 1 through

n .

In other words, since the last leading 1 is not in the last column, no leading 1 for any row is in the last column, due to the echelon layout of the leading 1’s (Definition RREF). We will now construct a solution to the system by setting each dependent variable to the entry of the final column in the row with the corresponding leading 1, and setting each free variable to zero. That sentence is pretty vague, so let us be more precise. Using our notation for the sets

D

and

F

from the reduced row-echelon form (Definition RREF):

\begin{aligned} x_{d_{i}} & = {[B]}_{i, n + 1}, 1 \leq i \leq r & x_{f_{i}} & = 0, 1 \leq i \leq n - r \end{aligned}

These values for the variables make the equations represented by the first

r

rows of

B

all true (convince yourself of this). Rows numbered greater than

r

(if any) are all zero rows, hence represent the equation

0 = 0

and are also all true. We have now identified one solution to the system represented by

B,

and hence a solution to the system represented by

A

(Theorem REMES). So we can say the system is consistent (Definition CS).

The beauty of this theorem being an equivalence is that we can unequivocally test to see if a system is consistent or inconsistent by looking at just a single entry of the reduced row-echelon form matrix. We could program a computer to do it!

🔗

Sage RCLS. Recognizing Consistency of a Linear System.

Another way of expressing Theorem RCLS is to say a system is consistent if and only if column

n + 1

is not a pivot column of

B .

Sage has the matrix method .pivot() to easily identify the pivot columns of a matrix. Let us use Archetype E as an example.

xxxxxxxxxx
 
coeff = matrix(QQ, [[ 2, 1,  7, -7],
                    [-3, 4, -5, -6],
                    [ 1, 1,  4, -5]])
const = vector(QQ, [2, 3, 2])
aug = coeff.augment(const)
aug.rref()

xxxxxxxxxx
 
aug.pivots()

We can look at the reduced row-echelon form of the augmented matrix and see a pivot column in the final column, so we know the system is inconsistent. However, we could just as easily not form the reduced row-echelon form and just look at the list of pivot columns computed by aug.pivots(). Since aug has 5 columns, the final column is numbered 4, which is present in the list of pivot columns, as we expect.

One feature of Sage is that we can easily extend its capabilities by defining new commands. Here we will create a function that checks if an augmented matrix represents a consistent system of equations. The syntax is just a bit complicated. lambda is the word that indicates we are making a new function, the input is temporarily named A (think Augmented), and the name of the function is consistent. Everything following the colon will be evaluated and reported back as the output.

xxxxxxxxxx
 
consistent = lambda A: not(A.ncols()-1 in A.pivots())

Execute this block above. There will not be any output, but now the consistent function will be defined and available. Now give it a try (after making sure to have run the code above that defines aug). Note that the output of consistent() will be either True or False.

xxxxxxxxxx
 
consistent(aug)

The consistent() command works by simply checking to see if the last column of A is not in the list of pivots. We can now test many different augmented matrices, such as perhaps changing the vector of constants while keeping the coefficient matrix fixed. Again, make sure you execute the code above that defines coeff and const.

xxxxxxxxxx
 
consistent(coeff.augment(const))

xxxxxxxxxx
 
w = vector(QQ, [3,1,2])
consistent(coeff.augment(w))

xxxxxxxxxx
 
u = vector(QQ, [1,3,1])
consistent(coeff.augment(u))

Why do some vectors of constants lead to a consistent system with this coefficient matrix, while others do not? This is a fundamental question, which we will come to understand in several different ways.

Notice that for a consistent system the row-reduced augmented matrix has

n + 1 \in F,

so the largest element of

F

does not refer to a variable. Also, for an inconsistent system,

n + 1 \in D,

and it then does not make much sense to discuss whether or not variables are free or dependent since there is no solution. Take a look back at Definition IDV and see why we did not need to consider the possibility of referencing

x_{n + 1}

as a dependent variable.

🔗

With the characterization of Theorem RCLS, we can explore the relationships between

r

and

n

for a consistent system. We can distinguish between the case of a unique solution and infinitely many solutions, and furthermore, we recognize that these are the only two possibilities.

🔗

Theorem CSRN. Consistent Systems, $r$ and $n$ .

Suppose

A

is the augmented matrix of a consistent system of linear equations with

n

variables. Suppose also that

B

is a row-equivalent matrix in reduced row-echelon form with

r

pivot columns. Then

r \leq n .

r = n,

then the system has a unique solution, and if

r < n,

then the system has infinitely many solutions.

🔗

Proof.

This theorem contains three implications that we must establish. Notice first that

B

has

n + 1

columns, so there can be at most

n + 1

pivot columns, i.e.

r \leq n + 1 .

r = n + 1,

then every column of

B

is a pivot column, and in particular, the last column is a pivot column. So Theorem RCLS tells us that the system is inconsistent, contrary to our hypothesis. We are left with

r \leq n .

When

r = n,

we find

n - r = 0

free variables (i.e.

F = {n + 1}

) and the only solution is given by setting the

n

variables to the the first

n

entries of column

n + 1

B .

When

r < n,

we have

n - r > 0

free variables. Choose one free variable and set all the other free variables to zero. Now, set the chosen free variable to any fixed value. It is possible to then determine the values of the dependent variables to create a solution to the system. By setting the chosen free variable to different values, in this manner we can create infinitely many solutions.

🔗

Subsection FV Free Variables

The next theorem simply states a conclusion from the final paragraph of the previous proof, allowing us to state explicitly the number of free variables for a consistent system.

🔗

Theorem FVCS. Free Variables for Consistent Systems.

Suppose

A

is the augmented matrix of a consistent system of linear equations with

n

variables. Suppose also that

B

is a row-equivalent matrix in reduced row-echelon form with

r

pivot columns. Then the solution set can be described with

n - r

free variables.

🔗

Proof.

See the proof of Theorem CSRN.

Example CFV. Counting free variables.

For each archetype that is a system of equations, the values of

n

and

r

are listed. Many also contain a few sample solutions. We can use this information profitably, as illustrated by four examples.

Archetype A has $n = 3$ and $r = 2 .$ It can be seen to be consistent by the sample solutions given. Its solution set then has $n - r = 1$ free variables, and therefore will be infinite.
Archetype B has $n = 3$ and $r = 3 .$ It can be seen to be consistent by the single sample solution given. Its solution set can then be described with $n - r = 0$ free variables, and therefore will have just the single solution.
Archetype H has $n = 2$ and $r = 3 .$ In this case, column 3 must be a pivot column, so by Theorem RCLS, the system is inconsistent. We should not try to apply Theorem FVCS to count free variables, since the theorem only applies to consistent systems. (What would happen if you did try to incorrectly apply Theorem FVCS?)
Archetype E has $n = 4$ and $r = 3 .$ However, by looking at the reduced row-echelon form of the augmented matrix, we find that column 5 is a pivot column. By Theorem RCLS we recognize the system as inconsistent.

We have accomplished a lot so far, but our main goal has been the following theorem, which is now very simple to prove. The proof is so simple that we ought to call it a corollary, but the result is important enough that it deserves to be called a theorem. (See Proof Technique LC.) Notice that this theorem was presaged first by Example TTS and further foreshadowed by other examples.

🔗

Theorem PSSLS. Possible Solution Sets for Linear Systems.

A system of linear equations has no solutions, a unique solution or infinitely many solutions.

🔗

Proof.

By its definition, a system is either inconsistent or consistent (Definition CS). The first case describes systems with no solutions. For consistent systems, we have the remaining two possibilities as guaranteed by, and described in, Theorem CSRN.

Here is a diagram that consolidates several of our theorems from this section, and which is of practical use when you analyze systems of equations. Note this presumes we have the reduced row-echelon form of the augmented matrix of the system to analyze.

🔗

Figure DTSLS. Decision Tree for Solving Linear Systems
🔗

We have one more theorem to round out our set of tools for determining solution sets to systems of linear equations.

🔗

Theorem CMVEI. Consistent, More Variables than Equations, Infinite solutions.

Suppose a consistent system of linear equations has

m

equations in

n

variables. If

n > m,

then the system has infinitely many solutions.

🔗

Proof.

Suppose that the augmented matrix of the system of equations is row-equivalent to

B,

a matrix in reduced row-echelon form with

r

nonzero rows. Because

B

has

m

rows in total, the number of nonzero rows is less than or equal to

m .

In other words,

r \leq m .

Follow this with the hypothesis that

n > m

and we find that the system has a solution set described by at least one free variable because

n - r \geq n - m > 0 .

A consistent system with free variables will have an infinite number of solutions, as given by Theorem CSRN.

Notice that to use this theorem we need only know that the system is consistent, together with the values of

m

and

n .

We do not necessarily have to compute a row-equivalent reduced row-echelon form matrix, even though we discussed such a matrix in the proof. This is the substance of the following example.

🔗

Example OSGMD. One solution gives many, Archetype D.

Archetype D is the system of

m = 3

equations in

n = 4

variables,

\begin{aligned} 2 x_{1} + x_{2} + 7 x_{3} - 7 x_{4} & = 8 \\ - 3 x_{1} + 4 x_{2} - 5 x_{3} - 6 x_{4} & = - 12 \\ x_{1} + x_{2} + 4 x_{3} - 5 x_{4} & = 4 \end{aligned}

and the solution

x_{1} = 0,

x_{2} = 1,

x_{3} = 2,

x_{4} = 1

can be checked easily by substitution. Having been handed this solution, we know the system is consistent. This, together with

n > m,

allows us to apply Theorem CMVEI and conclude that the system has infinitely many solutions.

These theorems give us the procedures and implications that allow us to completely solve any system of linear equations. The main computational tool is using row operations to convert an augmented matrix into reduced row-echelon form. Here is a broad outline of how we would instruct a computer to solve a system of linear equations.

🔗

Represent a system of linear equations in $n$ variables by an augmented matrix (an array is the appropriate data structure in most computer languages).
Convert the matrix to a row-equivalent matrix in reduced row-echelon form using the procedure from the proof of Theorem REMEF. Identify the location of the pivot columns, and their number $r .$
If column $n + 1$ is a pivot column, output the statement that the system is inconsistent and halt.
If column $n + 1$ is not a pivot column, there are two possibilities:
1. $r = n$ and the solution is unique. It can be read off directly from the entries in rows 1 through $n$ of column $n + 1 .$
2. $r < n$ and there are infinitely many solutions. If only a single solution is needed, set all the free variables to zero and read off the dependent variable values from column $n + 1,$ as in the second half of the proof of Theorem RCLS. If the entire solution set is required, figure out some nice compact way to describe it, since your finite computer is not big enough to hold all the solutions (we will have such a way soon).

🔗

The above makes it all sound a bit simpler than it really is. In practice, row operations employ division (usually to get a leading entry of a row to convert to a leading 1) and that will introduce round-off errors. Entries that should be zero sometimes end up being very, very small nonzero entries, or small entries lead to overflow errors when used as divisors. A variety of strategies can be employed to minimize these sorts of errors, and this is one of the main topics in the important subject known as numerical linear algebra.

🔗

In this section we have gained a foolproof procedure for solving any system of linear equations, no matter how many equations or variables. We also have a handful of theorems that allow us to determine partial information about a solution set without actually constructing the whole set itself. Donald Knuth would be proud.

🔗

Sage SS1. Solving Systems, Part 1.

Sage has a built-in command that will solve a linear system of equations, given a coefficient matrix and a vector of constants. We need to learn some more theory before we can entirely understand this command, but we can begin to explore its use. For now, consider these methods experimental and do not let it replace row-reducing augmented matrices.

The matrix method A.solve_right(b) will provide information about solutions to the linear system of equations with coefficient matrix A and vector of constants b. The reason for the “right” (and the corresponding command named with “left”) will have to wait for Sage MVP. For now, it is generally correct in this course to use the “right” variant of any Sage linear algebra command that has both “left” and “right” variants.

Let us apply the .solve_right() command to a system with no solutions, in particular Archetype E. We have already seen in Sage RCLS that this system is inconsistent.

xxxxxxxxxx
 
coeff = matrix(QQ, [[ 2, 1,  7, -7],
                    [-3, 4, -5, -6],
                    [ 1, 1,  4, -5]])
const = vector(QQ, [2, 3, 2])
coeff.solve_right(const)

This is our first discussion of Sage error messages, though undoubtedly you have seen several already! First, here we only show the first and last lines of the message since typically it contains a lot of information specific to whichever computer you may be using. But we always begin with the last line as the most important indication of what has happened. Here the “problem” is quite evident: we get an “error” message telling us that the matrix equation has no solutions. We can debate whether or not this is really an error, but that is the design decision taken in Sage — we just need to be aware of it, the .solve_right() is really only valuable when there is a solution.

Generally, when deciphering Sage error messages, you want to start at the bottom of the traceback and read up through the various routines that have been called. Execute the block above and you may see references to matrix methods such as ._solve_right_general() and then .solve_right(). With time and practice, these mysterious messages will become more and more helpful, so spend some time reading them in tandem with locating the real source of any problems you encounter.

What does .solve_right() do with a system that does have solutions? Let us take a look at Example ISSI again, as we did in Sage FDV.

xxxxxxxxxx
 
coeff = matrix(QQ, [[ 1,  4,  0, -1,  0,   7, -9],
                    [ 2,  8, -1,  3,  9, -13,  7],
                    [ 0,  0,  2, -3, -4,  12, -8],
                    [-1, -4,  2,  4,  8, -31, 37]])
const = vector(QQ, [3, 9, 1, 4])
coeff.solve_right(const)

This vector with 7 entries is indeed a solution to the system of equations (check this!). But from Example ISSI we know this system has infinitely many solutions. Why does Sage give us just one solution? Of the infinitely many solutions, why this one? How can a finite computer ever present us with infinitely many solutions? Do we have the time to read through an infinite list of solutions? Is there a “best” solution? This behavior should prompt these questions, and maybe more.

In order to totally understand the behavior of the .solve_right() command, we need to understand more of the theory of linear algebra. In good time. So for now, .solve_right() is a curiosity we will fully understand soon — specifically in Sage SS2 and Sage SS3.

🔗

Reading Questions TSS Reading Questions

1. How to analyze consistency?

How can we easily recognize when a system of linear equations is inconsistent or not?

🔗

2. How to identify independent/dependent variables?

Suppose we have converted the augmented matrix of a system of equations into reduced row-echelon form. How do we then identify the dependent and independent (free) variables?

🔗

3. Possibilities for solution sets.

What are the possible solution sets for a system of linear equations?

🔗

Exercises TSS Exercises

C10.

In the spirit of Example ISSI, describe the infinite solution set for Archetype J.

🔗

Exercise Group.

For Exercises C21–C28, find the solution set of the system of linear equations. Give the values of

n

and

r,

and interpret your answers in light of the theorems of this section.

🔗

C21.

\begin{aligned} x_{1} + 4 x_{2} + 3 x_{3} - x_{4} & = 5 \\ x_{1} - x_{2} + x_{3} + 2 x_{4} & = 6 \\ 4 x_{1} + x_{2} + 6 x_{3} + 5 x_{4} & = 9 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are

\begin{array}{r} [\begin{array}{c} 1 & 4 & 3 & - 1 & 5 \\ 1 & - 1 & 1 & 2 & 6 \\ 4 & 1 & 6 & 5 & 9 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 7 / 5 & 7 / 5 & 0 \\ 0 & 1 & 2 / 5 & - 3 / 5 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{array}] . \end{array}

For this system, we have

n = 4

and

r = 3 .

However, with a pivot column in the last column we see that the original system has no solution by Theorem RCLS.

🔗

C22.

\begin{aligned} x_{1} - 2 x_{2} + x_{3} - x_{4} & = 3 \\ 2 x_{1} - 4 x_{2} + x_{3} + x_{4} & = 2 \\ x_{1} - 2 x_{2} - 2 x_{3} + 3 x_{4} & = 1 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & - 2 & 1 & - 1 & 3 \\ 2 & - 4 & 1 & 1 & 2 \\ 1 & - 2 & - 2 & 3 & 1 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & - 2 & 0 & 0 & 3 \\ 0 & 0 & 1 & 0 & - 2 \\ 0 & 0 & 0 & 1 & - 2 \end{array}] . \end{array}

Thus, we see we have an equivalent system for any scalar

x_{2} :

\begin{aligned} x_{1} & = 3 + 2 x_{2} \\ x_{3} & = - 2 \\ x_{4} & = - 2 . \end{aligned}

For this system,

n = 4

and

r = 3 .

Since it is a consistent system by Theorem RCLS, Theorem CSRN guarantees an infinite number of solutions.

🔗

C23.

\begin{aligned} x_{1} - 2 x_{2} + x_{3} - x_{4} & = 3 \\ x_{1} + x_{2} + x_{3} - x_{4} & = 1 \\ x_{1} + x_{3} - x_{4} & = 2 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & - 2 & 1 & - 1 & 3 \\ 1 & 1 & 1 & - 1 & 1 \\ 1 & 0 & 1 & - 1 & 2 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 1 & - 1 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{array}] . \end{array}

For this system, we have

n = 4

and

r = 3 .

However, with a pivot column in the last column we see that the original system has no solution by Theorem RCLS.

🔗

C24.

\begin{aligned} x_{1} - 2 x_{2} + x_{3} - x_{4} & = 2 \\ x_{1} + x_{2} + x_{3} - x_{4} & = 2 \\ x_{1} + x_{3} - x_{4} & = 2 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & - 2 & 1 & - 1 & 2 \\ 1 & 1 & 1 & - 1 & 2 \\ 1 & 0 & 1 & - 1 & 2 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 1 & - 1 & 2 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}] . \end{array}

Thus, we see that an equivalent system is

\begin{aligned} x_{1} & = 2 - x_{3} + x_{4} \\ x_{2} & = 0 \end{aligned}

and the solution set is

{[\begin{matrix} 2 - x_{3} + x_{4} \\ 0 \\ x_{3} \\ x_{4} \end{matrix}] | x_{3}, x_{4} \in C} .

For this system,

n = 4

and

r = 2 .

Since it is a consistent system by Theorem RCLS, Theorem CSRN guarantees an infinite number of solutions.

🔗

C25.

\begin{aligned} x_{1} + 2 x_{2} + 3 x_{3} & = 1 \\ 2 x_{1} - x_{2} + x_{3} & = 2 \\ 3 x_{1} + x_{2} + x_{3} & = 4 \\ x_{2} + 2 x_{3} & = 6 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & 2 & 3 & 1 \\ 2 & - 1 & 1 & 2 \\ 3 & 1 & 1 & 4 \\ 0 & 1 & 2 & 6 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array}] . \end{array}

Since

n = 3

and

r = 4 = n + 1,

the last column is a pivot column and Theorem RCLS guarantees that the system is inconsistent. Thus, we see that the given system has no solution.

🔗

C26.

\begin{aligned} x_{1} + 2 x_{2} + 3 x_{3} & = 1 \\ 2 x_{1} - x_{2} + x_{3} & = 2 \\ 3 x_{1} + x_{2} + x_{3} & = 4 \\ 5 x_{2} + 2 x_{3} & = 1 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & 2 & 3 & 1 \\ 2 & - 1 & 1 & 2 \\ 3 & 1 & 1 & 4 \\ 0 & 5 & 2 & 1 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 0 & 4 / 3 \\ 0 & 1 & 0 & 1 / 3 \\ 0 & 0 & 1 & - 1 / 3 \\ 0 & 0 & 0 & 0 \end{array}] . \end{array}

Since

r = n = 3

and the system is consistent by Theorem RCLS, Theorem CSRN guarantees a unique solution, which is

\begin{aligned} x_{1} & = 4 / 3 \\ x_{2} & = 1 / 3 \\ x_{3} & = - 1 / 3 . \end{aligned}

🔗

C27.

\begin{aligned} x_{1} + 2 x_{2} + 3 x_{3} & = 0 \\ 2 x_{1} - x_{2} + x_{3} & = 2 \\ x_{1} - 8 x_{2} - 7 x_{3} & = 1 \\ x_{2} + x_{3} & = 0 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & 2 & 3 & 0 \\ 2 & - 1 & 1 & 2 \\ 1 & - 8 & - 7 & 1 \\ 0 & 1 & 1 & 0 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{array}] . \end{array}

For this system, we have

n = 3

and

r = 3 .

However, with a pivot column in the last column we see that the original system has no solution by Theorem RCLS.

🔗

C28.

\begin{aligned} x_{1} + 2 x_{2} + 3 x_{3} & = 1 \\ 2 x_{1} - x_{2} + x_{3} & = 2 \\ x_{1} - 8 x_{2} - 7 x_{3} & = 1 \\ x_{2} + x_{3} & = 0 \end{aligned}

🔗

Solution.

The augmented matrix for the given linear system and its row-reduced form are:

\begin{array}{r} [\begin{array}{c} 1 & 2 & 3 & 1 \\ 2 & - 1 & 1 & 2 \\ 1 & - 8 & - 7 & 1 \\ 0 & 1 & 1 & 0 \end{array}] \overset{RREF}{\to} [\begin{array}{c} 1 & 0 & 1 & 1 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array}] . \end{array}

For this system,

n = 3

and

r = 2 .

Since it is a consistent system by Theorem RCLS, Theorem CSRN guarantees an infinite number of solutions. An equivalent system is

\begin{aligned} x_{1} & = 1 - x_{3} \\ x_{2} & = - x_{3}, \end{aligned}

where

x_{3}

is any scalar. So we can express the solution set as

\begin{matrix} {[\begin{matrix} 1 - x_{3} \\ - x_{3} \\ x_{3} \end{matrix}] | x_{3} \in C} . \end{matrix}

🔗

M45.

The details for Archetype J include several sample solutions. Verify that one of these solutions is correct (any one, but just one). Based only on this evidence, and especially without doing any row operations, explain how you know this system of linear equations has infinitely many solutions.

🔗

Solution.

Demonstrate that the system is consistent by verifying any one of the four sample solutions provided. Then because

n = 9 > 6 = m,

Theorem CMVEI gives us the conclusion that the system has infinitely many solutions. Notice that we only know the system will have at least

9 - 6 = 3

free variables, but very well could have more. We do not know know that

r = 6,

only that

r \leq 6 .

🔗

M46.

Consider Archetype J, and specifically the row-reduced version of the augmented matrix of the system of equations, denoted as

B

here, and the values of

r,

D

and

F

immediately following. Determine the values of the entries

🔗

\begin{aligned} {[B]}_{1, d_{1}} & {[B]}_{3, d_{3}} & {[B]}_{1, d_{3}} & {[B]}_{3, d_{1}} & {[B]}_{d_{1}, 1} & {[B]}_{d_{3}, 3} & {[B]}_{d_{1}, 3} & {[B]}_{d_{3}, 1} & {[B]}_{1, f_{1}} & {[B]}_{3, f_{1}} \end{aligned}

(See Exercise TSS.M70 for a generalization.)

🔗

Exercise Group.

For Exercises M51–M57 say as much as possible about each system’s solution set. Be sure to make it clear which theorems you are using to reach your conclusions.

🔗

M51.

A consistent system of 8 equations in 6 variables.

🔗

Solution.

Consistent means there is at least one solution (Definition CS). It will have either a unique solution or infinitely many solutions (Theorem PSSLS).

🔗

M52.

A consistent system of 6 equations in 8 variables.

🔗

Solution.

With 6 rows in the augmented matrix, the row-reduced version will have

r \leq 6 .

Since the system is consistent, apply Theorem CSRN to see that

n - r \geq 2

implies infinitely many solutions.

🔗

M53.

A system of 5 equations in 9 variables.

🔗

Solution.

The system could be inconsistent. If it is consistent, then because it has more variables than equations Theorem CMVEI implies that there would be infinitely many solutions. So, of all the possibilities in Theorem PSSLS, only the case of a unique solution can be ruled out.

🔗

M54.

A system with 12 equations in 35 variables.

🔗

Solution.

The system could be inconsistent. If it is consistent, then Theorem CMVEI tells us the solution set will be infinite. So we can be certain that there is not a unique solution.

🔗

M56.

A system with 6 equations in 12 variables.

🔗

Solution.

The system could be inconsistent. If it is consistent, and since

12 > 6,

then Theorem CMVEI says we will have infinitely many solutions. So there are two possibilities. Theorem PSSLS allows to state equivalently that a unique solution is an impossibility.

🔗

M57.

A system with 8 equations and 6 variables. The reduced row-echelon form of the augmented matrix of the system has 7 pivot columns.

🔗

Solution.

7 pivot columns implies that there are

r = 7

nonzero rows (so row 8 is all zeros in the reduced row-echelon form). The last column must be a pivot column and Theorem RCLS allows to conclude that the system is inconsistent.

🔗

M60.

Without doing any computations, and without examining any solutions, say as much as possible about the form of the solution set for each archetype that is a system of equations.

🔗

Archetype A, Archetype B, Archetype C, Archetype D, Archetype E, Archetype F, Archetype G, Archetype H, Archetype I, Archetype J

🔗

M70.

Suppose that

B

is a matrix in reduced row-echelon form that is equivalent to the augmented matrix of a system of equations with

m

equations in

n

variables. Let

r,

D

and

F

be as defined in Definition RREF. What can you conclude, in general, about the following entries?

🔗

\begin{aligned} {[B]}_{1, d_{1}} & {[B]}_{3, d_{3}} & {[B]}_{1, d_{3}} & {[B]}_{3, d_{1}} & {[B]}_{d_{1}, 1} & {[B]}_{d_{3}, 3} & {[B]}_{d_{1}, 3} & {[B]}_{d_{3}, 1} & {[B]}_{1, f_{1}} & {[B]}_{3, f_{1}} \end{aligned}

If you cannot conclude anything about an entry, then say so. (See Exercise TSS.M46.)

🔗

T10.

An inconsistent system may have

r > n .

If we try (incorrectly!) to apply Theorem FVCS to such a system, how many free variables would we discover?

🔗

Solution.

Theorem FVCS will indicate a negative number of free variables, but we can say even more. If

r > n,

then the only possibility is that

r = n + 1,

and then we compute

n - r = n - (n + 1) = - 1

free variables.

🔗

T11.

Suppose

A

is the augmented matrix of a system of linear equations in

n

variables. and that

B

is a row-equivalent matrix in reduced row-echelon form with

r

pivot columns. If

r = n + 1,

prove that the system of equations is inconsistent.

🔗

Solution.

r = n + 1,

then

D = {1, 2, 3, \dots, n, n + 1}

and every column of

B

is a pivot column. In particular, column

n + 1

is a pivot column. By Theorem RCLS the system is inconsistent.

🔗

T20.

Suppose that

B

is a matrix in reduced row-echelon form that is equivalent to the augmented matrix of a system of equations with

m

equations in

n

variables. Let

r,

D

and

F

be as defined in Definition RREF. Prove that

d_{k} \geq k

for all

1 \leq k \leq r .

Then suppose that

r \geq 2

and

1 \leq k < ℓ \leq r

and determine what can you conclude, in general, about the following entries.

🔗

\begin{aligned} {[B]}_{k, d_{k}} & {[B]}_{k, d_{ℓ}} & {[B]}_{ℓ, d_{k}} & {[B]}_{d_{k}, k} & {[B]}_{d_{k}, ℓ} & {[B]}_{d_{ℓ}, k} & {[B]}_{d_{k}, f_{ℓ}} & {[B]}_{d_{ℓ}, f_{k}} \end{aligned}

If you cannot conclude anything about an entry, then say so. (See Exercise TSS.M46 and Exercise TSS.M70.)

🔗

T40.

Suppose that the coefficient matrix of a consistent system of linear equations has two columns that are identical. Prove that the system has infinitely many solutions.

🔗

Solution.

Since the system is consistent, we know there is either a unique solution, or infinitely many solutions (Theorem PSSLS). If we perform row operations (Definition RO) on the augmented matrix of the system, the two equal columns of the coefficient matrix will suffer the same fate, and remain equal in the final reduced row-echelon form. Suppose both of these columns are pivot columns (Definition RREF). Then there is single row containing the two leading 1’s of the two pivot columns, a violation of reduced row-echelon form (Definition RREF). So at least one of these columns is not a pivot column, and the column index indicates a free variable in the description of the solution set (Definition IDV). With a free variable, we arrive at an infinite solution set (Theorem FVCS).

🔗

T41.

Consider the system of linear equations

LS (A, b),

and suppose that every element of the vector of constants

b

is a common multiple of the corresponding element of a certain column of

A .

More precisely, there is a complex number

α,

and a column index

j,

such that

{[b]}_{i} = α {[A]}_{i j}

for all

i .

Prove that the system is consistent.

🔗

Solution.

The condition about the multiple of the column of constants will allow you to show that the following values form a solution of the system

LS (A, b) .

\begin{aligned} x_{1} & = 0 & x_{2} & = 0 & \dots & x_{j - 1} & = 0 & x_{j} & = α & x_{j + 1} & = 0 & \dots & x_{n - 1} & = 0 & x_{n} & = 0 \end{aligned}

With one solution of the system known, we can say the system is consistent (Definition CS).

A more involved proof can be built using Theorem RCLS. Begin by proving that each of the three row operations (Definition RO) will convert the augmented matrix of the system into another matrix where column

j

α

times the entry of the same row in the last column. In other words, the “column multiple property” is preserved under row operations. These proofs will get successively more involved as you work through the three operations.

Now construct a proof by contradiction (Proof Technique CD), by supposing that the system is inconsistent. Then the last column of the reduced row-echelon form of the augmented matrix is a pivot column (Theorem RCLS). Then column

j

must have a zero in the same row as the leading 1 of the final column. But the “column multiple property” implies that there is an

α

in column

j

in the same row as the leading

1 .

α = 0 .

By hypothesis, then the vector of constants is the zero vector. However, if we began with a final column of zeros, row operations would never have created a leading 1 in the final column. This contradicts the final column being a pivot column, and therefore the system cannot be inconsistent.

🔗

You have attempted 1 of 4 activities on this page.

🔗

Prev Top Next

Section TSS Types of Solution Sets

Subsection CS Consistent Systems

Definition CS. Consistent System.

Example RREFN. Reduced row-echelon form notation.

Example ISSI. Describing infinite solution sets, Archetype I.

Definition IDV. Independent and Dependent Variables.

Example FDV. Free and dependent variables.

Sage FDV. Free and Dependent Variables.

Theorem RCLS. Recognizing Consistency of a Linear System.

Proof.

(⇐)

(⇒)

Sage RCLS. Recognizing Consistency of a Linear System.

Theorem CSRN. Consistent Systems, r and n.

Proof.

Subsection FV Free Variables

Theorem FVCS. Free Variables for Consistent Systems.

Proof.

Example CFV. Counting free variables.

Theorem PSSLS. Possible Solution Sets for Linear Systems.

Proof.

Theorem CMVEI. Consistent, More Variables than Equations, Infinite solutions.

Proof.

Example OSGMD. One solution gives many, Archetype D.

Sage SS1. Solving Systems, Part 1.

Reading Questions TSS Reading Questions

1. How to analyze consistency?

2. How to identify independent/dependent variables?

3. Possibilities for solution sets.

Exercises TSS Exercises

C10.

Exercise Group.

C21.

C22.

C23.

C24.

C25.

C26.

C27.

C28.

M45.

M46.

Exercise Group.

M51.

M52.

M53.

M54.

M56.

M57.

M60.

M70.

T10.

T11.

T20.

T40.

T41.

Theorem CSRN. Consistent Systems, $r$ and $n$ .