Section 8.4 Some Common Recurrence Relations
In this section we intend to examine a variety of recurrence relations that are not finite-order linear with constant coefficients. For each part of this section, we will consider a concrete example, present a solution, and, if possible, examine a more general form of the original relation.
Subsection 8.4.1 A First Basic Example
Consider the homogeneous first-order linear relation without constant coefficients, with initial condition Upon close examination of this relation, we see that the th term is times the term, which is a property of factorial. is a solution of this relation, for if
In addition, since the initial condition is satisfied. It should be pointed out that from a computational point of view, our “solution” really isn’t much of an improvement since the exact calculation of takes multiplications.
The exponent of 2 in is growing according to the relation with Thus and Note that could also be written as for but this is not a closed form expression.
This product form of is not a closed form expression because as grows, the number of multiplications grow. Thus, it is really not a true solution. Often, as for above, a closed form expression can be derived from the product form.
Subsection 8.4.2 An Analysis of the Binary Search Algorithm
Subsubsection 8.4.2.1
Suppose you intend to use a binary search algorithm (see Subsection 8.1.3) on lists of zero or more sorted items, and that the items are stored in an array, so that you have easy access to each item. A natural question to ask is “How much time will it take to complete the search?” When a question like this is asked, the time we refer to is often the so-called worst-case time. That is, if we were to search through items, what is the longest amount of time that we will need to complete the search? In order to make an analysis such as this independent of the computer to be used, time is measured by counting the number of steps that are executed. Each step (or sequence of steps) is assigned an absolute time, or weight; therefore, our answer will not be in seconds, but in absolute time units. If the steps in two different algorithms are assigned weights that are consistent, then analyses of the algorithms can be used to compare their relative efficiencies. There are two major steps that must be executed in a call of the binary search algorithm:
- If the lower index is less than or equal to the upper index, then the middle of the list is located and its key is compared to the value that you are searching for.
- In the worst case, the algorithm must be executed with a list that is roughly half as large as in the previous execution. If we assume that Step 1 takes one time unit and
is the worst-case time for a list of items, thenFor simplicity, we will assume thateven though the conditions of Step 1 must be evaluated as false if You might wonder why is truncated in (8.4.1). If is odd, then for some the middle of the list will be the item, and no matter what half of the list the search is directed to, the reduced list will have items. On the other hand, if is even, then for The middle of the list will be the item, and the worst case will occur if we are directed to the items that come after the middle (the through items). Again the reduced list has items.
Solution to (8.4.1) and (8.4.2). To determine the easiest case is when is a power of two. If we compute , by iteration, our results are
The pattern that is established makes it clear that This result would seem to indicate that every time you double the size of your list, the search time increases by only one unit.
A more complete solution can be obtained if we represent in binary form. For each there exists a non-negative integer such that
For example, if therefore, If satisfies (8.4.3), its binary representation requires digits. For example, =
From the pattern that we’ve just established, reduces to A formal inductive proof of this statement is possible. However, we expect that most readers would be satisfied with the argument above. Any skeptics are invited to provide the inductive proof.
Subsubsection 8.4.2.2 Review of Logarithms
Any discussion of logarithms must start by establishing a base, which can be any positive number other than 1. With the exception of Theorem 5, our base will be 2. We will see that the use of a different base (10 and are the other common ones) only has the effect of multiplying each logarithm by a constant. Therefore, the base that you use really isn’t very important. Our choice of base 2 logarithms is convenient for the problems that we are considering.

Plot of the logarithm, bases 2, function
For example, because and because A graph of the function in Figure 8.4.2 shows that if the that is, when increases, also increases. However, if we move from to only increases from 10 to 11. This slow rate of increase of the logarithm function is an important point to remember. An algorithm acting on pieces of data that can be executed in time units can handle significantly larger sets of data than an algorithm that can be executed in or time units. The graph of would show the same behavior.
A few more properties that we will use in subsequent discussions involving logarithms are summarized in the following theorem.
Theorem 8.4.3. Fundamental Properties of Logarithms.
Definition 8.4.4. Logarithms base .
Theorem 8.4.5. How logarithms with different bases are related.
Note 8.4.6.
Subsubsection 8.4.2.3
Returning to the binary search algorithm, we can derive the final expression for using the properties of logarithms, including that the logarithm function is increasing so that inequalities are maintained when taking logarithms of numbers.
If the time that was assigned to Step 1 of the binary search algorithm is changed, we wouldn’t expect the form of the solution to be very different. If with then
A further generalization would be to add a coefficient to with where and is not quite as simple to derive. First, if we consider values of that are powers of 2:
where
The first term of this expression is a geometric sum, which can be written in closed form. Let be that sum:
We’ve multiplied each term of by and aligned the identical terms in and Now if we subtract the two equations,
Therefore,
Subsection 8.4.3 Analysis of Bubble Sort and Merge Sort
The efficiency of any search algorithm such as the binary search relies on fact that the search list is sorted according to a key value and that the search is based on the key value. There are several methods for sorting a list. One example is the bubble sort. You might be familiar with this one since it is a popular “first sorting algorithm.” A time analysis of the algorithm shows that if is the worst-case time needed to complete the bubble sort on items, then and The solution of this relation is a quadratic function The growth rate of a quadratic function such as this one is controlled by its squared term. Any other terms are dwarfed by it as gets large. For the bubble sort, this means that if we double the size of the list that we are to sort, changes to and so becomes . Therefore, the time needed to do a bubble sort is quadrupled. One alternative to bubble sort is the merge sort. Here is a simple version of this algorithm for sorting If the list is sorted trivially. If then:
- Divide
into and - Sort
and using a merge sort. - Merge the sorted lists
and into one sorted list. If the sort is to be done in descending order of key values, you continue to choose the higher key value from the fronts of and and place them in the back of
Note that will always have items and will have items; thus, if is odd, gets one more item than We will assume that the time required to perform Step 1 of the algorithm is insignificant compared to the other steps; therefore, we will assign a time value of zero to this step. Step 3 requires roughly comparisons and movements of items from and to thus, its time is proportional to For this reason, we will assume that Step 3 takes time units. Since Step 2 requires time units,
with the initial condition
Instead of an exact solution of these equations, we will be content with an estimate for First, consider the case of
Thus, if is a power of 2, Now if, for some then This can be proved by induction on As increases from to increases from to and is slightly larger than The discrepancy is small enough so that can be considered a solution of (8.4.9) and (8.4.10) for the purposes of comparing the merge sort with other algorithms. Table 8.4.7 compares with for selected values of
n | ||
10 | 45 | 34 |
50 | 1225 | 283 |
100 | 4950 | 665 |
500 | 124750 | 4483 |
1000 | 499500 | 9966 |
Subsection 8.4.4 Derangements
A derangement is a permutation on a set that has no “fixed points”. Here is a formal definition:
Definition 8.4.8. Derangement.
A derangement of a nonempty set is a permutation of (i.e., a bijection from into ) such that for all
If an interesting question might be “How many derangements are there of ” We know that our answer is bounded above by We can also expect our answer to be quite a bit smaller than since is the image of itself for of the permutations of
Let be the number of derangements of Our answer will come from discovering a recurrence relation on Suppose that If we are to construct a derangement of then Thus, the image of can be selected in different ways. No matter which of the choices we make, we can complete the definition of in one of two ways. First, we can decide to make leaving ways of completing the definition of since will be a derangement of Second, if we decide to select each of the derangements of can be used to define If is a derangement of such that then define f by
Note that with our second construction of while in the first construction, Therefore, no derangement of with can be constructed by both methods.
To recap our result, we see that is determined by first choosing one of images of and then constructing the remainder of in one of ways. Therefore,
This homogeneous second-order linear relation with variable coefficients, together with the initial conditions and completely defines Instead of deriving a solution of this relation by analytical methods, we will give an empirical derivation of an approximation of Since the derangements of are drawn from a pool of permutations, we will see what percentage of these permutations are derangements by listing the values of and The results we observe will indicate that as grows, hardly changes at all. If this quotient is computed to eight decimal places, for The reciprocal of this number, which seems to be tending toward, is, to eight places, 2.7182818. This number appears in so many places in mathematics that it has its own name, An approximate solution of our recurrence relation on is then
xxxxxxxxxx
def D(n):
if n<=2:
return n-1
else:
return (n-1)*(D(n-2)+D(n-1))
list(map(lambda k:[k,D(k),(D(k)/factorial(k)).n(digits=8)],range(1,16)))
Exercises 8.4.5 Exercises
1.
Solve the following recurrence relations. Indicate whether your solution is an improvement over iteration.
Answer.
an improvement. no improvement.
2.
3.
Solve as completely as possible:
(Hint: Write in octal form.)
Answer.
4.
Hint.
Prove by induction on
5.
Answer.
The indicated substitution yields Since for all Therefore
6.
Consider the sequence defined by for with the conditions and for all Use the substitution to solve for
7.
Solve as completely as possible:
Answer.
- A good approximation to the solution of this recurrence relation is based on the following observation:
is a power of a power of two; that is, is where , then By applying this recurrence relation times we obtain Going back to the original form of or We would expect that in general, is We do not see any elementary method for arriving at an exact solution. - Suppose that
is a positive integer with Then can be written in binary form, with and is equal to the sum If then we can estimate this sum to be between and Therefore,
8.
Suppose Step 1 of the merge sort algorithm did take a significant amount of time. Assume it takes 0.1 time unit, independent of the value of
- Write out a new recurrence relation for
that takes this factor into account. - Solve for
- Assuming the solution for powers of 2 is a good estimate for all
compare your result to the solution in the text. As gets large, is there really much difference?
You have attempted of activities on this page.