A special type of linear dynamical system occurs when the matrix is stochastic. A stochastic matrix is one where each entry of the matrix is between and and all of the columns of the matrix sum to
The reason for these conditions is that the entries of a stochastic matrix represent probabilities; in particular, they are transition probabilities. That is, each number represents the probability of one state changing to another.
If a system can be in one of possible states, we represent the system by an vector whose entries indicate the probability that the system is in a given state at time If we know that the system starts out in a particular state, then will have a in one of its entries, and everywhere else.
A Markov chain is given by such an initial vector, and a stochastic matrix. As an example, we will consider the following scenario, described in the book Shape, by Jordan Ellenberg:
A mosquito is born in a swamp, which we will call Swamp A. There is another nearby swamp, called Swamp B. Observational data suggests that when a mosquito is at Swamp A, there is a 40% chance that it will remain there, and a 60% chance that it will move to Swamp B. When the mosquito is at Swamp B, there is a 70% chance that it will remain, and a 30% chance that it will return to Swamp A.
(c)
You should have found that one of the eigenvalues of was The corresponding eigenvector satisfies This is known as a steady-state vector: if our system begins with state it will remain there forever.
Confirm that if the eigenvector is rescaled so that its entries sum to 1, the resulting values agree with the long-term probabilities found in the previous part.