To prove that not all states in a finite Markov chain can be transient, we need to carefully define and use some key concepts from Markov chain theory. Here’s the step-by-step argument:

Definitions

• Transient State: A state $i$ in a Markov chain is said to be transient if, starting from state $i$, there is a non-zero probability of never returning to it.
• Recurrent State: A state $i$ is recurrent if, starting from state $i$, the probability of returning to it is 1.
• Absorbing State: A state $i$ is absorbing if, once it is entered, there is no possibility of leaving it (i.e., the probability of staying in that state forever is 1).

Theoretical Framework

Consider a finite Markov chain with a state space $S = \{1, 2, \dots, N\}$, where $N$ is finite. Assume for contradiction that all states are transient. Our goal is to show that this assumption leads to a contradiction.

Step-by-Step Proof

1. Assumption: All states are transient. By definition, for every state $i \in S$, there exists a probability that once the chain enters state $i$, it may never return to that state.

2. Transition probabilities decrease over time: Since all states are transient, there is a non-zero probability that the chain never returns to any given state. This implies that over time, the probability of visiting each state decreases. Formally, for each state $i$, the probability of returning to $i$ after visiting it is less than 1.

3. Expected number of visits is finite: A key property of transient states is that the expected number of visits to each transient state is finite. Specifically, if state $i$ is transient, the expected number of times the chain visits state $i$ starting from any state is finite. In other words, if $N_i$ is the number of times state $i$ is visited, then $\mathbb{E}[N_i] < \infty$.

4. Total number of visits is finite: Since there are finitely many states and each state is transient, the expected number of visits to any state is finite. This means that the chain will, with probability 1, eventually stop visiting any of the states in the Markov chain, i.e., the chain will "escape" the finite set of states.

5. Contradiction: However, this conclusion leads to a contradiction. A finite Markov chain cannot escape the state space because it has only a finite number of states. Eventually, the chain must continue transitioning between states. Since the total probability across all transitions must sum to 1 at each step, the chain cannot "disappear." Therefore, it cannot be that every state is transient.

6. Conclusion: Since the assumption that all states are transient leads to a contradiction, it follows that not all states in a finite Markov chain can be transient. At least one state must be recurrent.

Thus, in any finite Markov chain, there must be at least one recurrent state.

---

Do you have any questions or want further details on any part of this proof? Here are some related questions for further exploration:

1. What is the difference between transient and recurrent states in a Markov chain?
2. How does the classification of states impact the long-term behavior of a Markov chain?
3. Can an infinite Markov chain have all transient states?
4. How does the notion of an absorbing state relate to recurrent states?
5. What are the implications of recurrent states on the stationary distribution of a Markov chain?

Tip: In Markov chains, recurrent states are crucial for determining the long-term behavior and equilibrium properties of the system. Understanding how these states work is key for analyzing steady-state probabilities.