n Step Transition Probability

Topics: Transition Probabilities in a Markov Chain - Markov Chain

---

(theorem)

In a Markov chain, the existence of one step stationary transition probabilities implies that for every $i,j$ and $n\in \mathbb{N}$:

$$\mathbb{P}(X_{t+n}=j\mid X_t=i)=\mathbb{P}(X_n=j\mid X_0=i)$$

We call these the $n$ step transition probabilities.

It can be handy to write these probabilities in a transition matrix.

Simpler Notation §

To simplify notation, we’ll agree on:

$$\begin{array}{rl}{p}_{ij}& =\mathbb{P}(X_{t+1}=j\mid X_t=i)\\ p_{ij}^{(n)}& =\mathbb{P}(X_{t+n}=j\mid X_t=i)\end{array}$$

Note that when $n=1$, then $p_{ij}^{(n)}=p_{ij}$.

Observe that, when $n=0$, we have $p_{ij}^{(n)}=\mathbb{P}(X_0=j\mid X_0=i)$, which is trivially $1$ when $i=j$ and $0$ otherwise.

Properties §

(theorem)

Derived from the properties of a probability measure, we have that:

• $p_{ij}^{(n)}\ge 0$, for every $i,j$ and $n=0,1,2,\dots$
• $\sum _{j=0}^s{p}_{ij}^{(n)}=1$, for every $i$ and $n=0,1,2,\dots$