On the average of probability distributions of a discrete Markov chain

Nikolaos Halidias

Published 2017-07-27Version 1

Let $X_n$ be a discrete time Markov chain with state space $S$ and initial probability distribution $\mu^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k \in \mathbb{N}$ with $k \leq n$ such that $X_k = j$ where $j \in S$? This probability is the average $\frac{1}{n} \sum_{k=1}^n \mu^{(k)}_j$ where $\mu^{(k)}_j = P(X_k = j)$. In this note we will study the limit of this average without assuming that the chain is irreducible. Finally, we study the limit of the average $\frac{1}{n} \sum_{k=1}^n g(X_k)$ where $g$ is a given function for a general Markov chain not necessarily irreducible.