Verifiable Conditions for Irreducibility, Aperiodicity and T-chain Property of a General Markov Chain

Alexandre Chotard, Anne Auger

Published 2015-08-07Version 1

We consider in this paper Markov chains on a state space being an open subset of $\mathbb{R}^n$ that obey the following general non linear state space model: $ \Phi_{t+1} = F \left(\Phi_t, \alpha(\Phi_t,\mathbf{U}_{t+1}) \right), t \in \mathbb{N}, $ where $(\mathbf{U}_t)_{t \in \mathbb{N}^*}$ (each $\mathbf{U}_t \in \mathbb{R}^p$) are i.i.d. random vectors, the function $\alpha$, taking values in $\mathbb{R}^m$, is a measurable typically discontinuous function and $(\mathbf{x},\mathbf{w}) \mapsto F(\mathbf{x},\mathbf{w})$ is a $C^1$ function. In the spirit of the results presented in the chapter~7 of the Meyn and Tweedie book on \emph{"Markov Chains and Stochastic Stability"}, we use the underlying deterministic control model to provide sufficient conditions that imply that the chain is a $\varphi$-irreducible, aperiodic T-chain with the support of the maximality irreducibility measure that has a non empty interior. Using previous results on our modelling would require that $\alpha(\mathbf{x},\mathbf{u})$ is independent of $\mathbf{x}$, that the function $(\mathbf{x},\mathbf{u}) \mapsto F(\mathbf{x},\alpha(\mathbf{x},\mathbf{u}) )$ is $C^\infty$ and that $U_1$ admits a lower semi-continuous density. In contrast, we assume that the function $(\mathbf{x},\mathbf{w}) \mapsto F(\mathbf{x},\mathbf{w})$ is $C^1$, and that for all $\mathbf{x}$, $\alpha(\mathbf{x},\mathbf{U}_1)$ admits a density $p_\mathbf{x}$ such that the function $(\mathbf{x},\mathbf{w}) \mapsto p_\mathbf{x}(\mathbf{w})$ is lower semi-continuous. Hence the overall update function $(\mathbf{x},\mathbf{u}) \mapsto F(\mathbf{x},\alpha(\mathbf{x},\mathbf{u}) )$ may have discontinuities, contained within the function $\alpha$. We present two applications of our results to Markov chains arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.