{
  "id": "1508.01644",
  "version": "v1",
  "published": "2015-08-07T09:52:27.000Z",
  "updated": "2015-08-07T09:52:27.000Z",
  "title": "Verifiable Conditions for Irreducibility, Aperiodicity and T-chain Property of a General Markov Chain",
  "authors": [
    "Alexandre Chotard",
    "Anne Auger"
  ],
  "comment": "30 pages",
  "categories": [
    "math.PR"
  ],
  "abstract": "We consider in this paper Markov chains on a state space being an open subset of $\\mathbb{R}^n$ that obey the following general non linear state space model: $ \\Phi_{t+1} = F \\left(\\Phi_t, \\alpha(\\Phi_t,\\mathbf{U}_{t+1}) \\right), t \\in \\mathbb{N}, $ where $(\\mathbf{U}_t)_{t \\in \\mathbb{N}^*}$ (each $\\mathbf{U}_t \\in \\mathbb{R}^p$) are i.i.d. random vectors, the function $\\alpha$, taking values in $\\mathbb{R}^m$, is a measurable typically discontinuous function and $(\\mathbf{x},\\mathbf{w}) \\mapsto F(\\mathbf{x},\\mathbf{w})$ is a $C^1$ function. In the spirit of the results presented in the chapter~7 of the Meyn and Tweedie book on \\emph{\"Markov Chains and Stochastic Stability\"}, we use the underlying deterministic control model to provide sufficient conditions that imply that the chain is a $\\varphi$-irreducible, aperiodic T-chain with the support of the maximality irreducibility measure that has a non empty interior. Using previous results on our modelling would require that $\\alpha(\\mathbf{x},\\mathbf{u})$ is independent of $\\mathbf{x}$, that the function $(\\mathbf{x},\\mathbf{u}) \\mapsto F(\\mathbf{x},\\alpha(\\mathbf{x},\\mathbf{u}) )$ is $C^\\infty$ and that $U_1$ admits a lower semi-continuous density. In contrast, we assume that the function $(\\mathbf{x},\\mathbf{w}) \\mapsto F(\\mathbf{x},\\mathbf{w})$ is $C^1$, and that for all $\\mathbf{x}$, $\\alpha(\\mathbf{x},\\mathbf{U}_1)$ admits a density $p_\\mathbf{x}$ such that the function $(\\mathbf{x},\\mathbf{w}) \\mapsto p_\\mathbf{x}(\\mathbf{w})$ is lower semi-continuous. Hence the overall update function $(\\mathbf{x},\\mathbf{u}) \\mapsto F(\\mathbf{x},\\alpha(\\mathbf{x},\\mathbf{u}) )$ may have discontinuities, contained within the function $\\alpha$. We present two applications of our results to Markov chains arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.",
  "revisions": [
    {
      "version": "v1",
      "updated": "2015-08-07T09:52:27.000Z"
    }
  ],
  "analyses": {
    "subjects": [
      "60J05"
    ],
    "keywords": [
      "general markov chain",
      "t-chain property",
      "verifiable conditions",
      "non linear state space model",
      "general non linear state space"
    ],
    "note": {
      "typesetting": "TeX",
      "pages": 30,
      "language": "en",
      "license": "arXiv",
      "status": "editable",
      "adsabs": "2015arXiv150801644C"
    }
  }
}