A Lower Bound on the Relative Entropy with Respect to a Symmetric Probability

Raphaël Cerf, Matthias Gorny

Published 2014-07-03, updated 2014-10-20Version 2

Let $\rho$ and $\mu$ be two probability measures on $\mathbb{R}$ which are not the Dirac mass at $0$. We denote by $H(\mu|\rho)$ the relative entropy of $\mu$ with respect to $\rho$. We prove that, if $\rho$ is symmetric and $\mu$ has a finite first moment, then \[ H(\mu|\rho)\geq \frac{\displaystyle{(\int_{\mathbb{R}}z\,d\mu(z))^2}}{\displaystyle{2\int_{\mathbb{R}}z^2\,d\mu(z)}}\,,\] with equality if and only if $\mu=\rho$.