Statistical Mechanics of Learning in the Presence of Outliers

Rainer Dietrich, Manfred Opper

Published 1998-03-26, updated 1999-02-25Version 2

Using methods of statistical mechanics, we analyse the effect of outliers on the supervised learning of a classification problem. The learning strategy aims at selecting informative examples and discarding outliers. We compare two algorithms which perform the selection either in a soft or a hard way. When the fraction of outliers grows large, the estimation errors undergo a first order phase transition.