arXiv Analytics

Sign in

arXiv:1005.2364 [cs.LG]AbstractReferencesReviewsResources

A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)

Volker Nannen

Published 2010-05-13, updated 2010-05-14Version 2

The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection.

Comments: 20 pages, Chapter 1 of The Paradox of Overfitting, Master's thesis, Rijksuniversiteit Groningen, 2003
Categories: cs.LG, cs.CC
Subjects: F.2.3
Related articles: Most relevant | Search more
arXiv:2111.04688 [cs.LG] (Published 2021-11-08, updated 2022-06-30)
Universal and data-adaptive algorithms for model selection in linear contextual bandits
arXiv:1607.07959 [cs.LG] (Published 2016-07-27)
Using Kernel Methods and Model Selection for Prediction of Preterm Birth
Ilia Vovsha et al.
arXiv:2409.09674 [cs.LG] (Published 2024-09-15)
Model Selection Through Model Sorting

Links

Toolbox