ODSL Forum: Brett Morris, A Practical Introduction to Leave One Out Cross Validation

Friday 18 Mar 2022, 14:00 → 15:00 Europe/Berlin

Presenter: Brett Morris, Center for Space and Habitability, University Bern
Title: A Practical Introduction to Leave One Out Cross Validation

Abstract:

Suppose you have two or more models which fit your data reasonably well. How do you choose which model performs best on unseen data? In this short, practical tutorial, I will introduce Leave-One-Out Cross-Validation (LOO-CV) as a technique for testing a model’s predictive accuracy, and briefly show stacking for model selection. I will motivate the talk around a series of example plots generated by a pedagogical Python notebook which you can experiment with at: https://gist.github.com/bmorris3/a69842ce9384966feba965eb0d726da6

Further Reading:

– Vehtari et al. (2015): https://arxiv.org/abs/1507.04544 

– Aki Vehtari’s "Cross-Validation FAQ": https://avehtari.github.io/modelselection/CV-FAQ.html

As usual, the format will be a short presentation followed by plenty of discussion.

Please let all interested know about the Journal Club (get them to send their Email address).  We are looking forward to lively discussions.