Out of Sample Performance
Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure David R. Roberts,Volker Bahn,Simone Ciuti,Mark S. Boyce,Jane Elith,Gurutzeta Guillera-Arroita,Severin Hauenstein,José J. Lahoz-Monfort,Boris Schröder,Wilfried Thuiller,David I. Warton,Brendan A. Wintle,Florian Hartig,Carsten F. Dormann https://onlinelibrary.wiley.com/doi/full/10.1111/ecog.02881
Cross-validation FAQ, Aki Vehtari
When to Impute? Imputation before and during cross-validation
Leakage and the Reproducibility Crisis in ML-based Science
Approximate leave-future-out cross-validation for Bayesian time series models
How Cross-Validation Can Go Wrong and What to Do About it.
Moving cross-validation from a research idea to a routine step in Bayesian data analysis
Model selection tutorials and talks, Aki Vehtari Underspecification Presents Challenges for Credibility in Modern Machine Learning (Paper Explained) -Has a neet example of holding out on camera model shows massive degredation in medical mission vision application.
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Consensus features nested cross-validation
Your Cross Validation Error Confidence Intervals are Wrong — here’s how to Fix Them