Out of Sample Performance

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure David R. Roberts,Volker Bahn,Simone Ciuti,Mark S. Boyce,Jane Elith,Gurutzeta Guillera-Arroita,Severin Hauenstein,José J. Lahoz-Monfort,Boris Schröder,Wilfried Thuiller,David I. Warton,Brendan A. Wintle,Florian Hartig,Carsten F. Dormann https://onlinelibrary.wiley.com/doi/full/10.1111/ecog.02881

Cross-validation FAQ, Aki Vehtari

When to Impute? Imputation before and during cross-validation

Leakage and the Reproducibility Crisis in ML-based Science

Rescaling and other forms of unsupervised preprocessing introduce bias into cross-validation,Amit Moscovich, Saharon Rosset

Approximate leave-future-out cross-validation for Bayesian time series models

How Cross-Validation Can Go Wrong and What to Do About it.

Moving cross-validation from a research idea to a routine step in Bayesian data analysis

Model selection tutorials and talks, Aki Vehtari Underspecification Presents Challenges for Credibility in Modern Machine Learning (Paper Explained) -Has a neet example of holding out on camera model shows massive degredation in medical mission vision application.

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Cross-validation: what does it estimate and how well does it do it? Stephen Bates, Trevor Hastie, Robert Tibshirani

Consensus features nested cross-validation

Your Cross Validation Error Confidence Intervals are Wrong — here’s how to Fix Them