| Title | Resampling Algorithms for mlr3 Framework |
| Version | 2026.4.26 |
| Description | A supervised learning algorithm inputs a train set, and outputs a prediction function, which can be used on a test set. If each data point belongs to a subset (such as geographic region, year, etc), then how do we know if subsets are similar enough so that we can get accurate predictions on one subset, after training on Other subsets? And how do we know if training on All subsets would improve prediction accuracy, relative to training on the Same subset? SOAK, Same/Other/All K-fold cross-validation, doi:10.1002/sam.70055 can be used to answer these questions, by fixing a test subset, training models on Same/Other/All subsets, and then comparing test error rates (Same versus Other and Same versus All). Also provides code for estimating how many train samples are required to get accurate predictions on a test set. |
| Imports | data.table, R6, checkmate, paradox, mlr3 (>= 1.0.0), mlr3misc, methods, Rcpp |
| Suggests | pbdMPI, ggplot2, animint2, mlr3tuning, lgr, future, future.apply, testthat, WeightedROC, nc, rpart, directlabels, mlr3pipelines, glmnet, mlr3learners, mlr3torch, torch, batchtools, mlr3batchmark, litedown |
| License | LGPL-3 |
| URL | https://github.com/tdhock/mlr3resampling |
| BugReports | https://github.com/tdhock/mlr3resampling/issues |
| VignetteBuilder | litedown |
| Author | Toby Hocking  [aut, cre], Daniel Agyapong [ctb], Michel Lang [ctb], Bernd Bischl [ctb], Jakob Richter [ctb], Patrick Schratz [ctb], Giuseppe Casalicchio [ctb], Stefan Coors [ctb], Quay Au [ctb], Martin Binder [ctb], Florian Pfisterer [ctb], Raphael Sonabend [ctb], Lennart Schneider [ctb], Marc Becker [ctb], Sebastian Fischer [ctb] |
Appendix
To cite the package mlr3resampling in publications, please use:
Hocking T, Thibault G, Bodine C, Arellano P, Shenkin A, Lindly O (2026). “SOAK: Same/Other/All K-fold cross-validation for estimating similarity of patterns in data subsets.” Statistical Analysis and Data Mining, 19(1). doi:10.1002/sam.70055 https://doi.org/10.1002/sam.70055.
@Article{,
title = {SOAK: Same/Other/All K-fold cross-validation for estimating similarity of patterns in data subsets},
author = {Toby Dylan Hocking and Gabrielle Thibault and Cameron Scott Bodine and Paul Nelsom Arellano and Alexander F Shenkin and Olivia Lindly},
year = {2026},
journal = {Statistical Analysis and Data Mining},
doi = {10.1002/sam.70055},
volume = {19},
number = {1},
}