In this video, we'll learn about K-fold cross-validation and how it can be used for selecting optimal tuning parameters, choosing between models, and selecting features. We'll compare cross-validation with the train/test split procedure, and we'll also discuss some variations of cross-validation that can result in more accurate estimates of model performance.
Download the notebook: https://github.com/justmarkham/scikit-learn-videos
Documentation on cross-validation: http://scikit-learn.org/stable/modules/cross_validation.html
Documentation on model evaluation: http://scikit-learn.org/stable/modules/model_evaluation.html
GitHub issue on negative mean squared error: https://github.com/scikit-learn/scikit-learn/issues/2439
An Introduction to Statistical Learning: http://www-bcf.usc.edu/~gareth/ISL/
K-fold and leave-one-out cross-validation: https://www.youtube.com/watch?v=nZAM5OXrktY
Cross-validation the right and wrong ways: https://www.youtube.com/watch?v=S06JpVoNaA0
Accurately Measuring Model Prediction Error: http://scott.fortmann-roe.com/docs/MeasuringError.html
An Introduction to Feature Selection: http://machinelearningmastery.com/an-introduction-to-feature-selection/
Harvard CS109: https://github.com/cs109/content/blob/master/lec_10_cross_val.ipynb
Cross-validation pitfalls: http://www.jcheminf.com/content/pdf/1758-2946-6-10.pdf
WANT TO GET BETTER AT MACHINE LEARNING? HERE ARE YOUR NEXT STEPS:
1) WATCH my scikit-learn video series:
https://www.youtube.com/playlist?list=PL5-da3qGB5ICeMbQuqbbCOQWcS6OYBr5A
2) SUBSCRIBE for more videos:
https://www.youtube.com/dataschool?sub_confirmation=1
3) JOIN "Data School Insiders" to access bonus content:
https://www.patreon.com/dataschool
4) ENROLL in my Machine Learning course:
https://www.dataschool.io/learn/
5) LET'S CONNECT!
- Newsletter: https://www.dataschool.io/subscribe/
- Twitter: https://twitter.com/justmarkham
- Facebook: https://www.facebook.com/DataScienceSchool/
- LinkedIn: https://www.linkedin.com/in/justmarkham/