Maybe you have a highly accurate model, but it's not calibrated, which means that you cannot use the predict_proba values for decision making. If that's the case we have some good news because there is a remedy in scikit-learn!
Docs:
https://scikit-learn.org/dev/modules/generated/sklearn.calibration.CalibratedClassifierCV.html
Website: https://probabl.ai/
LinkedIn: https://www.linkedin.com/company/probabl
Twitter: https://x.com/probabl_ai
Bluesky: https://bsky.app/profile/probabl.bsky.social
Discord: https://discord.probabl.ai
We also host a podcast called Sample Space, which you can find on your favourite podcast player. All the links can be found here:
https://rss.com/podcasts/sample-space/
#probabl