Conformal prediction fails on imbalanced drug data—class-conditional fix restores minority coverage
Conformal prediction promises honest error bounds for drug discovery models: pick a tolerance α, get prediction sets that cover the true label 90% of the time. On imbalanced datasets, this breaks.
Minority-class coverage collapsed across four real datasets—from 64.8% on blood-brain-barrier penetration to 4.2% on clinical-trial toxicity, where rare compounds went almost completely uncovered. The failure held across random forests, graph networks, and frozen language models (p < 0.001 in all cases).
A class-conditional variant recovers the guarantee. Simple fix, big practical stakes: standard conformal prediction can quietly under-protect the rarest, often most important compounds.