📐 Evaluation & Reliability
You can produce an uncertainty number for any input. This part asks the harder question — is that number telling you the truth?
Part 2 — Core UQ Methods
You worked through the main families of uncertainty quantification and implemented each one: the Bayesian perspective on neural networks, variational inference, MC Dropout, Deep Ensembles, Evidential Deep Learning, and Conformal Prediction. You saw that Bayesian methods estimate uncertainty while Conformal Prediction guarantees coverage, and that ensembles remain a remarkably strong baseline despite making no Bayesian claims at all.
About this part
A method that outputs an uncertainty score is not automatically useful. The score has to mean something — when a model says it is 80% confident, it should be right about 80% of the time. That property is calibration, and it has to be measured rather than assumed.
Part 3 covers the evaluation toolkit: calibration metrics and reliability diagrams, risk–coverage analysis for deciding when a model should abstain and hand a case to a clinician, and out-of-distribution detection for recognizing inputs that lie outside the training distribution entirely. These are the checks that stand between a promising method and a deployable system.