Repository navigation
Commit f33710c
Add 3D segmentation calibration tutorial (#2072)
## Description
Adds an introductory tutorial for the average calibration losses
proposed in Project-MONAI/MONAI#9092.
The notebook explains predictive calibration, why it matters for medical
image segmentation, and how to interpret reliability diagrams. It then
compares a segmentation-only baseline with one validation-selected Hard
L1-ACE configuration on complete 3D volumes from Medical Segmentation
Decathlon Task04 Hippocampus.
The example intentionally keeps the experiment simple: two training runs
with identical initialization, data order, augmentations, and
optimization settings. The calibrated objective uses a `1:1:0.30`
Dice:CE:Hard-L1-ACE ratio. The held-out test set is not accessed until
both validation-selected checkpoints have been fixed.
## Saved tutorial result
| Configuration | Dice | ACE | MCE |
| --- | ---: | ---: | ---: |
| Segmentation only | 0.799317 | 0.117391 | 0.295517 |
| Hard L1-ACE | 0.828645 | 0.090355 | 0.208534 |
This corresponds to a 23% reduction in held-out ACE and a 29% reduction
in MCE, while Dice increases by 0.0293 for this split and seed. The
notebook explicitly presents this as a demonstration rather than a
benchmark guarantee.
## Validation
- Executed all 36 cells end to end on an NVIDIA RTX 5090
- Passed the repository runner with a one-epoch execution
- Passed targeted notebook schema, syntax, formatting, PEP8, metadata,
and output checks
- Confirmed that saved outputs contain no execution errors or private
local paths
## Dependency
Depends on Project-MONAI/MONAI#9092 for `HardL1ACELoss` and
`SoftL1ACELoss`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added a Calibration tutorial using complete 3D MRI volumes from the
MSD Task04 Hippocampus dataset.
- Documented calibration metrics and losses, and the comparison of a
baseline with a validation-selected Dice:CE:Hard-L1-ACE configuration on
separate held-out test volumes.
- Added guidance for running the experiment, including a one-epoch CI
smoke test, dataset requirements, recommended GPU hardware, and required
MONAI APIs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Theo Barfoot <theo.barfoot@gmail.com>
Signed-off-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>1 parent 312ae90 commit f33710c
3 files changed
Lines changed: 1415 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
104 | 111 | | |
105 | 112 | | |
106 | 113 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
Large diffs are not rendered by default.
0 commit comments