Skip to content

Add the INSPIRE dataset - #308

Merged
florian6973 merged 2 commits into
devfrom
feat/add-dataset-inspire
Jul 30, 2026
Merged

Add the INSPIRE dataset#308
florian6973 merged 2 commits into
devfrom
feat/add-dataset-inspire

Conversation

@mmcdermott

@mmcdermott mmcdermott commented May 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Registers INSPIRE (perioperative medicine) in src/MEDS_DEV/datasets/INSPIRE/. Extraction is via MEDS_extract-INSPIRE from the upstream inspire-meds package.

Files added:

  • dataset.yaml — metadata + build_full shelling out to MEDS_extract-INSPIRE. No build_demo (upstream doesn't ship a demo recipe; Make build_demo optional, skip demo-less datasets in tests #312 makes the key optional and ensures integration tests skip it).
  • predicates.yaml — admission/discharge plus a few lab/vital predicates. Regex syntax bugs (^X* form) and commented-out predicate stubs from the contributor's branch were cleaned up.
  • requirements.txtINSPIRE-MEDS==0.0.11.
  • refs.bib.
  • README.md.

INSPIRE is not added to any task's supported_datasets.

Depends on #312

#312 makes build_demo optional in the registry and skips datasets that don't declare it from the integration test matrix. Targeted at feat/dataset-demo-availability for now; once #312 merges, this PR retargets to dev.

Test plan

Supersedes / refs

🤖 Generated with Claude Code

@codecov

codecov Bot commented May 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Files with missing lines Coverage Δ
src/MEDS_DEV/datasets/__init__.py 97.43% <100.00%> (+0.06%) ⬆️

@mmcdermott
mmcdermott changed the base branch from dev to feat/dataset-demo-availability May 13, 2026 16:33
@mmcdermott
mmcdermott force-pushed the feat/add-dataset-inspire branch from 8048c83 to 48c84ef Compare May 13, 2026 16:33
mmcdermott added a commit that referenced this pull request May 13, 2026
Prerequisite for the per-dataset registration PRs (#305 AUMCdb, #306
EHRShot, #307 HIRID, #308 INSPIRE, #309 NWICU, #310 SICdb, #311 eICU).
Most of those datasets' upstream extractors don't ship a publicly
installable demo, and the existing registry validation requires every
dataset to declare a build_demo command.

Switches the convention to: a dataset has a demo iff its commands
declare build_demo. Absence is the signal — no separate metadata field.

- `test_all_datasets_have_commands` now requires `build_full` (which
  every dataset still needs) and allows missing `build_demo`.
- `tests/conftest.py` drops datasets without `build_demo` from the
  integration test matrix, so a per-dataset CI lane for one collects
  zero parametrized tests and passes cleanly rather than trying to
  build data the dataset can't produce.
- `src/MEDS_DEV/datasets/__main__.py` raises a clear error when called
  with `demo=True` against a dataset that doesn't declare a
  build_demo command (instead of the previous KeyError).

No dataset.yaml files change here — those changes ship with the sister
per-dataset PRs that depend on this one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mmcdermott
mmcdermott force-pushed the feat/dataset-demo-availability branch from 17019a0 to a2d4038 Compare May 13, 2026 17:20
@mmcdermott
mmcdermott force-pushed the feat/add-dataset-inspire branch from 48c84ef to d79e827 Compare May 13, 2026 17:22
@mmcdermott
mmcdermott changed the base branch from feat/dataset-demo-availability to dev May 13, 2026 17:43
@mmcdermott
mmcdermott force-pushed the feat/add-dataset-inspire branch from d79e827 to 5cd613c Compare May 13, 2026 17:43
@florian6973

Copy link
Copy Markdown
Collaborator

I checked the INSPIRE-MEDS==0.0.11 release, and it appears to be broken because of an incompatible dependency update.

The package declares meds-transforms~=0.2, but INSPIRE_MEDS/pre_MEDS.py still imports and calls hydra_loguru_init from MEDS_transforms.utils. This function is no longer available in meds-transforms 0.2, so importing the ETL module raises an ImportError.

This seems to have been fixed in INSPIRE-MEDS==0.0.12 and above, where the hydra_loguru_init import and call were removed and the dependency was also updated.

@florian6973

Copy link
Copy Markdown
Collaborator

I think I understand why CI did not catch this. Although the dataset-lane (INSPIRE) check is green, INSPIRE has no build_demo command, so it is filtered out of the integration-test parametrization. The resulting empty parametrization is treated as skipped by pytest, and the full-integration job was also skipped.

As a result, INSPIRE-MEDS==0.0.11 was never installed or imported during CI, and MEDS_extract-INSPIRE was not executed. It may be useful to add a lightweight smoke test that installs each dataset’s requirements and verifies that its configured entry-point command can at least be imported or invoked with --help.

@florian6973
florian6973 merged commit 69ba7b5 into dev Jul 30, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants