Allow dataset builds to reuse downloaded raw source data

[florian6973]

Summary

Add a configurable raw_input_dir to meds-dev-dataset so users can reuse source data that has already been downloaded.

For example:

meds-dev-dataset \
  dataset=MIMIC-IV \
  raw_input_dir=/data/raw/mimiciv \
  output_dir=/data/meds/mimiciv

MIMIC-IV now uses the new {raw_input_dir} dataset-command placeholder.

Motivation

Dataset paths represent two distinct artifacts:

1. The native/raw source data consumed by an ETL.
2. The resulting MEDS cohort.

output_dir already controls where the generated MEDS cohort is stored, but dataset recipes currently hard-code raw downloads under {temp_dir}/raw. This makes it difficult to reuse large downloads across runs or separate MEDS-DEV output directories.

This is particularly relevant for credentialed datasets such as MIMIC-IV, where downloading the source data can be expensive.

Behavior

• If raw_input_dir is supplied, the dataset ETL receives that path.
• If it is omitted, MEDS-DEV preserves the existing behavior and uses {temp_dir}/raw.
• output_dir continues to identify the MEDS cohort being produced.
• A successful build continues to create output_dir/.done, allowing later invocations with the same output_dir to return without rerunning the ETL.
• do_overwrite=True continues to apply to output_dir; it does not remove a separate raw_input_dir.

The raw input and output paths should remain separate:

raw_input_dir --ETL--> output_dir
                         |
                         +-- used as dataset_dir by tasks and models

This change does not introduce a new registration mechanism for externally obtained MEDS cohorts. Those can continue to be passed directly as dataset_dir to task and model commands.

Changes

• Add raw_input_dir to the dataset-builder Hydra configuration.
• Expose {raw_input_dir} when formatting dataset build commands.
• Default it to {temp_dir}/raw for backward compatibility.
• Update the MIMIC-IV recipe.
• Document raw-source, MEDS-output, and downstream dataset paths.
• Add regression tests for both default and explicitly supplied paths.

Testing

• pytest --doctest-modules -m "not integration" -x: 55 passed.
• pre-commit run --all-files: all hooks passed.
• Comparable fast-suite coverage increased from 81.026% on dev to 81.098%; the number of missing statements remained 148.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Files with missing lines	Coverage Δ
src/MEDS_DEV/datasets/__main__.py	93.10% <100.00%> (+0.79%)	⬆️