Skip to content

Add fetch scaffold subdirs, test coverage, and CI workflow - #1

Merged
gwct merged 1 commit into
mainfrom
fetch-scaffolds
Jul 21, 2026
Merged

Add fetch scaffold subdirs, test coverage, and CI workflow#1
gwct merged 1 commit into
mainfrom
fetch-scaffolds

Conversation

@gwct

@gwct gwct commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

This pull request introduces a new --scaffold-subdirs option to the mafutils fetch command, allowing output files to be grouped into subdirectories named by reference scaffold instead of a flat output directory. The implementation ensures robust directory creation, error handling, and integration with both block and scaffold fetch modes. Documentation is updated to reflect the new option and recent test coverage improvements.

New Feature: Scaffold Subdirectory Output

  • Added --scaffold-subdirs option to mafutils fetch, grouping output files into <output>/<scaffold>/<basename> directories for improved organization. This is supported in both block and scaffold modes, with safeguards to avoid use with --single-output. [1] [2] [3]

Implementation Details

  • Introduced a shared buildOutputPath helper to generate output paths based on the scaffold_subdirs flag, and updated all relevant code paths to use it. [1] [2] [3] [4]
  • Ensured all necessary scaffold subdirectories are created up front in the main process to prevent race conditions in parallel worker scenarios. [1] [2]

Command-line and API Integration

  • Extended function signatures and argument passing throughout the fetch pipeline (including worker initialization and parallel execution) to propagate the new scaffold_subdirs option. [1] [2] [3] [4] [5] [6] [7]

Documentation Updates

  • Updated README.md to document the new --scaffold-subdirs option.
  • Enhanced DEVELOPMENT.md with details on new and existing test coverage, including tests for real-world MAF data and regression checks.

Continuous Integration

  • Added a new GitHub Actions workflow (.github/workflows/tests.yml) to automatically run the test suite on pushes and pull requests to main.… test workflow

@gwct
gwct merged commit cb43a1c into main Jul 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant