Repository navigation
Size STARsolo memory to index + sort buffer; make it configurable - #93
Merged
Merged
Conversation
The starsolo rules requested mem_mb=48000 but passed --limitBAMsortRAM
48000000000, so STAR's peak (genome index in RAM + 48 GB sort buffer)
exceeded the allocation and large samples were OOM-killed on SLURM.
- mem_mb defaults to the STAR_INDEX size on disk + 16 GB sort buffer +
4 GB headroom; override with STAR_mem_mb (sample/platform/chemistry/
DEFAULTS) or set-resources; scaled by the attempt number.
- --limitBAMsortRAM is derived from the job's final mem_mb, so any
override keeps the two in sync; warn when < 2 GB remains for sorting.
- SLURM profile: retries: 1 (STAR retries with 2x memory).
- --runThreadN {threads} instead of a hardcoded 12.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
An Alpine run was OOM-killed at starsolo. The rules requested
mem_mb = 48000but told STAR--limitBAMsortRAM 48000000000. STAR keeps the genome index (about 30 GB for human) in RAM on top of the sort buffer, so large samples exceeded the allocation while sorting.Fix
mem_mbdefaults to theSTAR_INDEXsize on disk + 16 GB sort buffer + 4 GB headroom (32 GB is assumed for the index when the directory is absent, e.g. in CI).STAR_mem_mb(sample, platform, chemistry or DEFAULTS), or withset-resources/--set-resources.--limitBAMsortRAMis derived from the job's finalmem_mb, so it follows any override. A warning is logged if less than 2 GB is left for sorting.retries: 1.mem_mbscales with the attempt number, so a failed STAR job retries with 2x memory.--runThreadN {threads}replaces the hardcoded 12.config.yamland the profile README, including Alpine's limit of about 3.8 GB per core onacpu.Verification (Snakemake 9.27.1 dry-runs)
--limitBAMsortRAM--set-resources starsolo_R2:mem_mb=96000STAR_mem_mb: 80000The full dry-run (38 jobs) and the
--profile profiles/slurmdry-run both pass.🤖 Generated with Claude Code