Summary
Add a configurable maximum memory budget for CharLib characterization and use it to derive a safe ProcessPoolExecutor worker-recycling policy.
Rationale
max_tasks_per_child=1 currently mitigates a downstream memory leak, likely in PySpice. On the reporter's system, each worker starts at roughly 500 MB and leaks roughly 400 MB whenever it is reused. Recycling after every task avoids unbounded growth, but can add process-startup overhead.
Requested changes
- Expose a user-configurable maximum memory budget for characterization.
- Combine that budget with the configured parallelism (
settings.jobs) to calculate an appropriate worker recycle limit (max_tasks_per_child).
- Reserve at least a 10% safety factor in the budget calculation.
- Preserve a safe default behavior when no memory budget is configured, including protection against the known per-task worker memory growth.
- Document the configuration, calculation/assumptions, and fallback behavior.
Affected areas
charlib/characterizer/characterizer.py (CharacterizationSettings and ProcessPoolExecutor setup)- Configuration syntax and CLI/documentation as needed
Acceptance criteria
- Users can configure a maximum characterization memory budget.
- The executor worker recycling policy is derived from the memory budget,
settings.jobs, and a safety reserve of at least 10%.
- Invalid or insufficient budgets yield a clear validation error or a documented safe fallback.
- The default configuration remains protected from the observed PySpice-associated memory leak.
- Automated coverage verifies budget validation and recycle-limit calculation.
Backlinks
Summary
Add a configurable maximum memory budget for CharLib characterization and use it to derive a safe
ProcessPoolExecutorworker-recycling policy.Rationale
max_tasks_per_child=1currently mitigates a downstream memory leak, likely in PySpice. On the reporter's system, each worker starts at roughly 500 MB and leaks roughly 400 MB whenever it is reused. Recycling after every task avoids unbounded growth, but can add process-startup overhead.Requested changes
settings.jobs) to calculate an appropriate worker recycle limit (max_tasks_per_child).Affected areas
charlib/characterizer/characterizer.py(CharacterizationSettingsandProcessPoolExecutorsetup)Acceptance criteria
settings.jobs, and a safety reserve of at least 10%.Backlinks