-
Notifications
You must be signed in to change notification settings - Fork 68
bench eval create should expose --max-retries alias #916
Copy link
Copy link
Open
Labels
P2Anti-pattern / type safety / docs precision / minor schema drift / non-deterministic but contained.Anti-pattern / type safety / docs precision / minor schema drift / non-deterministic but contained.area:evalIssue / PR lives primarily in the "eval" subsystem.Issue / PR lives primarily in the "eval" subsystem.bugSomething isn't workingSomething isn't workingreviewingFix PR open and awaiting reviewFix PR open and awaiting reviewstatus:in-progressHas assignee or linked draft PR.Has assignee or linked draft PR.
Description
Activity
Metadata
Metadata
Assignees
Labels
P2Anti-pattern / type safety / docs precision / minor schema drift / non-deterministic but contained.Anti-pattern / type safety / docs precision / minor schema drift / non-deterministic but contained.area:evalIssue / PR lives primarily in the "eval" subsystem.Issue / PR lives primarily in the "eval" subsystem.bugSomething isn't workingSomething isn't workingreviewingFix PR open and awaiting reviewFix PR open and awaiting reviewstatus:in-progressHas assignee or linked draft PR.Has assignee or linked draft PR.
Type
Projects
- StatusShow more project fieldsIn Progress
Mirrors Linear ENG-169: https://linear.app/benchflow/issue/ENG-169/cli-expose-max-retries-on-bench-eval-create
Problem
bench eval createsupports retry control via YAMLmax_retriesand CLI--retry-attempts, but users naturally try--max-retriesand getNo such option.Expected
--max-retriesshould be a first-class CLI alias for the current retry-attempt override and work for both--tasks-dirand--configeval runs. Passing both aliases should fail fast instead of silently choosing one.Current fix path
Daily scan 2026-07-11 confirmed this is a narrow BenchFlow CLI issue. A focused PR is being prepared.