Skip to content

[Feature]: verified OSS contribution loop for prioritized, non-duplicate agent work #524

Description

@kvnloo

Problem or Motivation

Agent Canvas can execute repository work, while OpenHands already requires a linked ready-for-dev issue with concrete desired behavior and acceptance criteria before a PR is eligible. The missing operator workflow is between a ready issue and a maintainable agent contribution:

  • several Agent Canvas runs can select or attempt the same issue;
  • a run can pass focused tests without proving the requested user behavior;
  • evidence can become detached from the final PR head after repair or rebase;
  • the implementing run can appear to validate its own work;
  • maintainers still have to reconstruct trajectory, environment, cost, and limitations from several surfaces.

This makes it hard to run OpenHands against a project priority queue without creating duplicate work or expensive review reconstruction.

Desired Behavior

As a maintainer, I can enable an Agent Canvas contribution workflow that begins only from an issue carrying ready-for-dev:

  1. Canvas lists repository-selected ready-for-dev issues in maintainer priority order.
  2. Starting a run creates an expiring claim tied to the issue, base SHA, actor, and Canvas run.
  3. A competing active claim or PR is visible before another run starts.
  4. The run works in an isolated branch/worktree and records its trajectory normally.
  5. The final result emits a compact contribution receipt bound to the exact PR head:
    • issue and acceptance criteria;
    • base/head SHAs and changed files;
    • pre-change reproduction when applicable;
    • focused tests plus the closest build or Agent Canvas user path;
    • trajectory/run ID, environment, model usage, and known limitations.
  6. A separate review run can inspect the issue, trajectory, receipt, diff, and exact head without writing to that branch.
  7. OpenHands reports ready for maintainer review, changes required, duplicate, or blocked; it never awards merge authority.

The existing issue, ready-for-dev automation, PR, and branch rules remain canonical. This is coordination and evidence around Agent Canvas, not another tracker.

Acceptance Criteria

  • Only issues carrying ready-for-dev are offered for autonomous implementation.
  • A claim records issue, base SHA, actor/run ID, scope, and expiry.
  • Active claims and linked PRs are shown before launch; expired claims stop blocking selection.
  • Each run uses isolated mutable source state.
  • The contribution receipt is regenerated or invalidated whenever the PR head changes.
  • Receipt evidence maps directly to the issue's acceptance criteria.
  • Evidence distinguishes focused tests, CI, mock/simulated behavior, and a real Agent Canvas path.
  • An independent read-only review run verifies the exact head.
  • Existing GitHub review and merge rules remain authoritative.
  • A pilot demonstrates one successful issue and one duplicate/stale claim.

Alternatives Considered

  • Select issues from labels only: ready-for-dev proves readiness, not that nobody else is already implementing it.
  • Rely on the PR test section: it does not automatically bind trajectory and runtime evidence to the current head.
  • Let each run post free-form logs: expensive for maintainers to compare and easy to make stale.
  • Create a separate OpenHands backlog: rejected; the repository's issues and readiness gate should remain the source of truth.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions