Skip to content

CHECKLIST.md

Latest commit

 

History

History
732 lines (611 loc) · 20.8 KB

CHECKLIST.md

File metadata and controls

732 lines (611 loc) · 20.8 KB
Raw

Ability Control - Implementation Checklist

Last Updated: 2026-01-25
Current Phase: Phase 2 - Handler Execution

📋 Phase 1: Foundation ✅ COMPLETE

Core Plugin

  • Pattern detection (regex-based)
  • Context injection (hints, tools, constraints)
  • Toast notifications
  • File-based logging
  • Plugin hooks (chat.message, event)

Configuration System

  • Config schema (Zod)
  • Config loader (.opencode/ability-control/config.json)
  • Enable/disable built-in abilities
  • Custom ability directories
  • Logging configuration

CLI

  • Interactive setup (npx ability-control init)
  • Directory structure creation
  • Example abilities
  • README generation

Documentation

  • README.md (installation, usage, examples)
  • LICENSE (MIT)
  • DESIGN.md (architecture, principles)
  • CHECKLIST.md (this file)

Cleanup

  • Delete obsolete postinstall script
  • Delete old .abilities/ directory
  • Fix config loader path
  • Rebuild plugin and CLI

🔨 Phase 2: Handler Execution ⏳ IN PROGRESS

Goal: Execute TypeScript functions when abilities trigger
Timeline: 4 weeks
Status: Not started

2.1 Handler Context API

  • Define HandlerContext interface
    • ability: Readonly<Ability> - Ability definition
    • params: Record<string, any> - Extracted parameters
    • client: PluginClient - OpenCode client
    • logger: Logger - Scoped logger
    • progress: ProgressTracker - Progress updates
  • Define HandlerResult interface
    • success: boolean
    • message: string
    • data?: any
    • error?: Error
  • Create handler template generator

2.2 Parameter Extraction

  • Extract regex capture groups
  • Map capture groups to handler params
  • Support named capture groups
  • Type validation (string, number, boolean)
  • Default values
  • Required vs optional params

2.3 Handler Loader

  • Dynamic import of TypeScript handlers
  • Validate handler exports (must export execute function)
  • Cache loaded handlers (performance)
  • Hot reload handlers in dev mode
  • Error handling (handler not found, invalid export)

2.4 Execution Engine

  • Execute handler with context
  • Timeout enforcement (default 30s)
  • Error handling and recovery
  • Return results to Claude
  • Log execution (start, end, duration, result)

2.5 Permission Validation

  • Validate allowed tools
  • Validate allowed paths (glob patterns)
  • Validate denied paths
  • Require user confirmation (if configured)
  • Block execution if validation fails

2.6 Loop Prevention (CRITICAL)

  • Layer 1: Execution Depth Tracking

    • Track execution depth per session
    • Prevent ability-to-ability calls
    • Detect circular references
    • Enforce max depth (default: 1)
    • Log loop prevention events

  • Layer 2: Message Source Detection

    • Detect user vs agent messages
    • Block agent self-questioning during execution
    • Track who triggered execution (user/agent/handler)
    • Show warning toast when agent loop detected

  • Layer 3: Pattern Detection

    • Track execution history per session
    • Detect same ability triggered multiple times (>3 in 60s)
    • Detect rapid switching (A→B→A→B→A)
    • Detect consecutive agent-triggered executions (>3)
    • Block session for 5 minutes on loop detection

  • Layer 4: Rate Limiting

    • Implement per-ability rate limits
    • Max 3 executions per minute per ability
    • Sliding window rate limiting
    • Log rate limit violations

  • Configuration

    • Add execution.maxDepth config
    • Add execution.rateLimit config
    • Add loopDetection.enabled config
    • Add loopDetection.agentMessageBlocking config
    • Add loopDetection.patternDetection config

  • CLI Commands

    • npx ability-control safety - Show safety status
    • npx ability-control unblock - Unblock session
    • npx ability-control history --loops - Show loop detections

2.6b Pattern Matching Improvements

  • Word Boundary Enforcement

    • Add \b word boundaries to patterns
    • Prevent substring matches (coo vs cookie)
    • Smart boundary detection (don't break existing patterns)

  • Specificity Scoring

    • Calculate pattern specificity (length, words, capture groups)
    • Use specificity as tiebreaker for equal confidence
    • Prefer longer, more specific patterns

  • Negative Patterns

    • Support exclude in trigger definition
    • Skip matches if exclusion words present
    • Add to YAML schema

  • Context-Aware Matching

    • Support requires (must contain words)
    • Support boostIf (increase confidence)
    • Support penalizeIf (decrease confidence)
    • Add to YAML schema

  • Confidence Calculation

    • Boost for exact matches
    • Boost for high match ratio (>70%)
    • Penalize for low match ratio (<30%)
    • Adjust based on context words

2.7 Progress Tracking

  • Create ProgressTracker class
  • Store progress in .tmp/ability-control/progress/
  • Update progress via toast notifications
  • Support percentage-based progress
  • Support step-based progress
  • CLI: npx ability-control active (show running abilities)

2.8 CLI Commands

  • npx ability-control test <text> - Test pattern matching
    • Show matched abilities
    • Show confidence scores
    • Show extracted parameters
  • npx ability-control run <name> - Execute ability
    • Support --dry-run flag
    • Support --params flag (JSON)
    • Show execution trace
  • npx ability-control validate - Validate all abilities
    • Check YAML syntax
    • Check schema validity
    • Check handler files exist
    • Check regex patterns compile
    • Check permissions valid

2.9 Testing

  • Unit tests for handler loader
  • Unit tests for parameter extraction
  • Unit tests for execution engine
  • Unit tests for permission validation
  • Integration tests (full ability execution)
  • Example abilities with handlers

2.10 Documentation

  • Handler API reference
  • Parameter extraction guide
  • Permission system guide
  • Testing guide
  • Example handlers

🎯 Phase 2b: Original Vision Features ⏳ NOT STARTED

Goal: Add missing features from original vision (Issue #33)
Timeline: 3 weeks
Status: Not started

2b.1 Conditional Context Loading

  • Schema Definition

    • Add context.conditional to YAML schema
    • Support when conditions (JavaScript expressions)
    • Support files array (context files to load)
    • Support hints array (conditional hints)
    • Support tools array (conditional tools)

  • Condition Evaluator

    • Parse JavaScript expressions safely
    • Evaluate conditions against input values
    • Support comparison operators (===, !==, >, <, >=, <=)
    • Support logical operators (&&, ||, !)
    • Sandbox evaluation (prevent code injection)

  • Context Loader

    • Load context.always files first
    • Evaluate each conditional
    • Load matching context files
    • Merge conditional hints/tools
    • Track loaded context size (optimization)

  • Validation

    • Validate condition syntax
    • Validate referenced files exist
    • Check context size limits
    • Warn if conditions never match

  • Example

    context:
  always:
    files: [video-basics.md]
  conditional:
    - when: inputs.platform === "youtube"
      files: [youtube-specs.md]
      hints: [Optimize for YouTube algorithm]

2b.2 Output Schemas

  • Schema Definition

    • Add outputs to YAML schema
    • Support type definitions (string, integer, boolean, array, object)
    • Support required vs optional
    • Support nested objects
    • Support array item types

  • Validation

    • Validate handler return values against schema
    • Generate TypeScript types from schema
    • Show validation errors clearly
    • Support custom validators

  • Type Generation

    • Generate TypeScript interfaces from output schema
    • Export types for handler implementations
    • Type-safe handler return values

  • Example

    outputs:
  videoPath: { type: string, required: true }
  duration: { type: integer, required: true }
  nextSteps: { type: array, items: { type: string } }

2b.3 Multi-Turn Guidance

  • Schema Definition

    • Add guidance to YAML schema
    • Support on_success message
    • Support on_failure message
    • Support template variables ({{inputs.platform}})
    • Support suggested next abilities

  • Guidance Formatter

    • Replace template variables with actual values
    • Format guidance messages
    • Inject into Claude's context after execution
    • Show suggested abilities

  • Integration

    • Show guidance in toast notifications
    • Inject guidance into system prompt
    • Track suggested abilities
    • Enable chaining abilities

  • Example

    guidance:
  on_success: |
    Video created! Suggested next abilities:
    - upload-video: Upload to {{inputs.platform}}
    - thumbnail-generator: Create thumbnail

2b.4 Agent-Specific Abilities

  • Schema Definition

    • Add agent configuration to agent markdown files
    • Support abilities.enabled array
    • Support abilities.disabled array
    • Support wildcard patterns (video-*)

  • Agent Loader

    • Parse agent markdown frontmatter
    • Extract ability configuration
    • Filter abilities per agent
    • Cache agent configurations

  • Scoping

    • Load only enabled abilities for current agent
    • Respect disabled abilities
    • Support global abilities (always enabled)
    • Support agent inheritance

  • Example

    # .opencode/agents/content-creator.md
---
abilities:
  enabled: [video-creation, blog-writing]
  disabled: [code-generation]
---

2b.5 Instructions Field

  • Schema Definition

    • Add instructions to YAML schema
    • Support multi-line markdown
    • Support template variables
    • Inject into system prompt

  • Formatter

    • Format instructions as markdown
    • Replace template variables
    • Inject into Claude's system prompt
    • Show in ability documentation

  • Example

    instructions: |
  Create videos with video-creation.
  
  Use when: User asks for video content
  Requires: topic, duration, platform
  Returns: Video path, script path, next steps

2b.6 Testing

  • Unit tests for conditional context evaluation
  • Unit tests for output schema validation
  • Unit tests for guidance formatting
  • Unit tests for agent scoping
  • Integration tests (full features)
  • Example abilities using all features

2b.7 Documentation

  • Conditional context guide
  • Output schema reference
  • Multi-turn guidance guide
  • Agent scoping guide
  • Complete YAML schema reference
  • Migration guide (from basic to advanced)

🔄 Phase 3: Workflow Orchestration ⏳ NOT STARTED

Goal: Multi-step workflows with dependencies
Timeline: 6 weeks
Status: Not started

3.1 Workflow Schema

  • Define workflow YAML schema
  • Support sequential steps
  • Support parallel steps
  • Support conditional steps
  • Support step dependencies
  • Validate workflow schema

3.2 Dependency Resolution

  • Build dependency graph
  • Detect circular dependencies
  • Calculate execution order
  • Identify parallelizable steps
  • Handle missing dependencies

3.3 Workflow Executor

  • Execute steps in order
  • Execute parallel steps concurrently
  • Pass data between steps
  • Handle step failures
  • Rollback on error
  • Resume from checkpoint

3.4 Task Integration

  • Create tasks in .tmp/tasks/
  • Track workflow as task
  • Track steps as subtasks
  • Update task status
  • Mark tasks complete
  • Integration with task-management skill

3.5 Progress Tracking

  • Track workflow progress
  • Track step progress
  • Show estimated time remaining
  • Show current step
  • Show completed steps

3.6 Error Handling

  • Retry failed steps
  • Skip optional steps
  • Rollback on failure
  • Cleanup on error
  • Log errors

3.7 CLI Commands

  • npx ability-control workflow <name> - Show workflow details
  • npx ability-control workflow <name> --graph - Show dependency graph
  • npx ability-control workflow <name> --validate - Validate workflow

3.8 Testing

  • Unit tests for dependency resolution
  • Unit tests for workflow executor
  • Integration tests (full workflow)
  • Example workflows

3.9 Documentation

  • Workflow schema reference
  • Dependency resolution guide
  • Error handling guide
  • Example workflows

🌐 Phase 4: Registry ⏳ NOT STARTED

Goal: Publish and install abilities from registry
Timeline: 4 weeks
Status: Not started

4.1 Registry API

  • Design registry API
  • Choose hosting (npm vs custom)
  • Authentication (API keys)
  • Rate limiting
  • Search endpoint
  • Publish endpoint
  • Install endpoint

4.2 Publish Command

  • npx ability-control publish <name>
  • Validate ability before publish
  • Bump version (patch, minor, major)
  • Generate changelog
  • Package ability (YAML + handlers + schemas)
  • Upload to registry
  • Tag git commit

4.3 Install Command

  • npx ability-control install <name>
  • Download from registry
  • Extract to .opencode/ability-control/abilities/
  • Update config.json
  • Install dependencies (npm packages)
  • Verify integrity (checksums)

4.4 Search Command

  • npx ability-control search <query>
  • Search by name
  • Search by description
  • Search by tags
  • Filter by author
  • Sort by downloads/stars

4.5 Version Management

  • Semantic versioning
  • Dependency tracking
  • Breaking change detection
  • Migration guides
  • Deprecation warnings

4.6 Security

  • Code scanning (detect malicious code)
  • Verified publishers
  • Checksum validation
  • Sandboxed execution
  • Report malicious abilities

4.7 Analytics

  • Track downloads
  • Track installs
  • Track usage
  • Popular abilities
  • Trending abilities

4.8 Testing

  • Unit tests for publish
  • Unit tests for install
  • Integration tests (full flow)
  • Security tests

4.9 Documentation

  • Registry API reference
  • Publishing guide
  • Installing guide
  • Security best practices

🎨 Phase 5: Developer Experience ⏳ NOT STARTED

Goal: Make it easy to create, test, and debug abilities
Timeline: 3 weeks
Status: Not started

5.1 CLI Commands

  • npx ability-control add <name> - Create new ability
    • Interactive prompts
    • Generate YAML file
    • Generate handler template
    • Generate schema template
    • Add to config.json
  • npx ability-control list - Show all abilities
    • Show enabled/disabled
    • Show version
    • Show source (built-in, custom, registry)
  • npx ability-control enable <name> - Enable ability
  • npx ability-control disable <name> - Disable ability
  • npx ability-control remove <name> - Remove ability
  • npx ability-control history - Show execution history
    • Filter by ability
    • Filter by date
    • Show success/failure
    • Show duration

5.2 Debug Mode

  • Enable via config: "logging": { "level": "debug" }
  • Log all pattern matching attempts
  • Log parameter extraction
  • Log handler execution
  • Log permission checks
  • Log execution trace

5.3 Testing Framework

  • Create @ability-control/testing package
  • Mock HandlerContext
  • Mock OpenCode client
  • Test pattern matching
  • Test handler execution
  • Test workflows
  • Example test suite

5.4 Documentation Generator

  • Generate README from YAML
  • Generate API docs from handlers
  • Generate examples from tests
  • Generate changelog from git

5.5 Migration Tools

  • npx ability-control migrate <name> --to <version>
  • Detect breaking changes
  • Show migration guide
  • Update YAML schema
  • Update handler signature

5.6 Testing

  • Unit tests for CLI commands
  • Integration tests (full CLI flow)
  • Example abilities

5.7 Documentation

  • CLI reference
  • Debug guide
  • Testing guide
  • Migration guide

🚀 Phase 6: Advanced Features ⏳ NOT STARTED

Goal: Integrate with OpenCode ecosystem
Timeline: 8 weeks
Status: Not started

6.1 Skill Integration

  • Load skills via skill: <name> in YAML
  • Inject skill content into Claude's prompt
  • Pass skill context to handlers
  • Test with task-management skill

6.2 Command Integration

  • Execute commands via command: /<name> in YAML
  • Pass command results to handlers
  • Test with /commit, /clean commands

6.3 Conflict Resolution

  • Implement confidence threshold (0.8)
  • Handle multiple matches
  • Prompt user to choose
  • Remember user preferences

6.4 Ability Collections

  • Group related abilities
  • Install collections
  • Share collections
  • Example: @react/components (create, update, delete)

6.5 Community Contributions

  • Contribution guide
  • Code of conduct
  • Issue templates
  • PR templates
  • Example abilities

6.6 Testing

  • Integration tests (skills, commands)
  • End-to-end tests
  • Performance tests

6.7 Documentation

  • Integration guide
  • Best practices
  • Community guide

📊 Progress Tracking

Overall Progress

  • Phase 1: ✅ 100% (Complete)
  • Phase 2: ⏳ 0% (Not started)
  • Phase 3: ⏳ 0% (Not started)
  • Phase 4: ⏳ 0% (Not started)
  • Phase 5: ⏳ 0% (Not started)
  • Phase 6: ⏳ 0% (Not started)

Current Sprint (Phase 2.1-2.3)

Goal: Handler execution basics
Timeline: Week 1-2

  • Define HandlerContext interface
  • Define HandlerResult interface
  • Extract regex capture groups
  • Map capture groups to params
  • Dynamic import handlers
  • Validate handler exports
  • Execute handler with timeout
  • Return results to Claude

Next Sprint (Phase 2.4-2.6)

Goal: Safety and permissions
Timeline: Week 3-4

  • Permission validation
  • Loop prevention
  • Progress tracking
  • CLI: test, run, validate

🎯 Success Criteria

Phase 2 Complete When:

  • Can execute TypeScript handlers
  • Can extract parameters from regex
  • Can validate permissions
  • Can prevent infinite loops
  • Can track progress
  • CLI commands work (test, run, validate)
  • Example abilities work (create-component)
  • Tests pass (unit + integration)
  • Documentation complete

Phase 3 Complete When:

  • Can execute multi-step workflows
  • Can resolve dependencies
  • Can execute parallel steps
  • Can rollback on error
  • Can integrate with task-management
  • Example workflows work
  • Tests pass
  • Documentation complete

Phase 4 Complete When:

  • Can publish to registry
  • Can install from registry
  • Can search registry
  • Can manage versions
  • Security scanning works
  • Tests pass
  • Documentation complete

📝 Notes

Design Decisions

  • No ability-to-ability calls - Prevents infinite loops
  • TypeScript only - Type safety, better DX
  • Zod for validation - Type-safe, great DX
  • npm registry first - Easier to start, custom registry later
  • Complement, don't replace - Work with skills/commands

Open Questions

  • Should we support JavaScript handlers? (Decision: No, TypeScript only)
  • Should we support Python handlers? (Decision: No, TypeScript only)
  • Should we use custom workflow engine? (Decision: Start simple, evaluate later)
  • Should we build custom registry? (Decision: Start with npm, add custom later)

Risks

  • Complexity - Workflows can get complex, need good debugging
  • Performance - Handler execution might be slow, need caching
  • Security - Malicious handlers, need sandboxing
  • Breaking changes - Schema changes break old abilities, need migrations

Mitigation

  • Complexity → Debug mode, execution traces, testing framework
  • Performance → Handler caching, lazy loading, parallel execution
  • Security → Code scanning, sandboxing, verified publishers
  • Breaking changes → Semantic versioning, migration tools, deprecation warnings

Last Updated: 2026-01-25
Next Review: After Phase 2 Sprint 1 (Week 2)