Feat ai scientist agent

[TerminallyLazy]

Summary

Implements the AI Scientist agent profile for Agent Zero - an autonomous research assistant that can generate novel research ideas, execute ML experiments, and write publication-ready papers. Inspired by https://arxiv.org/abs/2408.06292 from Sakana AI.

Key Features:

• Research Idea Generation - Generates novel ideas with literature search, multi-round refinement, and novelty scoring
• 4-Stage Experiment Pipeline - Initial implementation → Baseline tuning → Creative research → Ablation studies
• Paper Generation - Produces LaTeX manuscripts with automatic citation gathering via Semantic Scholar
• Profile-Specific UI - Three dedicated modal interfaces accessible via profile-specific buttons

Changes

New Agent Profile (agents/ai-scientist/)

• agent.json - Profile configuration with title, description, and context
• settings.json - Uses inherited model settings (no hardcoded models)
• _context.md - Role definition and methodology documentation
• prompts/ - Specialized system prompts for research methodology
• extensions/ - State initialization for research data containers
• tools/ - Four specialized tools:
  • generate_idea.py - Multi-round idea generation with novelty validation
  • run_experiment.py - 4-stage experiment pipeline with best-first tree search
  • write_paper.py - LaTeX paper generation with reflection rounds
  • semantic_scholar.py - Academic paper search integration

API Endpoints (python/api/)

• scientist_generate_ideas.py - Trigger idea generation
• scientist_get_ideas.py - Retrieve generated ideas
• scientist_start_experiment.py - Launch experiment pipeline
• scientist_get_experiments.py - List experiments with status
• scientist_get_experiment_progress.py - Real-time progress polling
• (Paper generation uses existing experiment data)

Frontend (webui/)

• Ideas Manager Modal - Topic input, idea cards with novelty badges, detailed view
• Experiment Dashboard Modal - Stage pipeline visualization, node tree, real-time progress
• Paper Generator Modal - Experiment selection, format options, generation progress
• Profile-Specific Buttons - Ideas/Experiments/Papers buttons appear only when AI Scientist profile is active

Architecture

  ┌─────────────────────────────────────────────────────────────┐
  │                     AI Scientist Profile                      │
  ├─────────────────────────────────────────────────────────────┤
  │  Tools                    │  UI Components                   │
  │  ├─ generate_idea         │  ├─ Ideas Manager Modal          │
  │  ├─ run_experiment        │  ├─ Experiment Dashboard Modal   │
  │  ├─ write_paper           │  └─ Paper Generator Modal        │
  │  └─ semantic_scholar      │                                   │
  ├─────────────────────────────────────────────────────────────┤
  │  Data Flow: context.data["research_ideas"|"experiments"|"papers"]│
  │  Storage: Persistent across sessions via AgentContext          │
  └─────────────────────────────────────────────────────────────┘

Technical Notes

• Tool Discovery: Tools require both Python implementations AND agent.system.tool.*.md instruction files for LLM awareness
• Data Storage: Uses context.data (not agent.data) for persistence across API calls
• Async Execution: Agent runs asynchronously; UI uses polling to detect completion
• JSON Parsing: Includes defensive parsing for LLM-generated JSON with wrapper extraction and field validation