diff --git a/plugins/test-management/skills/qa-sprint-report-by-testomatio b/plugins/test-management/skills/qa-sprint-report-by-testomatio new file mode 120000 index 0000000..03c7012 --- /dev/null +++ b/plugins/test-management/skills/qa-sprint-report-by-testomatio @@ -0,0 +1 @@ +../../../skills/qa-sprint-report-by-testomatio \ No newline at end of file diff --git a/skills/qa-sprint-report-by-testomatio/SKILL.md b/skills/qa-sprint-report-by-testomatio/SKILL.md new file mode 100644 index 0000000..40646c6 --- /dev/null +++ b/skills/qa-sprint-report-by-testomatio/SKILL.md @@ -0,0 +1,220 @@ +--- +name: qa-sprint-report-by-testomatio +description: Generate a QA Sprint Progress Summary Report from Testomat.io TMS data. Use when the user wants to produce a structured end-of-sprint report covering test design coverage, execution results, defect trends, and quality signals. The skill reads data via Testomat.io MCP tools, maps it to the report sections, and outputs an `.html` file by default (or `.md` if explicitly requested). +license: MIT +metadata: + author: Testomat.io + version: 1.0.0 +--- + +# QA Sprint Report by Testomat.io + +## When to Use + +- End of sprint — produce a shareable QA report for stakeholders. +- Sprint review — need a structured view of what was tested, passed, failed, blocked. +- Mid-sprint status — capture current execution progress. +- The user mentions "sprint report", "QA summary", "sprint progress", or similar. + +## How to Identify the Sprint + +The user may identify the sprint in one of these ways (in priority order): + +1. **Milestone name or ID** — e.g. "Sprint 42", "Sprint 2024.3" — use `milestones_list(type="Sprint")` to find it. +2. **Run group title** — e.g. "Sprint 42 Run" — use `rungroups_list` / `rungroups_get`. +3. **Direct run ID or run title** — use `runs_get` or `runs_list`. +4. **Plan title or ID** — use `plans_list` / `plans_get`. +5. **No identifier provided** — ask the user to specify the sprint milestone, run group, or run. + +> If the user provides only a sprint name like "Sprint 42", always resolve it to a concrete milestone/rungroup/run first. Do not assume the title directly maps to an ID. + +## Rules + +- **Do not fabricate data.** If MCP does not return a value, use `[None]` or hide the section. Never invent numbers or statuses. +- **Resolve sprint first.** Never assume a sprint name maps directly to a run ID without verification. +- **Hide unavailable sections.** If analytics (Section 8) is not available, omit it entirely rather than leaving placeholder text. +- **One blank line between actions.** In any generated code or CLI snippets. +- **Use TQL for filtering.** Always prefer `tql` filters over client-side filtering when calling `runs_list`, `tests_list`, `testruns_list`. +- **HTML is the default output.** Generate `.html` first; only offer `.md` if the user explicitly asks for it or their initial prompt includes a request to save as `.md`. + +--- + +# MCP Tools Mapping + +Each report section maps to specific MCP tools. See the full section structure and metrics in the template: + +- [qa-sprint-report.md](./references/qa-sprint-report.md) — section layout with fill-in placeholders + +### Quick Commands + +| Action | MCP Tool | Key Parameters | +|--------|----------|----------------| +| Find sprint milestone | `milestones_list` | `type="Sprint"`, `status="active"` | +| List runs for sprint | `runs_list` | `tql`: `milestone == '{id}'` | +| Get run details | `runs_get` | `run_id` | +| Get test results | `testruns_list` | `run_id`, `filter_status` | +| Get suite tests | `tests_list` | `suite_id`, `per_page=100` | +| Check analytics availability | `system_ping` | (Enterprise flag in response) | +| Get test health analytics | `analytics_tests` | `kind`, `days`, `from`, `to` | +| Get stats analytics | `analytics_stats` | `kind`, `from`, `to` | + +--- + +# Workflow + +### Step 1: Resolve Sprint Identity + +Ask the user if no sprint identifier is provided. Try to resolve: + +``` +1. milestones_list(type="Sprint") — find milestone matching sprint name +2. rungroups_list — find rungroup with sprint title +3. runs_list(tql="title % 'Sprint N'") — find runs by title +``` + +### Step 2: Gather Sprint Metadata & Time Range + +Collect for the report header: + +- `milestones_get(milestone_id)` or `rungroups_get(rungroup_id)` for sprint title/dates. +- Ask user for QA Lead name if not in TMS. + +**Critical: Extract Sprint Time Range** + +Resolve the sprint time range in this priority order: + +1. **User-provided** — If the user specified a time range in their original prompt, use it. +2. **Milestone/Rungroup dates** — Extract `start_date` / `due_date` from `milestones_get` or `rungroups_get`. +3. **Run timestamps** — If milestone/rungroup has no dates, derive from earliest `created_at` and latest `finished_at` across the sprint's runs (`runs_get`). +4. **Manual input** — If no time range can be determined from TMS, ask the user to specify the sprint period manually. + +The time range is required for: +- `analytics_tests(kind="...", from="YYYY-MM-DD", to="YYYY-MM-DD")` +- `analytics_stats(kind="...", from="YYYY-MM-DD", to="YYYY-MM-DD")` +- Filtering runs by sprint period + +### Step 3: Collect Run Data + +1. `runs_list(tql="milestone == '{id}'")` — all runs for the sprint milestone +2. For each run, call `runs_get` to get status, counts, environments +3. `testruns_list(run_id, filter_status)` — get passed/failed/skipped per run + +### Step 4: Build Report Sections + +Iterate sections 1–9 of the template, populating each with MCP data. Apply rules: + +- If a metric is not available → use `[None]` in the cell +- If an entire section has no data → hide the section (do not show placeholder text) +- Section 8 (Analytics) → check `system_ping` response for Enterprise capability; if not available, hide + +### Step 5: Write Output File + +Write the filled report to: +``` +{user-specified-path}/QA_Sprint_Progress_Report_{SprintName}_{YYYY-MM-DD}.html +``` + +If no path specified, save to current working directory. + +### Step 6: Offer Markdown Export + +After writing the `.html` file, ask the user if they also want to export the report as a `.md` file: + +``` +❓ The report has been saved to `{path}`. +Would you like to also export it as a `.md` file? + +1. ✏️ Yes, save as `.md` in the same location +2. 👍 No, keep the `.html` only +``` + +**If user picks option 1:** +1. Convert the HTML content to Markdown format. +1. Generate ".md" markdown format report using template from the references - [qa-sprint-report.md](./references/qa-sprint-report.md). +2. Save as `QA_Sprint_Progress_Report_{SprintName}_{YYYY-MM-DD}.md` in the **same directory** as the HTML file. + +**If user picks option 2:** +- Confirm the `.html` path and end the workflow. + +## HTML Template Styling + +When exporting to HTML, apply the following styles: + +### Color Palette + +| Token | Hex | Usage | +|-------|-----|-------| +| `--primary-color` | `#6366f1` | Headers, badges, links | +| `--primary-hover` | `#4f46e5` | Hover states | +| `--success-color` | `#10b981` | Passed/ready badges | +| `--danger-color` | `#ef4444` | Failed/blocked badges | +| `--warning-color` | `#f59e0b` | In-progress badges | +| `--info-color` | `#3b82f6` | Info badges | +| `--gray-50` to `--gray-900` | `#f9fafb` → `#111827` | Text hierarchy | + +### Typography + +- **Font Family:** Inter (Google Fonts) with fallback: `-apple-system, BlinkMacSystemFont, sans-serif` +- **Font Weights:** 300 (light), 400 (regular), 500 (medium), 600 (semibold), 700 (bold), 800 (extrabold) +- **Monospace:** `ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas` — for ticket IDs and code +- **Icons:** Font Awesome 6.4.0 (`fa-solid`) + +### Status Badges + +| Status | Background | Text Color | +|--------|------------|------------| +| `passed`, `ready` | `#d1fae5` | `#065f46` | +| `failed`, `blocked` | `#fee2e2` | `#991b1b` | +| `progress` | `#fef3c7` | `#92400e` | +| `info` | `#dbeafe` | `#1e40af` | + +### Key Elements + +| Element | Description | +|---------|-------------| +| `.main-container` | White glass card (`rgba(255,255,255,0.98)`) with `backdrop-filter: blur(20px)`, max-width 1300px, border-radius 20px | +| `.header` | Gradient background (`linear-gradient(135deg, #6366f1, #4f46e5)`), white text, flex layout | +| `.section-card` | White background, 1px border `--gray-200`, border-radius 12px, box-shadow | +| `.report-table` | Full-width, collapse borders, hover highlight on rows | +| `.meta-grid` | CSS Grid (`repeat(auto-fit, minmax(220px, 1fr))`), gap 20px | +| `.instruction-card` | Info box with left 5px border in `--primary-color`, background `#eef2ff` | +| `.tms-link` | Primary color link with hover underline | + +### HTML Structure + +```html +
+
+
+

QA Sprint Progress Report

+

[Project Name]

+
+
+ Sprint: [Sprint Number] +
+
+ +
+
+
How to Use This Template
+
    ...
+
+
+ +
+
+
1. Sprint Summary Scorecard
+
...
+
+ +
+
+``` + +--- + +# Report Template Reference + +| Section | Description | +|---------|-------------| +| [qa-sprint-report.md](./references/qa-sprint-report.md) | Full template with all sections, emoji, and fill-in placeholders | \ No newline at end of file diff --git a/skills/qa-sprint-report-by-testomatio/references/qa-sprint-report.md b/skills/qa-sprint-report-by-testomatio/references/qa-sprint-report.md new file mode 100644 index 0000000..9b90383 --- /dev/null +++ b/skills/qa-sprint-report-by-testomatio/references/qa-sprint-report.md @@ -0,0 +1,126 @@ +# QA Sprint Progress Summary Report Example + +The **QA Sprint Progress Summary Report** provides a structured, end-of-cycle view of all testing activities carried out during a sprint. It captures test design coverage, execution results, defect trends, and quality signals to keep stakeholders informed on product readiness. + +> All sections and metrics in this report are sourced from **Testomat.io TMS** via MCP tools. If a metric or data point is not available in the TMS, the corresponding section is omitted or the value is set to `[None]`. Sections that rely solely on unavailable data are hidden entirely. + +## 📋 General Sprint Meta +* **Project / Stream:** [Project Name / Stream] +* **QA Manager:** [Your Name] +* **Sprint Cycle:** [Sprint Number / Name] +* **Timeline Period:** [DD.MM.YYYY – DD.MM.YYYY] + +--- + +## 📊 1. Sprint Summary Scorecard + +| Metric | Value | Notes & Data Source | +| :--- | :---: | :--- | +| 🖊️ **New Test Cases Added to TMS** | [N] | Net new TCs authored in Testomat.io during this sprint. | +| 📋 **Total TCs in Project (End of Sprint)** | [N] | `tests_list` count at sprint close. | +| ⚙️ **Automation Rate** | [XX%] | Automated TCs ÷ Total TCs — from `analytics_stats(kind="automation-rate-by-date")`. | +| 📈 **Overall Pass Rate** | [XX%] | From `analytics_stats(kind="success-rate-by-date")` for the sprint window. | +| 🏁 **Tickets Tested** | [N] | Total number of tickets which were finished during the sprint. | +| ✅ **Tickets Completed (All Tests Passed)** | [N] | Formally approved and ready for delivery/release -> finished with all passed tests. | +| 🐛 **Defects/Bugs Identified** | [N] | Newly identified defects during execution. | + +--- + +## 🛠️ 2. Test Design & Coverage Expansion + +| Ticket ID | Suite / Feature Area | Test Case Title(s) | Total TCs | TMS State | +| :--- | :--- | :--- | :---: | :--- | +| **[PROJ-123]** | Authentication | [TC Title 1]; [TC Title 2] | [N] | [Manual / Automated] | +| **[PROJ-456]** | Data Export | [TC Title 1] | [N] | [Automated] | +| **[PROJ-789]** | Data Export | [TC Title 1]; [TC Title 2] | [N] | [Mix: Automated & Manual] | +| *— No new test cases added —* | | | | | + +> **TMS State** column: pull from `tests_list` → `state` field (`Manual` / `Automated` / `Mix: Automated & Manual`). + +--- + +## 🎯 3. Testing Execution Progress by Ticket, Suite + +> Report group is **Suite** (the primary TMS grouping). Each row corresponds to a Testomat.io Suite. Use `testruns_list(run_id="...")` to populate Passed / Failed / Skipped counts per suite. + +| Ticket Name / Suite / Feature Area | Run Title | Total TCs | Passed | Failed | Skipped | Blocked | Execution % | Suite Release Status | +| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- | +| [Suite Name — e.g. Billing] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | 🟡 In Progress | +| [Suite Name — e.g. Profile] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | ✅ Completed (All Tests Passed) | +| [Suite Name — e.g. Analytics] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | 🔴 Blocked | +| **TOTAL** | | **[N]** | **`[N]`** | **`[N]`** | **`[N]`** | **`[N]`** | **`[XX]%`** | **Overall Pass Rate:** `[XX%]` | + +> **Status Key:** +> ✅ **Completed (All Tests Passed):** 100% test execution completed with zero critical defects. +> 🟡 **In Progress:** Validations are actively running or includes in-progress cases; no major technical hurdles encountered (🟡 >51% pass = In Progress). +> 🔴 **Blocked:** Discovered failure, issues (🔴<50% pass or blocked). + +--- + +## ✅ 4. Tickets Moved to Sign-Off (already passed) +* `[PROJ-XXX]` — `[User Story Name]` | **Verified:** `[DD.MM.YYYY]` | **TMS Run:** `[Link to Testomat.io Run]` +* `[PROJ-YYY]` — `[User Story Name]` | **Verified:** `[DD.MM.YYYY]` | **TMS Run:** `[Link to Testomat.io Run]` +* *— No tickets signed off during this period —* + +--- + +## 🔁 5. Quality Feedback Loop (Returned to Dev) +* `[PROJ-881]` — *`[Feature Name]`*: `[AC #2 failed: App crashes when session expires / Linked to BUG-99]` (**Return Count:** `[1st time / 2nd time]`) +* *— No tickets returned to development —* + +--- + +## 🐛 6. Sprint Bug Tracking + +| Bug ID | Title / Summary Description | Priority | Linked Ticket | +| :--- | :--- | :---: | :--- | :--- | +| `[BUG-001]` | `[Short descriptive bug title]` | 🔴 Critical | `[PROJ-XXX]` | +| `[BUG-002]` | `[Short descriptive bug title]` | 🟠 Major | `[PROJ-XXX]` | +| *— No bugs registered this week —* | | | + +--- + +## 📊 8. TMS Test Health Analytics + +> Data sourced via MCP `analytics_tests` and `analytics_stats` for the **current sprint period** only in case if you have access to this analytics. +(If most of the metrics could not be obtained, this section should be removed from the results.) + +### 8.1 Test Health Overview +| Metric | Value | Notes | +| :--- | :---: | :--- | +| 🔀 **Flaky Tests** | [N] | `analytics_tests(kind="flaky", days=N)` | +| ⏭️ **Never Executed** | [N] | `analytics_tests(kind="never-executed", maturity_days=N)` | +| 🌿 **Evergreen (Stable)** | [N] | `analytics_tests(kind="evergreen", days=N)` | +| ⏭️ **Skipped Tests** | [N] | `analytics_tests(kind="skipped", days=N)` | +| 🐛 **Tests with Defects** | [N] | `analytics_tests(kind="defects", days=N)` | +| 🐌 **Slow Tests (>N ms)** | [N] | `analytics_tests(kind="slow", threshold_ms=N)` | + +### 8.2 Success Rate Trend +> `analytics_stats(kind="success-rate-by-date", from="DD.MM.YYYY", to="DD.MM.YYYY")` + +| Date | Success Rate % | +| :--- | :---: | +| [DD.MM.YYYY] | [XX%] | +| [DD.MM.YYYY] | [XX%] | +| [DD.MM.YYYY] | [XX%] | + +### 8.3 Execution Volume Trend +> `analytics_stats(kind="testruns-by-date", from="DD.MM.YYYY", to="DD.MM.YYYY")` + +| Date | Test Runs Executed | +| :--- | :---: | +| [DD.MM.YYYY] | [N] | +| [DD.MM.YYYY] | [N] | + +--- + +## 📊 9. Sprint Health & Project Readiness Assessment + +| Assessment Area | Risk Level | Details & Impact Narrative | +| :--- | :---: | :--- | +| 🟢 **Sprint Completion Risk** | `[Green / Yellow / Red]` | Narrative based on execution % and sign-off rate. | +| 📦 **Spillover / Scope Creep Risk** | `[None / Low / High]` | Highlight any items vulnerable to rolling over into the next cycle. | +| 🖥️ **Environment & CI/CD Stability** | `[Stable / Degraded]` | Log testing framework bottlenecks, env down times, or pipeline blockers. | + +**Sprint Summary Overview:** +[Write a summary 1-5 sentences overview by current sprint based on the provided info from this report.] \ No newline at end of file