Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 26 additions & 21 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Nuxt file-based routing:
- `/test-cases/[id]` — Case detail across runs (pass rate + duration stats, duration trend, status-history strip, recent executions, failure clusters)
- `/test-run-cases/[id]` — Single execution detail, **diagnosis-first**. A failing execution opens on a **Diagnosis** tab modeled on the cluster page: full-width error card, then a `grid grid-cols-1 xl:grid-cols-[3fr_2fr]` with a right rail (`TestCaseVerdictCard`, `FailureClusterCard`, `TestCaseAiCard`; `xl:order-2` so it follows the error on mobile) and a left evidence funnel (`TestCaseEvidenceCard`, `LocatorHealingPanel`, `EnvironmentDiffCard`, `VisualDiffCard`, console/network, `PageStateCard`, ARIA, `DomSnapshotCard`). A passing execution opens on **Steps** with an **Artifacts** tab. Both keep **Performance** (hints + Web Vitals, always enabled) and **History** (always enabled). Tab is synced to `?tab=` with legacy aliases (`error`→`diagnosis`, `traces`→`diagnosis`/`artifacts`); the page `provide`s `clusterSectionLocatorKey` so an AI-diagnosis citation can reveal/scroll to the matching evidence section.
- **Components** (`app/components/`): organized into domain subfolders; all auto-imported without folder prefix (Nuxt `pathPrefix: false`).
- **`shared/`** — UI primitives used across multiple pages: `RunStatusBadge`, `TestStatusBar`, `RunReports`, `TagBadge`, `TagsSelect`, `BrowserBadge`, `CiEnvCard`, `SourceInfoCard`, `OpenInIdeLink` (clickable source path/`file:line` → open in the local IDE; see the "Clickable source paths" rule), `CodeBlock`, `MarkdownPreview`, `ScreenshotLightbox`, `LocatorHealingPanel` (fetches `/api/test-runs/:id/cases/:caseId/locator-healing`; renders ranked alternatives + recommended fix; used on the test-case and cluster pages)
- **`shared/`** — UI primitives used across multiple pages: `RunStatusBadge`, `TestStatusBar`, `RunReports`, `TagBadge`, `TagsSelect`, `BrowserBadge`, `CiEnvCard` (CI provider/build/workflow + environment + tooling versions; pass an optional `browser` to fold in a dense browser line — icon + screen resolution, full config in the `BrowserBadge` tooltip), `SourceInfoCard`, `OpenInIdeLink` (clickable source path/`file:line` → open in the local IDE; see the "Clickable source paths" rule), `CodeBlock`, `MarkdownPreview`, `ScreenshotLightbox`, `LocatorHealingPanel` (fetches `/api/test-runs/:id/cases/:caseId/locator-healing`; renders ranked alternatives + recommended fix; used on the test-case and cluster pages)
- **`run/`** — Test run detail page (`/test-runs/[id]`): `RunSummary` (summary card + CI/Source/Other metadata), `TestCasesList` (paginated table, sticky headers, row highlighting via `meta.class.tr`), `WorkersTimeline` (clickable bars, emits `selectTestCase`), `RunCompare` (self-contained, watches internal `compareRunA`), `SlowEndpoints` (fetches `/api/test-runs/:id/network-requests` internally), `FailureGroups`, `RunReports`
- **`test-case/`** — Single-execution detail page (`/test-run-cases/[id]`): `TestCaseSummary` (pinned summary: status, location, duration, retries, worker, slowest step, timing vs avg, signal badges, annotation chips, wasted-time), `TestCaseVerdictCard` (regression/flaky/retry chips + clickable recent-runs strip + failing-streak verdict), `FailureClusterCard` (signature/error-type/occurrences + cluster AI verdict + hand-off; evolved from the old `FailureClusterBanner`), `TestCaseAiCard` (execution-scoped **Copy AI context** + **Diagnose with AI**, renders `DiagnosisResult` inline via the execution `/diagnose` + `/diagnosis` endpoints), `TestCaseEvidenceCard` (grouped screenshots/videos/traces/attachments with a folded peek), `TestCaseTracesCard`, `TestCaseConsoleCard`, `TestCaseNetworkRequests`, `TestCaseAttachmentsCard`, `PageStateCard`, `DomSnapshotCard`, `TestCaseHistoryChart`, `TestEvidenceSection`, `TestEvidenceScreenshots`, `TestEvidenceVideos`, `TestEvidenceSignals`, `TestEvidenceTraces`
- **`cluster/`** — Failure cluster detail (`/failure-clusters/[id]`): `ClusterSummary` (top summary: signature/metadata info card + inline Triage card, wrapped in `FoldableSummary`), `ClusterTestEvidence` (one tab per affected case, each with an "Open test run case" link, an evidence chip strip and reordered screenshots/traces/failing-steps/signals/source/ARIA blocks; caches each case's detail), `ClusterInvestigation` (baseline picker + SCM diff; its status line is derived from the shared `useScmStatusSummary` composable), `CommitPicker` (baseline selector with aggregate diff stats, caches per baseline), `CommitBrowserModal` (full-screen split modal: paginated commit list + per-commit diff), `ClusterExtractCasesModal`, `RegressionContext`. The page wraps the left-column sections (error, alternative locators, test evidence, what changed) in `CollapsibleSectionCard` (folded by default with a peek) and `provide`s a `clusterSectionLocator` (`useClusterSectionLocator`) so a diagnosis citation can unfold + scroll to the matching section.
Expand Down Expand Up @@ -271,7 +271,8 @@ Nuxt file-based routing:
| `npm run db:migrate` | Apply migrations |
| `npm run db:push` | Push schema (dev only) |
| `npm run db:studio` | Drizzle Studio |
| `npm run app:seed:demo` | Regenerate demo seed data |
| `npm run app:seed:demo` | Regenerate demo seed data (`public/demo/seed.sql`) |
| `npm run app:seed:dev` | Load the demo sample data into the local dev SQLite DB |
| `npm run app:generate:demo` | Build demo SPA |
| `npm run app:check:demo` | Verify demo routes |
| `node scripts/db-query.mjs "<sql>"` | Query the local SQLite DB directly |
Expand All @@ -282,6 +283,26 @@ node scripts/db-query.mjs "SELECT key, value FROM app_settings"
node scripts/db-query.mjs "SELECT id, name FROM projects" --json
```

### Running the app locally with sample data (verification & screenshots)

**When you need to see a change working in the real app** — verify a UI tweak, drive a flow, or capture a screenshot — run a **plain (non-demo) dev server backed by a dev DB seeded from the demo data**. Do this instead of hunting for a way to boot the app each time.

```bash
cd application
npm run app:seed:demo # 1. generate public/demo/seed.sql (skip if it already exists)
mkdir -p .data && npm run db:migrate # 2. create + migrate an empty dev DB (.data/piwi.db)
npm run app:seed:dev # 3. load the sample data (server must be stopped — DB lock)
NUXT_IGNORE_LOCK=1 npx nuxt dev --port 3002 # 4. plain dev server (auth disabled by default)
```

Then drive `http://localhost:3002` with Playwright using the pre-installed Chromium at `/opt/pw-browsers/chromium` (see `scripts/take-demo-screenshots.mjs` for a working harness). `app:seed:dev` is idempotent (`INSERT OR IGNORE`), so re-running it is safe; to refresh stale rows, wipe `.data` and repeat from step 2.

**Caveats (these cost real debugging time — read them first):**
- **Do NOT use `PIWI_DEMO_MODE=true` for local verification.** The demo runs entirely in-browser via a service worker + WASM SQLite that does **not** install in headless Chromium, so pages render blank. Demo mode is only for building the static demo SPA.
- **Test cases live under runs #21+.** Runs #1–20 have 0 cases (their rows target a migration-only table that the dev schema drops), so a test-run-case URL for those runs renders empty. Query the DB (`node scripts/db-query.mjs`) for a real `test_runs_cases.id` — e.g. `SELECT id FROM test_runs_cases ORDER BY id DESC LIMIT 5`.
- **Clusters with data:** #3, #4, #5, #7, #8.
- **Brand icons** (`i-simple-icons-*`, e.g. browser badges) resolve from the iconify CDN at runtime; with no outbound network they render blank (only the `lucide` collection is bundled locally). This is an environment limitation, not a bug.

### Reporter commands (from `reporter/`)

| Command | Purpose |
Expand Down Expand Up @@ -486,31 +507,15 @@ When implementing or extending sharding support:

Demo screenshots live in `application/public/demo/screenshots/*.png` and are committed to the repo. They appear as attachment thumbnails on cluster detail pages. To replace them with fresh real-app captures:

1. **Start a plain dev server** (no `PIWI_DEMO_MODE`) on a free port:
```bash
cd application
NUXT_IGNORE_LOCK=1 npx nuxt dev --port 3002
```
Auth is disabled by default — no login needed.

2. **Seed the dev DB** from the demo SQL (stop the server first to avoid a DB lock, or use a separate terminal while the server is running if it's not writing):
```bash
node scripts/seed-dev-from-demo.mjs
```
This reads `public/demo/seed.sql` and runs `INSERT OR IGNORE` into `.data/piwi.db`. It is idempotent.

3. **Restart the dev server** (it may have a stale DB on first boot):
```bash
NUXT_IGNORE_LOCK=1 npx nuxt dev --port 3002
```
1. **Seed a dev server** exactly as in [Running the app locally with sample data](#running-the-app-locally-with-sample-data-verification--screenshots) above (`npm run app:seed:dev` into a migrated `.data/piwi.db`, then `NUXT_IGNORE_LOCK=1 npx nuxt dev --port 3002`).

4. **Capture screenshots** with Playwright:
2. **Capture screenshots** with Playwright:
```bash
node scripts/take-demo-screenshots.mjs
```
This writes `public/demo/screenshots/*.png` directly. The script targets `localhost:3002` and uses the Chromium at `/opt/pw-browsers/chromium`.

5. **Commit the new PNGs** — they are committed to the repo so the demo SPA can serve them.
3. **Commit the new PNGs** — they are committed to the repo so the demo SPA can serve them.

### Demo trace + video (committed binary evidence)

Expand Down
13 changes: 13 additions & 0 deletions application/app/components/shared/CiEnvCard.vue
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
<script setup lang="ts">
import type { TestRunCiMetadata } from '~~/types/api';
import type { BrowserConfig } from '#shared/types';

defineProps<{
ci?: TestRunCiMetadata | null;
environment?: string | null;
playwrightVersion?: string | null;
reporterVersion?: string | null;
/** When provided, a dense browser line (icon + screen resolution) is folded in. */
browser?: BrowserConfig | null;
class?: string;
}>();
</script>
Expand Down Expand Up @@ -43,6 +46,16 @@ defineProps<{
<span v-if="playwrightVersion && reporterVersion" class="text-gray-300 dark:text-gray-600">·</span>
<span v-if="reporterVersion">Piwi v{{ reporterVersion }}</span>
</div>
<!-- Browser: dense (icon + screen resolution); full config on hover via the badge -->
<div v-if="browser" class="flex items-center gap-1.5">
<BrowserBadge :browser="{ ...browser, viewport: undefined }" size="sm" />
<span v-if="browser.viewport" class="tabular-nums">
{{ browser.viewport.width }}×{{ browser.viewport.height }}
<span v-if="browser.deviceScaleFactor && browser.deviceScaleFactor !== 1" class="text-gray-400"
>@{{ browser.deviceScaleFactor }}x</span
>
</span>
</div>
</div>
</BlockCard>
</template>
78 changes: 9 additions & 69 deletions application/app/components/test-case/TestCaseSummary.vue
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,7 @@ const startedAtMs = computed<number | null>(() => props.testCase?.startedAt ?? n
const { summaryColSpanClass, blockColSpanClass } = useDetailGrid(() => {
let count = 0;
if (props.scmInfo) count++;
if (props.ciInfo || props.environment) count++;
if (props.browser) count++;
if (props.ciInfo || props.environment || props.browser) count++; // CI / Env / Browser (merged)
count++; // Links card always visible
return count;
});
Expand Down Expand Up @@ -179,73 +178,14 @@ const { summaryColSpanClass, blockColSpanClass } = useDetailGrid(() => {
<!-- Source -->
<SourceInfoCard v-if="scmInfo" :scm="scmInfo" :class="blockColSpanClass" />

<!-- CI / Env -->
<CiEnvCard v-if="ciInfo || environment" :ci="ciInfo" :environment="environment" :class="blockColSpanClass" />

<!-- Browser -->
<BlockCard v-if="browser" :class="blockColSpanClass" title="Browser" icon="i-lucide-globe">
<div class="space-y-2">
<div class="flex items-center gap-2 flex-wrap">
<BrowserBadge :browser="browser ? { ...browser, viewport: undefined } : null" size="md" />
<span v-if="browser?.channel" class="text-xs text-gray-500">{{ browser.channel }}</span>
</div>
<div class="grid grid-cols-2 gap-x-4 gap-y-1 text-xs text-gray-600 dark:text-gray-400">
<div v-if="browser?.viewport" class="flex items-center gap-1">
<UIcon name="i-lucide-maximize-2" class="size-3 shrink-0 text-gray-400" />
{{ browser.viewport.width }}×{{ browser.viewport.height }}
<span v-if="browser.deviceScaleFactor && browser.deviceScaleFactor !== 1" class="text-gray-400"
>@{{ browser.deviceScaleFactor }}x</span
>
</div>
<div v-if="browser?.isMobile" class="flex items-center gap-1">
<UIcon name="i-lucide-smartphone" class="size-3 shrink-0 text-gray-400" />
Mobile
<span v-if="browser.hasTouch" class="text-gray-400">· Touch</span>
</div>
<div v-else-if="browser?.hasTouch" class="flex items-center gap-1">
<UIcon name="i-lucide-hand" class="size-3 shrink-0 text-gray-400" />
Touch
</div>
<div v-if="browser?.locale" class="flex items-center gap-1">
<UIcon name="i-lucide-languages" class="size-3 shrink-0 text-gray-400" />
{{ browser.locale }}
</div>
<div v-if="browser?.timezoneId" class="flex items-center gap-1">
<UIcon name="i-lucide-clock" class="size-3 shrink-0 text-gray-400" />
{{ browser.timezoneId }}
</div>
<div v-if="browser?.colorScheme && browser.colorScheme !== 'light'" class="flex items-center gap-1">
<UIcon name="i-lucide-moon" class="size-3 shrink-0 text-gray-400" />
{{ browser.colorScheme }}
</div>
<div
v-if="browser?.reducedMotion && browser.reducedMotion !== 'no-preference'"
class="flex items-center gap-1"
>
<UIcon name="i-lucide-pause-circle" class="size-3 shrink-0 text-gray-400" />
Reduced motion
</div>
<div v-if="browser?.offline" class="flex items-center gap-1">
<UIcon name="i-lucide-wifi-off" class="size-3 shrink-0 text-gray-400" />
Offline
</div>
<div v-if="browser?.javaScriptEnabled === false" class="flex items-center gap-1">
<UIcon name="i-lucide-code-xml" class="size-3 shrink-0 text-gray-400" />
JS disabled
</div>
<div v-if="browser?.geolocation" class="flex items-center gap-1">
<UIcon name="i-lucide-map-pin" class="size-3 shrink-0 text-gray-400" />
{{ browser.geolocation.latitude.toFixed(2) }}, {{ browser.geolocation.longitude.toFixed(2) }}
</div>
</div>
<div
v-if="browser?.userAgent"
class="text-xs text-gray-400 break-all leading-snug pt-1 border-t border-default"
>
{{ browser.userAgent }}
</div>
</div>
</BlockCard>
<!-- CI / Env / Browser -->
<CiEnvCard
v-if="ciInfo || environment || browser"
:ci="ciInfo"
:environment="environment"
:browser="browser"
:class="blockColSpanClass"
/>

<!-- Links -->
<BlockCard :class="blockColSpanClass" title="Links" icon="i-lucide-link">
Expand Down
1 change: 1 addition & 0 deletions application/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
"app:generate": "nuxt generate",
"app:generate:demo": "cross-env PIWI_DEMO_MODE=true nuxt generate",
"app:seed:demo": "node scripts/generate-demo-seed.mjs",
"app:seed:dev": "node scripts/seed-dev-from-demo.mjs",
"app:build": "nuxt build",
"app:dev": "nuxt dev",
"app:preview": "nuxt preview",
Expand Down
4 changes: 2 additions & 2 deletions application/public/demo/seed.version.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"hash": "346138c33050734010aa8a4b35a290440f08825d5866dc6128e348d246246758",
"generatedAt": "2026-07-15T21:28:45.695Z"
"hash": "957dc11f122aeb81e4bfc2d97b8582b12ee84f9c6a8ed59c9479c2b580cd66a2",
"generatedAt": "2026-07-16T19:52:23.939Z"
}
85 changes: 76 additions & 9 deletions application/scripts/generate-demo-seed.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -720,24 +720,91 @@ const PROJECT_CONFIGS = {
4: { numRuns: 8, totalTests: 7, baseDuration: 190000, baseAvg: 5800, baseP90: 12000, failRate: 0.25, flakyRate: 0.1 },
};

// Browser configs per project
// Browser configs per project. These carry the richer context fields
// (deviceScaleFactor, isMobile/hasTouch, locale, timezone, colorScheme, userAgent)
// so the browser badge tooltip and the CI / Env card's dense browser line have
// real content to show.
const CHROME_UA =
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36';
const BROWSER_CONFIGS = {
1: [
// e2e-checkout — Chromium most runs, Firefox occasionally
{ projectName: 'Chromium', browserName: 'chromium', channel: null, viewport: { width: 1280, height: 720 } },
{ projectName: 'Firefox', browserName: 'firefox', channel: null, viewport: { width: 1280, height: 720 } },
{
projectName: 'Chromium',
browserName: 'chromium',
channel: 'chrome',
viewport: { width: 1280, height: 720 },
deviceScaleFactor: 1,
isMobile: false,
hasTouch: false,
locale: 'en-US',
timezoneId: 'America/New_York',
colorScheme: 'light',
userAgent: CHROME_UA,
},
{
projectName: 'Firefox',
browserName: 'firefox',
channel: null,
viewport: { width: 1280, height: 720 },
deviceScaleFactor: 1,
isMobile: false,
hasTouch: false,
locale: 'en-US',
timezoneId: 'America/New_York',
colorScheme: 'light',
userAgent: 'Mozilla/5.0 (X11; Linux x86_64; rv:125.0) Gecko/20100101 Firefox/125.0',
},
],
2: [
// api-integration — always Chromium
{ projectName: 'Chromium', browserName: 'chromium', channel: null, viewport: { width: 1280, height: 720 } },
// api-integration — headless Chromium in CI
{
projectName: 'Chromium',
browserName: 'chromium',
channel: null,
viewport: { width: 1280, height: 720 },
deviceScaleFactor: 1,
isMobile: false,
hasTouch: false,
locale: 'en-US',
timezoneId: 'UTC',
colorScheme: 'light',
userAgent:
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/124.0.0.0 Safari/537.36',
},
],
3: [
// ui-components — always Chromium
{ projectName: 'Chromium', browserName: 'chromium', channel: null, viewport: { width: 1280, height: 720 } },
// ui-components — Chromium at 2x DPR in dark mode (component visual tests)
{
projectName: 'Chromium',
browserName: 'chromium',
channel: null,
viewport: { width: 1280, height: 720 },
deviceScaleFactor: 2,
isMobile: false,
hasTouch: false,
locale: 'en-GB',
timezoneId: 'Europe/London',
colorScheme: 'dark',
userAgent: CHROME_UA,
},
],
4: [
// mobile-safari — always WebKit with iPhone viewport
{ projectName: 'Mobile Safari', browserName: 'webkit', channel: null, viewport: { width: 390, height: 844 } },
// mobile-safari — WebKit emulating an iPhone (mobile + touch + 3x DPR)
{
projectName: 'Mobile Safari',
browserName: 'webkit',
channel: null,
viewport: { width: 390, height: 844 },
deviceScaleFactor: 3,
isMobile: true,
hasTouch: true,
locale: 'en-US',
timezoneId: 'America/Los_Angeles',
colorScheme: 'light',
userAgent:
'Mozilla/5.0 (iPhone; CPU iPhone OS 17_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Mobile/15E148 Safari/604.1',
},
],
};

Expand Down
Loading