automate-manual-test-cases: investigate page, run tests, ask for missing secrets - #150
Open
DavertMik wants to merge 1 commit into
Open
automate-manual-test-cases: investigate page, run tests, ask for missing secrets#150DavertMik wants to merge 1 commit into
DavertMik wants to merge 1 commit into
Conversation
…ing secrets Close three gaps that let the agent ship unverified automation: - Add Step 1.4 "Investigate the Live Target Page" — open the page and read the real DOM before writing selectors; never invent them from docs or a different page. - Make Step 4.1 a three-state outcome (passed/failed/blocked). A run that errors before executing (missing env, build/init) is blocked, not done, and must be the headline, not a footnote. - Treat a missing credential/env var as a blocker: STOP and ask the user; never assume a secret. Carve this out of the "minimize questions" bias. - Fix examples that modeled premature "Test passed!/All done!" and add a "Blocked by a Missing Credential" example. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DavertMik
force-pushed
the
fix/automate-investigate-run-ask
branch
from
July 5, 2026 22:23
a1bf729 to
b6b4108
Compare
olexandr13
approved these changes
Jul 22, 2026
|
|
||
| #### 1.4 Investigate the Live Target Page | ||
|
|
||
| **Never write UI automation from documents and code alone.** Locators invented from a manual test case, a description, or a *different* page are guesses — they will not run. Before writing selectors for a page, open that page and read its real DOM. |
Contributor
There was a problem hiding this comment.
I don't like this rule...
I think there are cases where generating autotests without actually running app is ok
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes three gaps that let the agent ship unverified automation (guessed selectors, code that never ran, silent about a missing token).
Changes
Test passed!/All done!; added a "Blocked by a Missing Credential" example.🤖 Generated with Claude Code