Review and verification
Read how Ysra checked a Code task's change — your repository's own commands, a review against your criteria, independent reviewers and a real browser — and what happens when a check fails.
On this page
Verification is how Ysra proves a Code task’s change works before calling it done: your repository’s own commands, a reviewer that didn’t write the code, and a real browser for anything a person clicks. This page is for anyone deciding whether to keep a change: read the result before you commit, push or deploy. It covers each layer, what happens when a check fails, and how to read the report; how the task’s states follow from it is on How a Code thread works.
The rule#
Every result is tied to the exact version of the change it looked at:
evidence for one version can never approve another. Checks run when the work
needs them, when a plan step or a review calls for them, or when you ask for
them; a small change you asked for “without tests” runs none. Every ending
says what ran in one line, such as “Ran: npm test (passed).” See
How a task ends.
What is checked#
| Layer | Who | What it proves |
|---|---|---|
| Your checks | Your project’s own commands | Install, lint, type-check, test, build and migrations pass on this change |
| Acceptance review | A separate, read-only reviewer | Each acceptance criterion is proved, failed or unproved, with evidence |
| Independent review | A panel of read-only reviewers, chosen for the change | Correctness, your API, stored data, security, accessibility… |
| Browser proof | Executed browser scenarios | What a person does on the page actually works |
| Security floor | Automatic scan | No secrets written into code, no changes to protected paths |
The criteria themselves come from your brief: see Plans and acceptance criteria.
Results#
The side panel beside a thread has a Preview tab, a Changes tab (the diff) and a Report tab (each criterion with its evidence, your checks, the browser check and the credits used). See Review the diff.
| Result | Meaning |
|---|---|
| Passed | Proved on this exact version. |
| Failed | A concrete defect, with evidence. |
| Not applicable | The criterion doesn’t apply to this change. |
A criterion nothing was run for shows no result; the ending's receipt line says what ran.
What happens when …#
A check fails#
Only a concrete defect sends the work back, as a repair scoped to that defect, starting from the reviewed version and re-running only the checks the fix could affect. Ysra stops repairing when a fix no longer changes the result, rather than going round in circles.
If the project's checks still fail, the task ends Didn't work and says what failed. The change is kept; you can still use or deliver it.
A test was already failing#
A test that failed before the change is reported as pre-existing: neither blamed on Ysra nor quietly “fixed”, and it doesn’t block the change.
A test is flaky#
Changed tests may be repeated. A test that passes and fails on the same code is reported as flaky and never counted as proof.
A check changes the code#
If a build rewrites source files, earlier results no longer count and the new version is checked again.
A check can’t run#
A check that can’t run is a failure, not a pass. An infrastructure problem (a port, a missing sign-in, a service that didn’t start) never triggers a code repair: it is reported as the environment’s problem.
A repository with no tests#
Ysra checks the change itself: it runs the program, tries a sample input, opens the page, or writes a small test of its own, then names what it ran:
| What happened | What the thread says |
|---|---|
| Something ran and worked | “This repository has no tests, so I checked it myself: python cli.py --help worked.” |
| Something it ran failed | The same, plus “Look at what failed before relying on it.” |
| Nothing ran | “I made the change, but nothing ran to check it: this repository has no tests or checks I could run, and I didn’t run it myself. Try it before relying on it.” |
It never reports checks as passed when none ran.
You go back to a saved point#
Checks and reviews that ran after that point described different code, so they no longer count; the restored version is unchecked until they run again. See Steering and saved points.
You deliver a change that wasn’t fully checked#
Ysra never blocks: the commit, push or deploy goes through, and the receipt line says what ran and how it went. See Ysra warns, it never blocks.
The layers in detail#
Your checks#
- Only what applies. Commands are chosen by the parts of the repository the change touches, so a docs change doesn’t run the whole backend suite.
- Tests Ysra writes count as evidence, never as the only proof.
- For repositories with unusual layouts, required checks can be declared explicitly so they always run for the paths they cover. Write to support@ysra.ai to set this up.
Acceptance review#
A separate reviewer reads the exact change, your criteria, the files changed and deleted, and the evidence so far. It can’t edit anything. The rules it is held to:
- A pass can’t override a failing check.
- A pass must be about the criterion’s actual subject, not something nearby.
- Interactive behaviour can’t pass on source code alone: it needs browser evidence from a real run.
- “We couldn’t test this” is unproved: never a defect, never a pass.
- If the code changed during the review, the review doesn’t count.
Independent review#
Read-only reviewers look at the areas the change touches: a docs fix gets a light review, a checkout flow a full one. A defect is only reported against your change with evidence from it, the same defect is reported once, and suggestions outside your request stay advisory.
Browser proof#
When a criterion describes something a person does, Ysra:
- breaks it into concrete checks, such as an invalid date shows an error;
- writes browser scenarios from your actual pages and data;
- runs the app and executes each one: navigation, typing, clicking, reloading, desktop and mobile widths, signed-in flows;
- records every step: what it did, what the page showed, screenshots and console messages.
A written scenario is not proof; only an executed one is. Searches are tested with a term that should match and one that shouldn’t, and “it worked” means the action changed the page, not that the page already looked that way.
Security floor#
Every change is scanned for credentials and secrets written into code, and for changes to protected paths such as environment files, keys and repository internals. Findings name the rule and the file, never the secret. It is a baseline, not a penetration test: for a full review, use Mayz.
Before you deliver#
| Look at | Question |
|---|---|
| Repository checks | Did your own commands pass on this exact version? |
| Acceptance | Which criteria are met, and is anything unproved? |
| Changes | Does the diff do what you asked, and nothing else? |
To try the change yourself, see Try the change yourself, or a preview in the cloud. Nothing is committed or pushed without your OK; see Commit and push.
Related#
- How a Code thread works — the states a task ends in.
- Plans and acceptance criteria — where the criteria come from.
- Mayz reviews — an independent review of a whole project.
- Steering and saved points — going back, and what it does to checks.
- Walkthrough — a site whose look couldn’t be rendered, with the walkthrough saying so.