Most changes get a quick glance at best. The subtle bugs, a wrong guard, an off-by-one, a missed case, slip through and surface later, when they cost more to fix.
polywatch catches them without slowing Claude down. It reviews every change in the background, has Claude Opus 5.5 check the most serious claims against the code, and shows you at most two findings on your next prompt. Confirmed bugs go back to Claude to fix.
A second model reading every change is cheap. The hard part is what to do with what it says. We tried the two obvious answers first, and both failed.
We replayed 50 commits from a real project. 25 were later fixed; 25 never were. The reviewer rejected 24 of the first group and all 25 of the second. A trust score built on its verdict separated them no better than chance (AUC 0.48). So polywatch ignores the verdict. It is not a gate.
What the reviewer does produce is specific claims, and some are real bugs: in 7 of the 25 later-fixed commits, the bug the fix repaired was already on its list. But it was one of 3.2 issues per commit, with nothing to say which. A list nobody reads catches nothing.
polywatch's job is to put those few real bugs in front of you and keep the rest out of the way. A careful model checks each serious claim against the code and throws out what does not hold, and a ranking you train decides what is worth a look.
polywatch is a Claude Code plugin: four hooks and a background worker. Nothing runs inside the turn except writing a small file, so Claude never waits.
A PostToolUse hook notes every Edit, Write and MultiEdit. When Claude finishes answering your prompt, the Stop hook snapshots the turn with your request and starts a background worker.
DeepSeek V4.1-Flash, from another model family, reads the files Claude edited and lists specific claims: a file, a line or function, and a statement someone can check. A deterministic router labels the change EASY, MEDIUM or HARD; on HARD changes your testCommand runs, if you set one.
Claude Opus 5.5 checks the two most serious claims, one at a time, against the cited code and, when it is short enough, your request. A claim the code does not bear out, or whose fix the request rules out, is refuted and never shown.
Each finding is scored by how often findings of its kind (confirmed or not, by severity) turned out to be real. Kinds below the threshold are hidden, so unconfirmed medium and low claims stay out of sight until your outcomes say they are worth showing.
On your next prompt, the UserPromptSubmit hook shows you the top two. Only confirmed findings go to Claude, with the instruction to verify each one and fix those that hold, at most two rounds in a row.
You mark a finding real or false. Each outcome moves the score for its kind, in a ledger that stays in your project.
On your next prompt, a short block. This is a real one, from the Raft example below:
When nothing survives, it says polywatch: nothing worth your attention.
No wall of warnings. Weaker findings are counted, not listed, and refuted claims are counted and dropped.
Each finding says whether Opus confirmed it, and why. Claims that do not hold are gone before you see anything.
With your next prompt, Claude receives the confirmed findings and Opus's evidence, and is asked to verify each one against the code, fix the ones that hold and say why any do not.
polywatch outcome <id> <n> real|false records whether a finding was a real bug. polywatch calibration shows how often each kind has been real so far.
Node.js 20 or later and Claude Code. No dependencies to install.
In your environment, not in the repo. Without an Anthropic key the reviewer still runs, but nothing is confirmed and nothing goes back to Claude.
# macOS / Linux export DEEPSEEK_API_KEY="sk-..." # reviewer export ANTHROPIC_API_KEY="sk-ant-..." # Opus confirmation # Windows setx DEEPSEEK_API_KEY "sk-..." setx ANTHROPIC_API_KEY "sk-ant-..."
git clone https://github.com/cognitive-fab/polywatch # try it for one session claude --plugin-dir ./polywatch # or install it claude plugin marketplace add ./polywatch claude plugin install polywatch@polywatch-local
Ask Claude for a change. The review starts when Claude finishes, and the findings arrive with your next prompt. Prompts that change fewer than three lines are not reviewed.
# recent reviews, ranked findings and Opus's evidence (or ask Claude for /trust) polywatch report # was finding n of a review a real bug? this is what trains the ranking polywatch outcome <id> 1 real polywatch outcome <id> 2 false "intended: the lock is reentrant" # review files by hand, outside a session polywatch review src/lock.js --task "make release check the holder"
.polywatch.json in the project root. Every field is optional.
{
"budgetUsdPerTurn": 0.5, // hard stop per prompt
"confirm": "top", // which claims Opus checks: top | high | all | none
"maxFindings": 2,
"minScore": 0.25,
"adjudicator": { "maxClaims": 2, "includeRequest": "short" },
"exclude": ["secrets/**"]
}
Comments shown for explanation; the file itself is plain JSON. "confirm": "high" roughly halves the confirmation cost.
A project file travels with the repository, so it cannot choose which command runs or where your API keys are sent. testCommand, baseUrl and apiKeyEnv are read from ~/.polywatch.json, or from a project you list there under trustedProjects:
// ~/.polywatch.json { "trustedProjects": ["/home/me/code/my-service"] }
exclude are never sent; .env, keys and anything named like a secret or credential are excluded out of the box. For code that must not leave your control, set reviewer.provider to anthropic, or point baseUrl at a local OpenAI-compatible server. polywatch's own state, which holds prompts and file contents, is git-ignored automatically.
Real runs on the example files that ship with polywatch, with the default settings. The files carry no comments pointing at a defect. Costs are what the calls billed.
export function release(state, thread) { if (state.held) { // lock-bug.js if (state.held && state.holder === thread) { // lock-fixed.js return { held: false, holder: null }; } return state; }
On the buggy file, Opus confirms the reviewer's claim, and that one finding is all you see:
polywatch: 1 finding worth a look (1 confirmed) · MEDIUM · $0.0059
1. [confirmed, high] lock-bug.js release(): release() does not verify that thread equals
state.holder; any thread can release the lock even when another thread holds it […]
On the fixed file the reviewer still objects twice: acquire returns the state unchanged instead of spinning, and two calls could race. Opus refutes both, because this is “a pure state-transition model” where returning the state unchanged models a blocked step. You see nothing:
polywatch: nothing worth your attention · MEDIUM · 2 refuted · $0.0212
A vote handler simplified from etcd's raft.go, with a planted bug: the up-to-date check compares log lengths and ignores the candidate's last log term. polywatch shows two confirmed findings and hides two weaker ones (the block under What you get). The first is the planted bug. The second is a real departure from etcd that nobody planted: a heartbeat from a higher term never advances the node's term. Opus's evidence, which Claude receives with each finding:
evidence: The heartbeat branch only sets `node.leader = msg.from; node.electionElapsed = 0;`
and never updates `node.term`, `node.role` or `node.votedFor`. A later vote request with
node.term < msg.term < heartbeat term therefore passes the `msg.term > node.term` branch,
which clears `leader` and `votedFor`, and the vote is granted at a term lower than the
heartbeat's.
Fixing the vote rule alone does not make the file correct. On the partly fixed version, the stale heartbeat and a second departure from etcd (a denied vote in a higher term does not reset the election timer) are confirmed, for $0.038.
We planted bugs in specifications of real systems from The price of trust: an Asterinas spinlock, a lock service and etcd Raft. There, a model checker tells us which specifications are correct, so every finding can be scored. DeepSeek V4-Pro planted one behavioral bug in each of 173 verified specifications.
We replayed the /code-review method on 97 of the same specifications, with the plugin's own prompts and rubric: Sonnet 5 review agents, a Haiku 4.5 scorer rating each issue 0 to 100, and issues scored 80 or more reported. One judge, shown the planted change, scored both.
| 97 specs | polywatch | /code-review |
|---|---|---|
| Planted bugs the checker catches, reported | 39 of 40 | 25 of 40 |
| … of which on etcd Raft | 9 of 10 | 2 of 10 |
| Planted bugs the checker misses, reported | 20 of 20 | 17 of 20 |
| Correct specs with something reported | 7 of 37, all etcd | 2 of 37 |
| Cost per spec | $0.015 | $0.117 |
| When it runs | every prompt | when you remember to ask |
Both reviewers raised almost every planted bug. The difference is the filter. /code-review's scorer gave 15 real planted bugs a 75, often while saying the issue was real, and the threshold of 80 dropped them. polywatch's Opus check kept them, at about an eighth of the cost. /code-review did raise fewer false alarms on the hardest system, etcd Raft.
A replay, not the plugin itself: /code-review reviews GitHub pull requests with five agents, and three of them need a CLAUDE.md, git history or earlier pull requests that a new spec file does not have. The replay ran the two that apply.
polywatch reviews once per prompt: all the files Claude edited in answer to it. The cost depends on how much code it has to read.
| Code | Per prompt |
|---|---|
| Small files, such as a lock service | under $0.01 |
| A large specification, such as etcd Raft | about $0.05 |
| A production codebase with large files | about $0.10 |
| Ceiling, whatever the change | $0.50 |
| Prompts that change fewer than 3 lines | not reviewed |
Confirmation is most of it: the review costs a fraction of a cent to two cents, and each claim Opus checks about two to five. At 30 prompts a day on a production codebase, expect about three dollars. "confirm": "high" or "maxClaims": 1 brings that down.
It runs in the background and costs cents, so there is no reason to save it for special occasions.
A reviewer judges code against what it was supposed to do. A precise request, tests or a specification make that possible. Without one, correct code can look wrong: an earlier run on the fixed lock raised a question about reentrancy, and Opus confirmed it, because nothing said whether the lock should be reentrant.
On etcd Raft, polywatch confirmed every planted bug, but correct and broken specifications got confirmed findings at about the same rate (4 of 15 against 6 of 20, with the request in the confirmation prompt). It finds local bugs; it does not find what makes a protocol fail its traces. That is what polygraph is for.
polywatch 0.2.0 is new. The ranking starts from guesses and means little until your ledger holds a few dozen outcomes.