A Polygraph engine · the audit

polygraph — audit code that already exists, against reality

Most stateful code in production was never modeled at all. polygraph doesn't ask you to write a spec first — it generates independent specifications from the code itself, replays real traces against them, and model-checks whichever specification survives contact with reality.

The pipeline

What goes in, the stages it runs, what it leaves behind. Step through the stages to trace the run.

fig 7
polygraph
read the code back to itself, then check every reachable state
Your tests check the paths you thought of. This checks the ones you didn’t — over the finite domains the contract declares.
audit · the deterministic gate
01
Scope
contract.json names the observable state keys, the action alphabet, the data domains and the terminal states. That is the audit’s scope — and its limit.
→
02
Ground truth
Wrap the dispatch once and drive scenarios. Traces are {pre, action, data, post} windows from the code actually executing — never from expectations.
→
03
Generate ×N
An LLM derives N independent executable specs from the source. Several are generated and vote, so one bad generation cannot decide the outcome.
→
04
before trust
Controls
A hand-written reference spec must replay 100% and a deliberately mutated one must fail — proving the harness can tell good from bad before any generated spec means anything.
→
05
Replay + triage
Every window against every spec. Disagreements sort into spec-error (the LLM misread), code-finding (investigate) and contract-error (mis-scoped).
→
06
optional → TLC
Model check
The faithful spec is iterated exhaustively from init against your invariants. Every violation arrives with the shortest action path — a ready-made repro.
consumes
source (any language) · contract.json · traces/ · invariants.mjs
produces
specs 1..N · replay scores + triage · findings with shortest counterexamples
Replay finds unfaithfulness; the model check finds bugs. A faithful spec reproduces the bug right along with the code, so replay alone cannot see it.

How it works

You don't need a spec to start. polygraph builds and tests candidates for you.

1

Generate independent specs

Multiple candidate models of what the code does, derived from the code, not from what the comments or the original author claimed.

2

Replay real traces

Actual execution history is run against every candidate spec; the ones that don't match reality are discarded.

3

Model-check the survivor

The specification that matches reality is then checked exhaustively, surfacing the reachable states nobody exercised in production yet.

Fits with: The plugin to run first on a codebase you didn't write, before deciding whether polynv, polygen, or polyvers apply.