Invariant — architecture
An autonomous adversarial QA agent. A reasoning loop (Claude) drives a real browser (Playwright) against a target, infers business-logic invariants, attacks them, and on a contradiction emits a reproducible regression test. Scoped for demo impact and reliability — not enterprise breadth.
System at a glance
Three processes, one event stream. The browser stays open as a session; the agent loop reads its state, asks Claude what to try next, and pushes every thought to the UI over a WebSocket.
┌─────────────┐ WS /audit/stream ┌──────────────────────────────┐
│ Next.js UI │ ◀───── events ─────── │ FastAPI core │
│ dashboard │ ─── POST /audit ───▶ │ AuditOrchestrator (per run) │
└─────────────┘ │ ├─ PlaywrightSession (browser) │
│ ├─ ClaudeReasoner (SDK) │
│ └─ InvariantStore (state) │
└──────────────┬───────────────┘
│ read/write
┌─────▼─────┐
│ SQLite │ runs · findings · tests
└───────────┘Stack & responsibilities
Frontend
Next.js 15 App Router, TypeScript, Tailwind + shadcn/ui, Framer Motion for the feed/finding transitions.
- Subscribes to the run WebSocket
- Renders feed, browser captures, findings, tests
- Zero business logic — pure projection of server events
Backend
FastAPI orchestrates one AuditOrchestrator per run, streaming Pydantic events.
- Playwright drives a headless Chromium session
- Anthropic SDK for inference + judging
- SQLite persists runs, findings, generated tests
Folder structure
apps/web/ # Next.js 15 dashboard
app/
page.tsx # landing
dashboard/page.tsx # live audit workspace
components/
feed/AgentFeed.tsx FindingDrawer.tsx
browser/BrowserPreview.tsx Filmstrip.tsx
tests/CodeViewer.tsx ui/* # shadcn
lib/useAuditStream.ts # WebSocket hook
store/audit.ts # Zustand store
services/agent/ # FastAPI backend
main.py # routes + WS
orchestrator.py # AuditOrchestrator: the loop
browser/session.py # PlaywrightSession + DOM digest
reasoning/
reasoner.py # Claude calls
prompts/ rules.md attack.md judge.md
invariants/store.py models.py # Pydantic
testgen/playwright_writer.py
db.py schema.sql targets/shop/ # vulnerable demo appAPI routes
| Method | Path | Purpose |
|---|---|---|
| POST | /audit | Start a run for a target URL → returns run_id. |
| WS | /audit/{run_id}/stream | Live event stream: reasoning, actions, captures, findings, tests. |
| GET | /audit/{run_id} | Full run report (findings + tests) for replay/hydration. |
| GET | /audits | History list for the sidebar. |
| GET | /findings/{id}/test | Download the generated .spec.ts file. |
| POST | /audit/{run_id}/stop | Cancel a run and close the browser session. |
Database schema
SQLite, three tables. Everything the UI shows is reconstructable from here, so a refresh mid-demo loses nothing.
CREATE TABLE runs ( id TEXT PRIMARY KEY, -- "#A-1042" target_url TEXT NOT NULL, status TEXT, -- running|done|stopped phase TEXT, -- explore|hypothesize|attack|report started_at TIMESTAMP, runtime_ms INTEGER, invariants JSON -- inferred rules + status ); CREATE TABLE findings ( id TEXT PRIMARY KEY, run_id TEXT REFERENCES runs(id), invariant_id TEXT, severity TEXT, title TEXT, expected TEXT, actual TEXT, impact TEXT, steps JSON, screenshots JSON, action_log JSON ); CREATE TABLE tests ( finding_id TEXT REFERENCES findings(id), filename TEXT, code TEXT, framework TEXT DEFAULT 'playwright' );
Agent event loop
The orchestrator runs a bounded loop. Each iteration is one probe; every step emits an event the UI renders verbatim.
Claude prompt architecture
Three narrow, structured prompts — each returns JSON validated by a Pydantic model. Small surfaces = reliable demos.
1 · rules.md Invariant inference
In: DOM digest + screenshot. Out: Invariant[] — { id, statement, predicate, surface }. “List rules a correct checkout must never violate.”
2 · attack.md Adversarial planner
In: one invariant + current DOM. Out: ActionPlan — ordered Playwright steps + the expected-vs-watch assertion.
3 · judge.md Contradiction judge
In: before/after state + plan. Out: Verdict — { violated, severity, actual, impact }. Rejects noise before it becomes a finding.
Tooling Structured output
Each call uses a tool/JSON schema so the orchestrator never parses prose. Temperature low; the browser is the source of truth.
Playwright execution
One long-lived session per run. The agent speaks an intent vocabulary; the session translates to Playwright and records everything.
class PlaywrightSession:
async def digest(self) -> DomDigest: # roles, forms, routes, testids
async def run(self, plan: ActionPlan) -> ActionLog:
for step in plan.steps: # fill / click / goto / reload
await self._apply(step)
self.log.append(step, screenshot=await self.shot())
return self.log # replayed verbatim into the testRegression test generation
No LLM guesswork in the output: the recorded ActionLog+ the invariant's predicate template-render into a deterministic spec. The bug becomes a permanent guardrail.
def write(finding, log) -> str:
return TEMPLATE.render(
title = finding.title,
steps = [step.as_playwright() for step in log],
assert_ = finding.invariant.as_expect(), # expected, not actual
)Frontend state & events
A single Zustand store is the projection of the server event stream — the same model this prototype runs on.
type AuditEvent =
| { t: 'phase'; phase: Phase }
| { t: 'reason'; text: string }
| { t: 'invariant'; id: string; statement: string }
| { t: 'action'; label: string; screenshot: string }
| { t: 'finding'; finding: Finding }
| { t: 'test'; findingId: string; code: string }
| { t: 'done'; runtimeMs: number };
// reducer appends to feed, swaps the browser frame, spawns findings.