Chapter 10

Conformance & fixtures

Reflects Holonograph v0.23.0 · ~9 min read

A capability surface is what you say your agent can do — the tools it exposes, the rules it should follow. Conformance turns that description into reproducible checks: a set of fixtures a scheduled run can execute, and a verdict that tells you whether each rule held. The flow is derive → approve → check.

What it is

Three moving pieces:

• The derive step reads a capability surface (its tool declarations and prose rules) and proposes candidate fixtures — one per rule, each with the strongest assertion the rule allows.
• The fixtures step is the human-in-the-loop approval gate. A named approver reviews the candidates, edits or discards, and commits the ones they trust. Nothing runs as a check until it's approved.
• The check step runs the approved fixtures against real events over a window and returns a verdict per fixture plus a summary.

Enforceability taxonomy

Not every rule can be checked mechanically. The derive step labels each candidate with one of three enforceability categories, so the coverage report is honest about which rules the machine can actually hold you to:

Why it matters

A prose rule that can't be checked is a rule the system will drift on. Conformance moves rules from "our team believes this" to "the last check confirmed this at 06:00 Tuesday" — the coverage report tells you exactly how many of the surface's rules you have mechanical accountability for, and the needs-repair panel tells you which ones the derive step couldn't formalize on its own.

How to use it

1. Derive candidates from the surface

Send the capability surface — its tools and its rules — and get back a set of candidate fixtures with per-rule enforceability labels.

POST /lens/conformance/derive
x-holonograph-run-mode: production
Content-Type: application/json

{
  "surfaceId": "classify-intent",
  "tools": [ /* tool declarations */ ],
  "rules": [
    "If the user asks for a refund, invoke the refund_lookup tool before responding.",
    "Never quote a specific dollar amount unless refund_lookup returned one.",
    "Be polite."
  ]
}

The response gives you the candidates, a coverage summary (e.g. "2 of 3 rules mechanically enforceable"), and a needs-repair list for rules the derive step could not fully formalize.

2. Review and approve

Each candidate is a JSON fixture describing an assertion (or a set of assertions, guarded by a when predicate). Review the derived shape, edit if needed, and submit each approved fixture with an approverId.

POST /lens/conformance/fixtures
x-holonograph-run-mode: production
Content-Type: application/json

{
  "approverId": "brian",
  "sourceSurfaceId": "classify-intent",
  "fixture": {
    "fixtureId": "refund-requires-lookup",
    "ruleId": "billing-triage-refund-flow-01",
    "ruleSource": "Refund rules v3",
    "description": "If the user asks for a refund, refund_lookup must be invoked before responding.",
    "enforceability": "trace-enforceable",
    "appliesTo": { "surfaceId": "classify-intent" },
    "when": [{ "kind": "marker-contains", "value": "refund" }],
    "assertion": { "kind": "tool-invoked", "tool": "refund_lookup" }
  }
}

Response: an eventId for the approval event. The fixture is now live in the store and will be picked up by subsequent checks.

3. Run a check

Point the checker at either an inline fixture or a stored one, plus a window and a surface filter.

POST /lens/conformance/check
x-holonograph-run-mode: production
Content-Type: application/json

{
  "fixtureId": "refund-requires-lookup",
  "surfaceId": "classify-intent",
  "since": "2026-07-01T00:00:00Z",
  "until": "2026-07-05T00:00:00Z"
}

The response includes a top-level verdict, a per-event breakdown, the count of events checked, and the window actually used.

Reading the output

Assertion kinds

Guarded assertions

A fixture can carry a when predicate that decides whether the assertions apply to a given event. The predicate uses the same assertion kinds as the assertions themselves. If when is absent, the assertions apply to every event in the window.

{
  "fixtureId": "no-dollar-figures-without-lookup",
  "when": [{ "kind": "marker-not-contains", "value": "refund_lookup" }],
  "assertion": { "kind": "marker-not-contains", "value": "$" }
}

An event that fails the when predicate returns not-applicable for this fixture instead of pass/fail.

Verdict precedence

When multiple assertions run against one event, the overall event verdict is the strongest of the individual assertion verdicts, in this order:

1. fail — one assertion definitively failed.
2. pass — at least one assertion passed and none failed.
3. requires-judge — a judge assertion needs to be run before the verdict is complete.
4. unevaluable — the event was missing the shape needed to run the assertion at all.
5. not-applicable — the when guard filtered this event out.

The check's top-level verdict is the strongest across events, using the same precedence.

Coverage report

The derive response's coverage summary is the honest headline: K of N rules are mechanically enforceable. It's the number to track over time — as your rules mature and the derive step's classifier improves, the ratio should climb. What it can't yet cover ends up in the needs-repair panel.

Needs-repair panel

A rule ends up in needsRepair when the derive step couldn't turn it into a check on its own — usually because the rule is ambiguous, or references a system state the fixture format doesn't reach yet. Two ways to clear the entry: rephrase the rule so the derive step can formalize it, or hand-author a fixture and approve it directly.

Reference

Endpoints

POST /lens/conformance/derive — response fields

POST /lens/conformance/check — request & response