Adapter Acceptance Tests¶
An adapter is correct when its code follows the framework's rules. It is accepted when the system, driven by a real stimulus on real hardware, produces the behavior PQ expects to see. This catalog is the second statement: for every stimulus a device family can receive, what must appear in PQ — which event, on which Thing, carrying what.
The scenarios are vendor-agnostic on purpose. They name no product, no address and no card number;
they name a role ({door}, {camera}, {partition}) that is resolved from the device tree the
adapter imported. The same scenario therefore runs against any adapter of that category, and a
verdict from one rig is comparable with a verdict from another.
Generated from the matrix
This page and the scenario pages under it are generated from the acceptance-test matrix — the
YAML the certification runner itself executes. Prose lives in the template, data lives in the
YAML, and nothing here is written twice. Matrix v9, 81
scenarios, content hash sha256:7bbaf105db1a3bee2ea5d52018bdae5eb83e399f4234541187bf413db0ee627d.
The scenario catalog¶
- Cross-cutting — 20 scenarios
- Access control — 26 scenarios
- Intrusion (EZS) — 18 scenarios
- Video (CCTV / NVR) — 17 scenarios
Which of them apply to a given adapter is not a judgement call. It is computed from three inputs:
the adapter's category (from adapter-registration.yaml), its declared capabilities (from
capabilities-coverage.yaml), and the topology of the rig it is run against. A scenario whose
capability the adapter does not declare, or whose topology the rig cannot satisfy, is recorded as
not applicable with the shortfall as its reason — it is never quietly skipped, and the rig is never
rebuilt by hand to make a scenario runnable.
How a scenario is written¶
Each scenario states a stimulus and the PQ-visible behavior that must follow it, and then carries executable steps — the acts to perform and what the timeline must show afterwards.
| Key | Meaning |
|---|---|
require |
a state that must already hold — read, never waited for |
command + on |
issue a command at a role |
stimulus |
a person performs one of the scenario's declared physical acts |
restore: true |
this step is the restoring half of that act (link back, cover closed, door shut) |
restart: process |
kill the adapter and start it again |
fragment |
turn transport fragmentation on or off |
expect |
records that must arrive, as filter expressions |
reject |
records that must not arrive within the window |
timeout |
seconds the window stays open |
Two rules explain most of what looks surprising in the steps. require is not expect in a
hurry: the framework publishes a status only when it changes, so a state that already holds
produces no record and waiting for it would time out like a missing transition. And a successful
command has no audited event of its own — the framework audits only failures, so a scenario that
proves "the command was carried out rather than dropped" asserts the command result, while the
effect the operator asked for is proven by the domain event or status that reports it.
Where the verdict is a comparison no timeline can express — two tree dumps around a re-import, a
number inside a payload, two video streams at different resolutions — the scenario says so in
agent_verdict instead of pretending the steps decided it.
Waves — when a scenario runs¶
A certification run is ordered so it bothers a person as little as possible. The order follows the cost of getting the rig back to a known state, not risk: the rig is in test service, so breaking something is the result certification exists to produce.
| Wave | Character | Needs a person | Content |
|---|---|---|---|
| 1 | unattended | no | discovery, connection, addressing, commands, synchronization, restart |
| 2 | at the reader | yes | cards, PINs, keypad operations, panic |
| 3 | zone and door disruption | yes | alarms and restores, entry/exit, forced and held doors, motion in a camera's view |
| 4 | session-breaking | yes | tamper, power, a removed module, a cut link |
Wave 1 runs to the end on its own and its result decides whether it is worth calling anyone to the rig at all — a scenario that fails on the automatic machinery underneath it is neither confirmed nor refuted by hardware.
Physical acts¶
The acts a person can be asked to perform are a closed vocabulary. A new device family reuses the act it already performs rather than minting a synonym — motion in a camera's field of view is a walked detector, covering a lens is a tamper — so every scenario that degrades to the same act degrades to the same wave. The acts this catalog currently calls for:
| Act | Meaning |
|---|---|
present_credential |
hold a physical credential at a reader (card, token, biometric) |
enter_pin |
type a PIN on a keypad |
use_credential |
exercise a named credential wherever it belongs |
keypad_operation |
perform an operator action at the keypad (arm, disarm) |
keypad_panic |
trigger the keypad's panic or duress function |
trip_zone |
violate a zone: open a contact, walk a detector — or walk through a camera's field of view |
restore_zone |
return a violated zone, or a settled scene, to its normal state |
open_door |
open a monitored door leaf without a grant, then close it |
hold_door |
keep a monitored door open past its held-open timeout, then close it |
cut_link |
break the connection between the adapter and the device, then restore it |
tamper |
open a tamper-protected enclosure or cover — or obstruct a camera lens — then restore it |
power_fault |
remove mains or battery power, then restore it |
remove_module |
detach a supervised module — a bus card, a camera, a recorder's disk — then reattach it |
Some of them the lab performs itself where the transport allows it, and the scenario then runs unattended; where it cannot, the same act falls to a person and the report says so with the reason.
Verdicts¶
| Verdict | Meaning |
|---|---|
passed |
the PQ-visible behavior was observed in the full expected shape |
partial |
the capability is present but its shape is degraded or different, and characterized — rolls up to certified with limitations, not a failure |
failed |
a hard violation of the adapter's contract: behavior missing or broken though it could be done right |
blocked_by_hw |
could not be physically exercised on the rig |
n_a |
legitimately not applicable — the device is modeled differently, or the protocol has no such thing |
A hardware or protocol limit is never a failed. We certify the adapter's conformance to the
framework, not the hardware, and only the adapter's own defects block a result.
Preconditions¶
Named, reusable setup blocks a scenario references by id. A precondition is not a test and carries no verdict. PQ-side blocks the runner establishes itself through the production API; rig-side blocks describe a physical state a person has to bring about first.
Version history¶
The catalog carries one monotonic version for the whole set. An adapter's certification stamps the version its verdicts were produced against, so a result reads as certified against Matrix v9. When the catalog advances, the runner scopes the delta: scenarios newer than the stamped version need a fresh verdict, existing ones carry forward.
v1 — Initial catalog — cross-cutting (connection, liveness, audit visibility, config integrity, time sync, supervision, discovery), intrusion (arm/disarm, bypass, alarm+reset, entry/exit, sounder), access (credential mgmt + events, card formats, sync mode, reset/restore, double-sync, schedules, access levels, holidays, antipassback, door control, reader mode).
v2 — Adds numeric PIN synchronization coverage for even and odd supported digit counts.
v3 — Adds command→event pairing coverage — an operator command must be confirmed by its audited execution event in the remote variant with operator identity (AAT-CROSS-CUTTING-UNIVERSAL-07) — and redundant-state-command fast-fail: a repeat command on an already-satisfied state fails fast with a reason instead of hanging on a confirming event that cannot come (AAT-INTRUSION-ARM-DISARM-06).
v4 — Adds addressing-fidelity coverage (door confirmation attribution, partition isolation), registry/restart behavior (stale-registry command failure, restart without replay or resync, state read back after startup), channel arbitration under an exclusive hold, unresolved-address visibility, synchronization ownership and person-delete cascade, transitional-state disarm, force arming, and keypad life-safety. Splits PIN coverage into synchronization round-trip and keypad authentication. New scenarios carry stimulus/wave/depends_on and, where physical, a machine-readable stimulus_request. Adds transport-framing coverage — a payload split across segments is reassembled before parsing — and states mid-session link loss as a silent link rather than a closed socket, so keepalive/watchdog detection is what the scenario proves.
v5 — Re-anchors five scenarios that claimed a PQ stimulus although only the device can answer them: reset & restore, double sync and sync mode split into an automatic half (what PQ sent) and a physical half (what the device then does with a credential), holidays and sounder control move whole to a human stimulus. Their previous verdicts do not carry over — the evidence changed. Physical acts become a list (stimulus_requests), so one scenario can state several, and an act a lab verb performs names it in satisfied_by; the link cut is such an act and enters the kind vocabulary as cut_link.
v6 — Every scenario carries executable steps: the acts to perform (require / command / stimulus / restart / fragment) and what the timeline must then show (expect, reject, order). The command→event pairing therefore lives on the card, not in the lab, which stays free of taxonomy knowledge. Things are addressed by role placeholders the generator resolves from the rig, so the matrix keeps naming no ids. Where the verdict is a comparison the timeline cannot express — two tree dumps, a payload value, replay dedup — agent_verdict says so instead of pretending. Existing verdicts carry forward: the steps restate claims the scenarios already made in prose. AAT-ACCESS-ACCESS-LEVELS-01 gains the two-door topology its second door always implied.
v7 — Corrections a first live run on a Satel Integra rig forced. Command ids in steps are the canonical pq.command.* — the tool checks them against the Thing's declared commands, and a short id is rejected. A successful command has no audited event: the framework audits only failures (pq.event.technical.command.failed), so scenarios assert the command result itself (kind=commandResult;result=success) instead of a .succeeded event that does not exist — AAT-CROSS-CUTTING-UNIVERSAL-03 and every scenario that echoed it. A cold start raises no pq.event.connection.restored either, because nothing was lost yet, so AAT-CROSS-CUTTING-UNIVERSAL-01 expects presence, pq.event.system.adapter.started and lifecycle.ready — recovery stays with AAT-CROSS-CUTTING-UNIVERSAL-02. Which partial-arm command a panel exposes is its own business (arm.partial, .stay, .sleep), so AAT-INTRUSION-ARM-DISARM-04 names the role {any_partial_arm_command} and the generator resolves it from the Thing's declared commands. A configuration change is answered by the production pq.command.restart on the root Thing, which rebuilds the adapter's registry — an ordinary command step, distinct from restart: process, which kills the adapter and starts it again. AAT-CROSS-CUTTING-UNIVERSAL-08 takes the command; -09 and -14 keep the process restart their prose describes.
v8 — Two scenarios are re-anchored on what they actually assert. Command-targeting fidelity is a statement about addressing, not about access, so AAT-ACCESS-TARGETING-01 becomes the universal AAT-CROSS-CUTTING-COMMAND-TARGETING-01; doors remain the pair it uses only while requires_topology can name categories but not a declared command, and the partition variant (AAT-INTRUSION-TARGETING-01) folds into it once it can. And the unit of authorization is the access point, not the door (MODEL-002) — AAT-ACCESS-ACCESS-LEVELS-01 therefore accepts any two access points, which is what lets a panel authorizing users per partition prove the binding flows through person.Devices instead of a static bitmask. Command targeting is also asserted per type rather than once: each type is addressed by its own scheme on the wire, so a panel that gets doors right can still transpose relay outputs or zones, and the scenario runs on every type of which the rig holds a pair. AAT-ACCESS-RESET-RESTORE-01 stops expecting pq.event.access.credential.deleted.all: the taxonomy carries the event but the reset path emits it nowhere, so the scenario asserts the reset's own result and the synchronization that follows — and records that a panel-wide credential wipe is, for now, absent from the audit trail.
v9 — Adds the video domain (scenarios/video.yaml) — the first category whose Things are cameras rather than doors or partitions, so {camera} enters the role vocabulary. The physical acts reuse the closed stimulus vocabulary instead of growing it: motion in a field of view is a trip_zone, covering a lens is a tamper, unplugging a camera or a disk from a recorder is a remove_module — the act a person performs is the same one, only the target differs. Coverage is channel discovery and blindness, live/playback/snapshot including the main-vs-substream tier and recovery of a live view after an outage, PTZ and presets, analytics (motion, rules, obstruction), and storage faults with stream teardown. Two of them state gaps rather than claiming a pass: AAT-VIDEO-CHANNEL-03 carries framework_gap because a duplicated channel address is today only a log line inside the adapter — the shadowed Thing keeps looking healthy while it can no longer report anything — and AAT-VIDEO-LIVE-02 records that the step grammar cannot yet carry a command argument, so the tier under test is named with the verdict. The new preconditions say what a video rig must bring: a modeled camera, footage over a known range, a registered live stream, a deliberate address collision, and — the one that silently invalidates every physical video scenario — detections that are not merely enabled but actually pushed to the adapter.