Adapter Acceptance Tests¶
An adapter is correct when its code follows the framework's rules. It is accepted when the system, driven by a real stimulus on real hardware, produces the behavior PQ expects to see. This catalog is the second statement: for every stimulus a device family can receive, what must appear in PQ — which event, on which Thing, carrying what.
The scenarios are vendor-agnostic on purpose. They name no product, no address and no card number;
they name a role ({door}, {camera}, {partition}) that is resolved from the device tree the
adapter imported. The same scenario therefore runs against any adapter of that category, and a
verdict from one rig is comparable with a verdict from another.
Two kinds of checks
Technical requirements protect the platform: memory safety, stable long-running operation and
correct use of the framework (for example, no direct EventBuilder calls). Protequ requires them.
The build enforces many of them with analyzers; review checks the rest.
Behavior checks describe how a device appears in PQ (for example, a detector alarm also raises its partition). We recommend that your adapter meets all of them. If you decide not to, for any reason, that is a legitimate choice and it is yours. How closely an adapter matches PQ behavior can only influence a customer's decision whether to install it.
Generated from the matrix
This page and the scenario pages under it are generated from the acceptance-test matrix — the
YAML the certification runner itself executes. Prose lives in the template, data lives in the
YAML, and nothing here is written twice. Matrix v17, 95
scenarios, content hash sha256:3fc8afa8478511956826ef4fbefeca94b8750a9158337745d70c031664b0bb12.
The scenario catalog¶
- Cross-cutting — 21 scenarios
- Access control — 36 scenarios
- Intrusion (EZS) — 21 scenarios
- Video (CCTV / NVR) — 17 scenarios
Which of them apply to a given adapter is not a judgement call. It is computed from three inputs:
the adapter's category (from adapter-registration.yaml), its declared capabilities (from
capabilities-coverage.yaml), and the topology of the rig it is run against. A scenario whose
capability the adapter does not declare, or whose topology the rig cannot satisfy, is recorded as
not applicable with the shortfall as its reason — it is never quietly skipped, and the rig is never
rebuilt by hand to make a scenario runnable.
How a scenario is written¶
Each scenario states a stimulus and the PQ-visible behavior that must follow it, and then carries executable steps — the acts to perform and what the timeline must show afterwards.
| Key | Meaning |
|---|---|
require |
a state that must already hold — read, never waited for |
command + on |
issue a command at a role |
stimulus |
a person performs one of the scenario's declared physical acts |
restore: true |
this step is the restoring half of that act (link back, cover closed, door shut) |
restart: process |
kill the adapter and start it again |
fragment |
turn transport fragmentation on or off |
expect |
records that must arrive, as filter expressions |
reject |
records that must not arrive within the window |
timeout |
seconds the window stays open |
Two rules explain most of what looks surprising in the steps. require is not expect in a
hurry: the framework publishes a status only when it changes, so a state that already holds
produces no record and waiting for it would time out like a missing transition. And a successful
command has no audited event of its own — the framework audits only failures, so a scenario that
proves "the command was carried out rather than dropped" asserts the command result, while the
effect the operator asked for is proven by the domain event or status that reports it.
Where the verdict is a comparison no timeline can express — two tree dumps around a re-import, a
number inside a payload, two video streams at different resolutions — the scenario says so in
agent_verdict instead of pretending the steps decided it.
Waves — when a scenario runs¶
A certification run is ordered so it bothers a person as little as possible. The order follows the cost of getting the rig back to a known state, not risk: the rig is in test service, so breaking something is the result certification exists to produce.
| Wave | Character | Needs a person | Content |
|---|---|---|---|
| 1 | unattended | no | discovery, connection, addressing, commands, synchronization, restart |
| 2 | at the reader | yes | cards, PINs, keypad operations, panic |
| 3 | zone and door disruption | yes | alarms and restores, entry/exit, forced and held doors, motion in a camera's view |
| 4 | session-breaking | yes | tamper, power, a removed module, a cut link |
Wave 1 runs to the end on its own and its result decides whether it is worth calling anyone to the rig at all — a scenario that fails on the automatic machinery underneath it is neither confirmed nor refuted by hardware.
Physical acts¶
The acts a person can be asked to perform are a closed vocabulary. A new device family reuses the act it already performs rather than minting a synonym — motion in a camera's field of view is a walked detector, covering a lens is a tamper — so every scenario that degrades to the same act degrades to the same wave. The acts this catalog currently calls for:
| Act | Meaning |
|---|---|
present_credential |
hold a physical credential at a reader (card, token, biometric) |
enter_pin |
type a PIN on a keypad |
use_credential |
exercise a named credential wherever it belongs |
keypad_operation |
perform an operator action at the keypad (arm, disarm) |
keypad_panic |
trigger the keypad's panic or duress function |
trip_zone |
violate a zone: open a contact, walk a detector — or walk through a camera's field of view |
restore_zone |
return a violated zone, or a settled scene, to its normal state |
open_door |
open a monitored door leaf without a grant, then close it |
hold_door |
keep a monitored door open past its held-open timeout, then close it |
cut_link |
break the connection between the adapter and the device, then restore it |
tamper |
open a tamper-protected enclosure or cover — or obstruct a camera lens — then restore it |
power_fault |
remove mains or battery power, then restore it |
remove_module |
detach a supervised module — a bus card, a camera, a recorder's disk — then reattach it |
Some of them the lab performs itself where the transport allows it, and the scenario then runs unattended; where it cannot, the same act falls to a person and the report says so with the reason.
Verdicts¶
| Verdict | Meaning |
|---|---|
passed |
the PQ-visible behavior was observed in the full expected shape |
partial |
the capability is present but its shape is degraded or different, and characterized — rolls up to certified with limitations, not a failure |
failed |
a hard violation of the adapter's contract: behavior missing or broken though it could be done right |
blocked_by_hw |
could not be physically exercised on the rig |
n_a |
legitimately not applicable — the device is modeled differently, or the protocol has no such thing |
A hardware or protocol limit is never a failed. We certify the adapter's conformance to the
framework, not the hardware, and only the adapter's own defects block a result.
Preconditions¶
Named, reusable setup blocks a scenario references by id. A precondition is not a test and carries no verdict. PQ-side blocks the runner establishes itself through the production API; rig-side blocks describe a physical state a person has to bring about first.
Version history¶
The catalog carries one monotonic version for the whole set. An adapter's certification stamps the version its verdicts were produced against, so a result reads as certified against Matrix v9. When the catalog advances, the runner scopes the delta: scenarios newer than the stamped version need a fresh verdict, existing ones carry forward.
v1 — Initial catalog — cross-cutting (connection, liveness, audit visibility, config integrity, time sync, supervision, discovery), intrusion (arm/disarm, bypass, alarm+reset, entry/exit, sounder), access (credential mgmt + events, card formats, sync mode, reset/restore, double-sync, schedules, access levels, holidays, antipassback, door control, reader mode).
v2 — Adds numeric PIN synchronization coverage for even and odd supported digit counts.
v3 — Adds command→event pairing coverage — an operator command must be confirmed by its audited execution event in the remote variant with operator identity (AAT-CROSS-CUTTING-UNIVERSAL-07) — and redundant-state-command fast-fail: a repeat command on an already-satisfied state fails fast with a reason instead of hanging on a confirming event that cannot come (AAT-INTRUSION-ARM-DISARM-06).
v4 — Adds addressing-fidelity coverage (door confirmation attribution, partition isolation), registry/restart behavior (stale-registry command failure, restart without replay or resync, state read back after startup), channel arbitration under an exclusive hold, unresolved-address visibility, synchronization ownership and person-delete cascade, transitional-state disarm, force arming, and keypad life-safety. Splits PIN coverage into synchronization round-trip and keypad authentication. New scenarios carry stimulus/wave/depends_on and, where physical, a machine-readable stimulus_request. Adds transport-framing coverage — a payload split across segments is reassembled before parsing — and states mid-session link loss as a silent link rather than a closed socket, so keepalive/watchdog detection is what the scenario proves.
v5 — Re-anchors five scenarios that claimed a PQ stimulus although only the device can answer them: reset & restore, double sync and sync mode split into an automatic half (what PQ sent) and a physical half (what the device then does with a credential), holidays and sounder control move whole to a human stimulus. Their previous verdicts do not carry over — the evidence changed. Physical acts become a list (stimulus_requests), so one scenario can state several, and an act a lab verb performs names it in satisfied_by; the link cut is such an act and enters the kind vocabulary as cut_link.
v6 — Every scenario carries executable steps: the acts to perform (require / command / stimulus / restart / fragment) and what the timeline must then show (expect, reject, order). The command→event pairing therefore lives on the card, not in the lab, which stays free of taxonomy knowledge. Things are addressed by role placeholders the generator resolves from the rig, so the matrix keeps naming no ids. Where the verdict is a comparison the timeline cannot express — two tree dumps, a payload value, replay dedup — agent_verdict says so instead of pretending. Existing verdicts carry forward: the steps restate claims the scenarios already made in prose. AAT-ACCESS-ACCESS-LEVELS-01 gains the two-door topology its second door always implied.
v7 — Corrections a first live run on panel hardware forces. Command ids in steps are the canonical pq.command.* — the tool checks them against the Thing's declared commands, and a short id is rejected. A successful command has no audited event: the framework audits only failures (pq.event.technical.command.failed), so scenarios assert the command result itself (kind=commandResult;result=success) instead of a .succeeded event that does not exist — AAT-CROSS-CUTTING-UNIVERSAL-03 and every scenario that echoed it. A cold start raises no pq.event.connection.restored either, because nothing was lost yet, so AAT-CROSS-CUTTING-UNIVERSAL-01 expects presence, pq.event.system.adapter.started and lifecycle.ready — recovery stays with AAT-CROSS-CUTTING-UNIVERSAL-02. Which partial-arm command a panel exposes is its own business (arm.partial, .stay, .sleep), so AAT-INTRUSION-ARM-DISARM-04 names the role {any_partial_arm_command} and the generator resolves it from the Thing's declared commands. A configuration change is answered by the production pq.command.restart on the root Thing, which rebuilds the adapter's registry — an ordinary command step, distinct from restart: process, which kills the adapter and starts it again. AAT-CROSS-CUTTING-UNIVERSAL-08 takes the command; -09 and -14 keep the process restart their prose describes.
v8 — Two scenarios are re-anchored on what they actually assert. Command-targeting fidelity is a statement about addressing, not about access, so it lives in the cross-cutting domain as AAT-CROSS-CUTTING-COMMAND-TARGETING-01; doors remain the pair it uses only while requires_topology can name categories but not a declared command, and the partition variant (AAT-INTRUSION-TARGETING-01) folds into it once it can. And the unit of authorization is the access point, not the door (MODEL-002) — AAT-ACCESS-ACCESS-LEVELS-01 therefore accepts any two access points, which is what lets a panel authorizing users per partition prove the binding flows through person.Devices instead of a static bitmask. Command targeting is also asserted per type rather than once: each type is addressed by its own scheme on the wire, so a panel that gets doors right can still transpose relay outputs or zones, and the scenario runs on every type of which the rig holds a pair. AAT-ACCESS-RESET-RESTORE-01 stops expecting pq.event.access.credential.deleted.all: the taxonomy carries the event but the reset path emits it nowhere, so the scenario asserts the reset's own result and the synchronization that follows — and records that a panel-wide credential wipe is, for now, absent from the audit trail.
v9 — Adds the video domain (scenarios/video.yaml) — the first category whose Things are cameras rather than doors or partitions, so {camera} enters the role vocabulary. The physical acts reuse the closed stimulus vocabulary instead of growing it: motion in a field of view is a trip_zone, covering a lens is a tamper, unplugging a camera or a disk from a recorder is a remove_module — the act a person performs is the same one, only the target differs. Coverage is channel discovery and blindness, live/playback/snapshot including the main-vs-substream tier and recovery of a live view after an outage, PTZ and presets, analytics (motion, rules, obstruction), and storage faults with stream teardown. Two of them state gaps rather than claiming a pass: AAT-VIDEO-CHANNEL-03 carries framework_gap because a duplicated channel address is today only a log line inside the adapter — the shadowed Thing keeps looking healthy while it can no longer report anything — and AAT-VIDEO-LIVE-02 records that the step grammar cannot yet carry a command argument, so the tier under test is named with the verdict. The new preconditions say what a video rig must bring: a modeled camera, footage over a known range, a registered live stream, a deliberate address collision, and — the one that silently invalidates every physical video scenario — detections that are not merely enabled but actually pushed to the adapter.
v10 — Generalizes odd-digit PIN credential certification from keypad-only authentication to the adapter-declared access point. The scenario now proves that the synchronized PIN and its authorization data reach the device unit that actually enforces access: a door/gate/turnstile grants, while a partition-style panel accepts the credential-backed keypad operation such as arm or disarm. The {access_point} role enters the vocabulary for scenarios whose target is the authorization unit rather than a fixed device category.
v11 — Adds autonomous packet/fact-level access synchronization isolation coverage. Full sync proves the owned AAT dataset moves as a whole; incremental sync proves one person's change does not rewrite other people; selective sync proves a selection containing one person leaves other pending people untouched. The scenarios are vendor-agnostic at the verdict layer: sync facts, generated sync dumps, address mappings, and vendor packet traces together define the touched identity set.
v12 — Adds unattended repeated remote-open coverage for access doors. The new scenario issues pq.command.access.open three times on the same door and requires each cycle to succeed, publish the standard Door release event, and return to lock.secured without a stale position.open status when no physical door contact is opened. The matrix documentation also records the exact states= filter used when a scenario must prove the full status set.
v13 — Adds the address-lifecycle invariant to selective synchronization coverage (AAT-ACCESS-SYNC-ISOLATION-04): no device address may be created or updated and then deleted inside a single sync. The stimulus is one person's credential set growing and then shrinking — a card gaining a standalone PIN and losing it again — because that is where an update turns into an unrelated delete plus an unrelated add whenever the record's identity key moves with the credential. The verdict is the surviving device state and the person's address mappings, not the sync's own success report, so a sync that destroys what it just uploaded fails even while reporting success.
v14 — Closes the self-certifying loophole in operator-command audit (AAT-CROSS-CUTTING-UNIVERSAL-07). That scenario exempts commands documented as ACK-only, and the exemption let an adapter declare a security actuation — a remote door open, a relay activation — ACK-only and pass without an audited remote event. The exemption is now scoped: it reaches only commands whose action has no .remote variant in the event taxonomy. A new universal scenario (AAT-CROSS-CUTTING-UNIVERSAL-16) asserts the positive obligation for the rest — every command whose action carries a .remote variant (door control, arm/disarm, output switching) surfaces that attributed remote event when issued from PQ. Membership is read from the taxonomy, not from the adapter's supervision declaration, so a security actuation cannot be classified out of audit scope. The role {any_actuation_command} enters the value vocabulary for it.
v15 — Adds partition-propagation coverage for zone alarms (AAT-INTRUSION-ALARM-04): a zone alarm must surface on at least one partition the zone belongs to, not only on the zone. It targets devices that report the trip against the zone address alone (Dominus type-1 detector events), where a verbatim forward leaves the partition reading un-alarmed. When the packet names the partition that is authoritative; when it names only the zone the partition alarm is induced from the modeled zone→partition membership. Pairs with compliance RULE-060.
v16 — Every scenario carries a kind: technical when it checks a requirement that protects the platform (stable long-running operation, resource cleanup, replay and duplicate safety), or behavior when it checks how a device appears in PQ. A failed behavior scenario is reported to the customer as not matching PQ behavior, not as a broken adapter. No scenario changes its steps or its expected result.
v17 — Adds live credential enrollment (AAT-ACCESS-ENROLLMENT-01..04), which the catalog named as a capability and no scenario covered: a card capture and a biometric template capture must each become a CREDENTIAL in PQ rather than only an event, an enrollment nobody completes must leave neither a PQ credential nor an orphan record on the device, and an enrolled template must reach a device it was not captured on. The last one is the difference between a credential and a local record: it is what lets PQ move a template between readers and revoke it per credential instead of per person. Cancellation is deliberately NOT covered - the taxonomy has no cancel command, and the gap is recorded in the scenario file rather than closed with an invented verb. New precondition: person_synced_without_the_enrolled_credential.