Stateful Relayer
How the Relayer keeps state: classification, durability rules, and crash consistency. Resolves Relayer-challenges § 3.1; implementation context in Relayer-architecture.
Principle
The spec's invariant stands: no relayer state is security-critical (Relayer § State): consensus state and caches are rebuildable from the source chain, and losing the relay-request store is a liveness gap (applications re-register), never a safety one. Durable storage exists for two lesser reasons:
- Economic durability (zk-proof mode): a generated proof costs GPU-minutes; losing it to a crash burns money, not safety. In direct mode the bundle is rebuilt from the mirror for free, and this reason disappears.
- Operational continuity:
QueryStatushistory, retry budgets, and the relay-request store must survive restarts.
The database is never a trust component. Reviewers should read it as a cache-plus-job-queue, not a ledger.
State classification
Each store has a different source of truth, and therefore a different durability requirement:
| Store | Source of truth | Cost if lost | Home | Recovery |
|---|---|---|---|---|
| Op queue (tasks, retries, status) | — (it is the record) | In-flight work + status history | Postgres (pg-queue) | Survives crashes by Postgres WAL durability |
| Proof bundles | Nowhere in zk-proof mode (minted by GPU workers); the consensus mirror in direct mode | GPU-minutes per proof (zk-proof); a repackaging (direct) | Same DB, written transactionally with job completion | Never lost once generated |
| Consensus mirror | The source chain | Hours of re-ingestion | Memory + periodic snapshot blob on disk | Load latest snapshot, replay forward (Syncing relayer) |
| Block cache | The source chain | Cheap re-fetch | Disk KV, pruned | Re-ingest |
| Relay requests | The registering application (RegisterRelayRequest) | Liveness gap until applications re-register | Same DB (the relayer is the system of record) | Cross-check against the Router's ApplicationRegistry; re-validate on startup (Syncing relayer § Request sync) |
The consensus mirror is a witness blob, not relational data: it is snapshotted and replayed like node state-sync, never decomposed into database rows.
The op queue is the state machine
A relay task moves through queued → proving → submitted → delivered | failed, exactly the QueryStatus enum (Relayer § QueryStatus). The status API is an indexed view over the op queue; there is no separate status store to keep consistent.
Crash-consistency invariants
- A proof bundle is persisted in the same transaction that completes its proving job, before any broadcast attempt. This is the only hard write-ordering rule in the relayer.
- Local semantics are at-least-once. Exactly-once is enforced on-chain by the Router's delivered-
MessageIDlog (Router § "Exactly-once delivery"): a duplicate submission reverts harmlessly. Correctness never depends on local exactly-once; duplicates cost only gas, bounded by the local(request_id, MessageID)dedupe (Relayer § Security Considerations) plus query-before-prove and jitter (Relayer-architecture § Multi-relayer). - Mirror snapshots are advisory. A corrupt snapshot is discarded and rebuilt by re-ingestion; the mirror is periodically cross-validated against the on-chain checkpoint commitments. A bad witness can only produce a proof that fails to verify — it never reaches applications.
Retention
| Data | Keep until |
|---|---|
| Block cache entries | Proofs for that block delivered and retry budget exhausted |
| Delivered / failed submissions | Configurable audit window (operator policy) |
| Consensus mirror snapshots | Last K snapshots (rollback margin for a corrupt latest) |
Backend choice
Decision (2026-07, supersedes the earlier SQLite-first rule): Postgres from day 1. The implementation reuses Union's pg-queue (with voyager-vm) as actual code rather than as a pattern, and pg-queue is Postgres-native: code reuse outweighs the operational cost the embedded-first rule was avoiding, and the team already operates Postgres. This also removes the old upgrade step: the prover-fleet scenario (multiple prover workers leasing jobs from one shared queue, build-order Phase 3, Relayer-architecture § Build order) is served by the same Postgres job table (FOR UPDATE SKIP LOCKED leasing) with no migration. The op model does not change either way.
Related documents
- Relayer-architecture: where this state model sits in the component layout
- Syncing relayer: recovery paths that rebuild the caches described here
- Relayer § State: the spec-level store definitions
- Router § "Exactly-once delivery": the on-chain idempotency floor