Relayer Architecture
Implementation architecture for the Relayer: how the specified component is actually built. Core patterns are adapted from Union's Voyager relayer, right-sized for TMP's trust model and proving-centric pipeline. The state model lives in Stateful relayer; bootstrap and sync in Syncing relayer. Vocabulary (checkpoint, consensus mirror, submission vs delivery, stage ↔ module/plugin mapping) is canonical in Relayer § Terminology.
Guiding principles
Four patterns borrowed from Voyager:
| Pattern | What it gives us |
|---|---|
Protocol-as-Spec | The protocol is a trait the engine is generic over, not logic hardcoded into it. TMP will keep evolving pre-mainnet; the engine survives protocol changes |
| Orthogonal axes | Chain / consensus / interface / client-type are independent dimensions that compose. Adding a chain = config + a small module, not a fork |
| Op-VM on a durable queue | Relaying is a persistent computation tree (call/seq/conc/promise): crash recovery, declarative ordering, and batching fall out of the model |
| Module vs Plugin | Read-only capabilities (modules) are cleanly separated from stateful queue-owning logic (plugins) |
Three Voyager/IBC assumptions that do not carry over to TMP:
| Voyager / IBC assumption | TMP reality | Consequence |
|---|---|---|
| Relayer is trustless and permissionless | Relayer is permissioned, trusted for canonical-fork connection + liveness (Relayer § Trust Model) | No fee-market equilibrium machinery; multi-relayer = coordination among a small authorized set |
Packet lifecycle (recv/ack/timeout) over channels | One-way delivery of registered RelayRequests; exactly-once enforced by the Router's delivered-MessageID log (Router § "Exactly-once delivery") | The Spec exposes RelayRequest/MessageID/ConsensusState: no channels, acks, or timeouts |
| A proof is a cheap Merkle query against a node | In zk-proof mode a proof is a consensus replay in a circuit (minutes per proof); in direct mode it is a packaged consensus bundle | Proving is a first-class pipeline stage, not an optional plugin: it dictates the storage design (Stateful relayer) even though it degenerates to packaging for direct-mode light clients |
Storage decision (2026-07, supersedes the earlier SQLite-first right-sizing rule): Postgres from day 1. The Voyager patterns above are reused as actual code — Union's voyager-vm + pg-queue crates (Apache-2.0; unpublished, so git-pinned to a commit or vendored) — and pg-queue is Postgres-native. Code reuse outweighs the operational cost the embedded-first rule was avoiding, and the team already operates Postgres. The op model remains the architecture; the backend is still replaceable in principle (fallback: a right-sized op-VM implemented in-house — six op kinds, two wait ops).
Layering
The engine (orchestrator + op-VM + RPC) knows nothing about TMP specifically. Everything protocol-shaped lives in the Spec implementation and the per-chain modules/plugins. The RPC server fronts the APIs from Relayer § APIs: RegisterRelayRequest / UnregisterRelayRequest / QueryStatus for applications, AddSourceChain / RotateAuthKey / Pause / Resume for the operator.
The Spec abstraction
The engine is generic over a Spec trait that binds TMP's vocabulary. Every slot references its canonical definition. The trait introduces no new concepts:
| Slot | TMP binding | Canonical definition |
|---|---|---|
| Message identity | MessageID = (commitment_path, target_field, root) | Common-types § Message |
| Extraction predicate | RelayFilter + MatchPattern (encoding-aware match + extract) | Relayer § MatchPattern |
| Delivery registration | RelayRequest (filter + destination + application + request_id) | Relayer § RelayRequest |
| Consensus state | ConsensusState per tracked chain | Common-types § Consensus State |
| Checkpoint selection | compatible(checkpoint, block) — a relation, not a height offset | Consensus Verification § Jump-skip verification |
| Operations | VerifyMessage, UpdateConsensusState | Relayer § Mechanism |
Two crisp rules:
- Protocol specifics live only in the
Specimpl and per-chain modules. Anif protocol == …inside the op-VM or orchestrator means the logic is in the wrong place. - Deliberately absent: channels, connections, acks, timeouts. TMP has none of these; do not reintroduce IBC vocabulary through the implementation.
Note the contrast with IBC relayers: where Tendermint hardcodes "the root for height h lives at consensus state h+1", TMP replaces the height offset entirely with the compatibility relation: one stored checkpoint verifies many blocks (Consensus state update).
Composition axes
The five ChainModule slots from Relayer § Per-chain modules decompose along independent axes, so implementations are reused rather than multiplied (N chains × M VMs):
| Axis | Answers | Examples |
|---|---|---|
| Chain | Endpoints, finality, block structure of one chain | eth-mainnet, cosmoshub |
| Consensus | How to track + prove this consensus engine | casper-ffg, tendermint |
| Interface | How to talk to the destination VM (Router calls, tx encoding) | evm, cosmwasm |
| Client type | (consensus, interface) pairing deployed on a destination | casper-ffg/evm |
A state-query or tx-submission implementation for evm is written once and reused for every EVM-compatible chain; adding such a chain is config, not code.
Modules (read-only)
Uniform read-only capabilities, callable by any plugin and exposed through the RPC server. Modules never touch the queue.
| Module | Provides | Spec anchor |
|---|---|---|
Ingest | Finalized blocks / headers / attestations from canonical endpoints | Relayer § Source Data Ingestion |
Finality | The chain's finality predicate | Relayer § Per-chain modules |
DestQuery | Stored light-client checkpoints + Router registries on the destination | feeds EnsureCompatibleCheckpoint and the RegisterRelayRequest subscription precondition |
ClientBootstrap | Initial trusted checkpoint for a newly added chain | Syncing relayer § Bootstrap |
Ingested data is unverified and off-chain matching has no security weight (Relayer § Source Data Ingestion): the MatchPattern decides what to prove, but the destination trusts only the proof's finality + inclusion checks. A wrong match wastes prover cycles or produces a bundle the light client rejects — a liveness bug, never an unauthorized delivery.
Plugins (stateful, own queue)
Each plugin owns a topic queue and an interest filter over ops.
| Plugin | Responsibility | Spec anchor |
|---|---|---|
RequestRegistry | Own the RelayRequest store; serve RegisterRelayRequest / UnregisterRelayRequest; check the Router-subscription precondition via DestQuery | Relayer § APIs |
Extractor | Apply registered match_patterns to finalized blocks → (MessageID, Message) tasks tagged with request_id | Relayer § Message Extraction |
CheckpointManager | Track consensus-mirror divergence; schedule UpdateConsensusState per the update policy | Relayer § Consensus State Tracking, Consensus state update |
ProofPipeline | In zk-proof mode drive proving jobs on the prover daemon (§ Prover daemon below); in direct mode assemble the consensus-data bundle from the mirror; persist bundles transactionally | Relayer § Proof Generation, Stateful relayer |
Submitter | Choose the payload shape from the light client's caches (DestQuery), sign with auth_key, batch, broadcast, confirm, retry (a LeafNotCached / BlockNotVerified revert means resubmit with a fuller payload) | Relayer § Message Delivery |
Partition (optional) | Interest filter hash(MessageID) % N to shard work across instances | § Multi-relayer below |
Prover daemon (proverd), zk-proof mode only
For (source, destination) pairs in zk-proof mode, proving runs in a separate Go process, mirroring Union's production Voyager ↔ galoisd split: the relayer engine is Rust (to reuse voyager-vm/pg-queue), while the prover must be Go (gnark witness types). The two talk over gRPC; the seam keeps the two implementation choices independent: either side can be replaced without touching the other.
- galoisd-style async job API. Raw chain data in, proof out, with a
maxJobsadmission guard. Never exposed as a public service (DoS). - Witness building lives in the daemon, not the relayer. The
ProveRequestcarries raw beacon data (finality update, committee, participation); the Beacon-API→circuit adaptation happens where the gnark types live. - Chain-agnostic server core + per-chain adapters, one daemon instance per chain. Plumbing is shared as code; runtime is isolated per chain (GB-scale proving keys, blast radius, per-chain hardware profiles). The proto carries a
chain_idfrom day 1 so adding a source chain never breaks the wire contract. - Solidity-target proving is mandatory for on-chain-bound proofs (gnark's
WithProverTargetSolidityVerifier): default-option proofs verify in Go but revert on-chain due to the Pedersen-commitment hash mismatch.
Op-VM
The op algebra is unchanged from Voyager (and reused as code: the voyager-vm crate on pg-queue). Every unit of relayer work is a node in a persistent tree built from six op kinds:
| Op | Semantics |
|---|---|
Call | A primitive task: ingest, prove, submit, wait |
Data | An intermediate result flowing through the queue |
Seq | Strict ordering: the next child starts only when the prior one fully resolves |
Conc | Concurrency: children run independently |
Promise | Fan-in: collects the results of several child ops and aggregates them through a callback |
Queue | Routing: hands an op to a named plugin's topic queue |
TMP needs exactly two wait ops:
| Op | Blocks until | Resolves |
|---|---|---|
WaitForCompatibleCheckpoint { chain, block } | The destination light client holds a checkpoint compatible with block | The state-update ↔ message-proof ordering race (Relayer-challenges § 3.4) |
WaitForConfirmation { tx } | A submission is included + final on the destination | Reliable retry without double-spend of gas |
Voyager's source-side reorg machinery (WaitForTrustedHeight, height-relative delays) is not needed: TMP ingests finalized data only.
Crash invariant (the load-bearing rule; rationale in Stateful relayer):
A proof bundle is persisted in the same transaction that completes its proving job, before any broadcast attempt. Local delivery semantics are at-least-once; exactly-once is enforced on-chain by the Router's delivered-
MessageIDlog (Router § "Exactly-once delivery"), so a duplicate submission reverts harmlessly. The local dedupe on(request_id, MessageID)(Relayer § Security Considerations) saves gas; correctness never depends on it.
Worked example — relaying one message
The Extractor matches a leaf in finalized block B against the match_pattern of a registered request, yielding a (MessageID, Message) pair. The pair becomes a Seq of four ops:
- Ensure a compatible checkpoint (
Call). Query-before-prove: askDestQuerywhether the destination light client already holds a checkpoint compatible withB. If yes, no-op (another authorized relayer may have pushed it); if no, prove and submit anUpdateConsensusStateforB. - Wait for the checkpoint (
Call WaitForCompatibleCheckpoint). The ordering constraint: the message proof must verify against a checkpoint already installed on the light client. - Build the message payload (
Call). The expensive stage in zk-proof mode, plain packaging in direct mode: block finality plus message inclusion, one bundle (Relayer § Proof Generation). - Submit (
QueuetoSubmitter). Sign withauth_key, batch with sibling messages from the same block, broadcast, confirm, tagged with the originatingrequest_id.
Messages extracted from the same block share one finality proof: the ProofPipeline aggregates them in a Promise, so each additional message pays only the cheaper inclusion proof (Light Client § Cost amortization).
Multi-relayer (small authorized set)
Full design in Multi-relayer, the canonical doc for cross-instance coordination. Summary: relayers coordinate through the destination chain (membership from authorized_relayers, deterministic sharding by hash(MessageID) % N, completion from the Router's delivered-MessageID log), with timeout takeover for liveness and an optional self-verifying proof pool. The Partition plugin's interest filter implements the sharding; the Submitter implements query-before-prove and jitter. Fee-market / incentive equilibrium (ICS-29-style) is out of scope unless relaying becomes permissionless (Relayer-challenges § 1.1).
Build order
| Phase | Goal | Scope |
|---|---|---|
| 1 | Correctness, one path, real proofs | One source chain, one MatchPattern encoding, simple imperative loop. The first path is a zk-proof-mode light client, so proving cannot be deferred: the zk proof is the product. Get checkpoint compatibility + initial bootstrap right |
| 2 | Op-VM + recovery | Rehouse the flow into voyager-vm op trees on the pg-queue Postgres queue without changing behavior; add the two wait ops; QueryStatus as views over the queue |
| 3 | Modularize + prover fleet | Split modules/plugins along the axes; multiple proverd workers lease from the shared Postgres job queue |
| 4 | Multi-relayer | Query-before-prove dedup, Partition, shared proof cache |
| 5 | Scale | Recursive proof aggregation, additional ChainModules, proving-market integration if ever needed |
Pitfalls
- Protocol logic leaking into the op-VM/orchestrator, which loses the entire benefit of
Spec. - Re-importing IBC vocabulary (channels, acks, height offsets) the protocol deliberately removed; checkpoint selection is the compatibility relation, never
h+1. - Letting each relayer instance re-generate the same expensive proof. Make proofs a shared commodity before authorizing a second relayer.
- Adding distributed infrastructure beyond the single Postgres instance before a prover fleet exists. One
pg-queuedatabase is the committed baseline; measure, then scale. - Describing the relayer as trustless. TMP's permissioned-relayer trust model is deliberate defense-in-depth (Messenger § Security model); misstating it in code comments or docs undermines the security narrative.
Related documents
- Relayer: the component spec this architecture implements
- Stateful relayer: state classification, durability rules, crash consistency
- Syncing relayer: bootstrap, catch-up, subscription sync
- Relayer-challenges: open-challenge map and resolution order
- Multi-relayer: cross-instance coordination among the authorized set
- Messenger: the security model behind the permissioned-relayer decision