msgrpc over MQTT 5 — frame layout

Status: implemented. MqttTransport speaks this by default (protocol: 5); protocol: 4 keeps the older $-delimited header for brokers that need it. Verified against a live broker with vanilla mqtt.js on the far side, in src/Mqtt5.test.ts.

Shared subscriptions (sharedGroup / replicaId) and bounded sessions (sessionExpirySeconds) are implemented too.

Why

Today an outsider wanting to call plant.writeSetpoint(1200) must publish, on msgrpc/v1/rpc/plantServer:

{"source":"hmi","target":"plantServer","time":1785187832623,"seq":0}$<msgpack>

with the msgpack decoding to a doubly-nested envelope, and must know that type: 'POST' means "call", that path is the instance name, and that replies correlate by payload.id on msgrpc/v1/rpc/hmi. None of that is discoverable, and in MQTT tooling it renders as an opaque blob.

MQTT 5 has request/response in the protocol: Response Topic says where to reply, Correlation Data matches reply to request. Moving to it makes a frame self-describing in any MQTT 5 client and in standard tooling, and unlocks two things that matter more than interop for control systems: message expiry and shared subscriptions.

Topics

topic	carries	subscribed by
<prefix>/req/<peer>	calls and subscribe/unsubscribe requests	peers that serve
<prefix>/rsp/<peer>	results and errors	peers that call
<prefix>/evt/<peer>	events pushed to a subscriber	peers that subscribe to events
<prefix>/presence/<peer>	retained online / offline (unchanged)	all

Requests are on their own topic because shared subscriptions only make sense there. Replicas of a server subscribe $share/<group>/<prefix>/req/plantServer and the broker distributes requests among them. If responses shared that topic they would be load-balanced too, and a reply meant for one requester would land on a replica instead.

Splitting rsp from evt costs one subscription and buys least-privilege ACLs: a pure client never subscribes to req, a pure server never subscribes to evt.

The default prefix moves msgrpc/v1 → msgrpc/v2. v1 and v2 peers therefore share a broker without seeing each other, and a bridge peer can run one transport of each during migration.

Encoding

MsgPack by default, JSON accepted. MsgPack sits between JSON and protobuf on size and parse cost without a schema toolchain, and has small allocation-light C implementations for constrained targets — which matters when the fleet includes embedded devices sending a lot of data.

• contentType states which is in use: application/msgpack or application/json.
• payloadFormatIndicator is 0 for msgpack, 1 for JSON, so tooling renders payloads correctly.
• A responder replies in the request's contentType. A JSON-speaking third party gets JSON back without negotiating anything, and source-rpc peers stay on msgpack throughout.

User properties

All msgrpc control fields are prefixed mr-, so a broker or gateway that injects its own user properties (clientid, username, peerhost and similar) cannot be mistaken for one of ours. The prefix is kept to three characters because every key is carried in full on every packet, and packet overhead is a real cost on constrained links.

MQTT permits a user property to repeat. A frame with any mr-* property present more than once is rejected, rather than taking the first or last — a duplicated control field is an ambiguity worth refusing, not resolving.

property	on	meaning
mr-v	all	frame format version, currently 3
mr-src	all	sending peer name
mr-kind	all	call | subscribe | unsubscribe | result | error | event | ticket
mr-path	call, subscribe, event	exposed instance name
mr-method	call, subscribe	method name
mr-event	event	event name
mr-code	error	RpcErrorCode
mr-ver	call, subscribe	contract version the caller declares
mr-ttl	call, subscribe	milliseconds the caller will still wait, counted from sending
mr-idem	call	names the command this is an attempt at, when the caller distinguishes the two
mr-fence	call	the owner generation the caller observed for mr-path, when it fences
mr-deferred	result	1 when this result is a receipt and the answer comes later as a ticket
mr-outcome	ticket	progress | resolved | rejected
mr-seq, mr-epoch	event	this emission's position in the server's count, and the incarnation it counts within
mr-nonce, mr-ts, mr-sig	signed frames	replay and signature fields

Why mr-fence had to exist

A fence is checked by being present. A responder that finds no fence on a call does not fall back to a weaker check — it applies none, and runs the command under whatever ownership holds the instance now. So a layout with no representation for a fence does not degrade the guarantee, it removes it, and silently: the caller sees an ordinary successful call and has no way to learn its fence never travelled.

That is what this layout did until frame version 3. mr-fence did not exist, toOutboundFrame dropped the payload's fence, and every fenced call over MQTT 5 arrived unfenced — including the queued and redelivered ones a fence exists for in the first place. The socket.io path carried it throughout, so the feature's own tests went on passing while the transport a plant actually runs on ignored it.

Why mr-ttl as well as messageExpiryInterval

They answer different questions. Expiry is the broker's: whole seconds, decremented while queued, and it stops at the moment of delivery. mr-ttl is the caller's own statement of how long it will wait, it is signed, and it survives being relayed onto a transport that is not MQTT at all.

A responder uses both. Expiry, which the broker rewrites, may only narrow the ttl and never extend it, so what is left when the two are combined is the caller's signed budget minus the time the broker actually held the message — measured by the broker, with nobody's clock compared to anybody else's.

A duration rather than a deadline is deliberate: an absolute time is only as good as the agreement between two clocks, and one of the peers on this network is a browser page whose clock belongs to whoever is sitting at it. A wrong clock would refuse every command that page sent, which is worse than the late execution this exists to prevent.

Request

topic                    msgrpc/v2/req/plantServer
responseTopic            msgrpc/v2/rsp/hmi
correlationData          <16 random bytes>
contentType              application/msgpack
payloadFormatIndicator   0
messageExpiryInterval    10                       # seconds, from mr-ttl rounded up
userProperties
  mr-v                   3
  mr-src                 hmi
  mr-kind                call
  mr-path                plant
  mr-method              writeSetpoint
  mr-ttl                 10000                    # ms the caller will still wait
  mr-fence               e-7f21c9                 # only when the caller fences on owner generation
  mr-nonce               <base64>                 # signed frames only
  mr-ts                  1785187832623            # signed frames only
  mr-sig                 <base64>                 # signed frames only
payload                  <msgpack of [1200]>      # the argument array, nothing else

correlationData replaces the id field. mr-src is retained even though responseTopic implies it, because identity has to be bound explicitly by the signature rather than inferred from a topic.

The Response Topic is where the answer goes. Not a topic derived from mr-src — a caller that subscribes somewhere of its own choosing is answered there, which is what MQTT 5 request/response means and what an outside implementer would expect. Two rules bound it, because the caller is choosing a topic somebody else will publish to:

• it must be a publishable topic: no wildcards, no control characters, and not under $;
• it must sit under the transport's prefix, which is the boundary broker ACLs are usually drawn on. allowResponseTopic replaces that rule where an installation needs something else.

A request naming a topic outside the rule is refused, not quietly answered on a derived topic: a caller waiting on the topic it named is not helped by a reply sent elsewhere.

For mr-kind: subscribe the payload is the argument array holding the event name, e.g. ["alarm"], so every request has one shape.

Response

topic                    msgrpc/v2/rsp/hmi        # whatever responseTopic said
correlationData          <echoed verbatim>
contentType              application/msgpack      # mirrors the request
userProperties
  mr-v                   3
  mr-src                 plantServer
  mr-kind                result
  mr-nonce, mr-ts, mr-sig                         # signed frames only
payload                  <msgpack of 1200>        # the return value, encoded bare

Errors keep the shape with mr-kind: error, an mr-code carrying the RpcErrorCode, and a payload of {name, message, stack?}:

userProperties   mr-v=3  mr-src=plantServer  mr-kind=error  mr-code=Forbidden
payload          <msgpack of {"name":"RpcError","message":"not permitted to call plant.writeSetpoint"}>

Putting the code in a property means an operator can see why a call failed in MQTT Explorer without decoding the payload.

Deferred answers, and the one place a correlation is reused

A method that answers later replies twice on one correlation: a receipt now, and the answer when the work finishes. The receipt is an ordinary result carrying mr-deferred: 1, and its payload is the ticket — an id and an expiry — rather than the answer:

userProperties   mr-v=3  mr-src=plantServer  mr-kind=result  mr-deferred=1
payload          <msgpack of {"id":"…","expiresAt":1785187892623}>

Everything after it is mr-kind: ticket on the same correlationData, with mr-outcome saying whether the exchange is over. progress may arrive any number of times; resolved and rejected arrive once and end it. A rejected ticket carries {name, message, stack?} the way an error does; the others carry the value.

userProperties   mr-v=3  mr-src=plantServer  mr-kind=ticket  mr-outcome=progress
payload          <msgpack of 50>

userProperties   mr-v=3  mr-src=plantServer  mr-kind=ticket  mr-outcome=resolved
payload          <msgpack of {"rows":100000}>

This is the only place the one-publish-one-correlation rule bends, and it bends deliberately. The spec refuses to represent a batch for exactly this reason — a batch has as many correlations as it has calls, so it would need a second pairing rule beside MQTT's own. A deferred call needs no second rule: it has one correlation and more than one publish against it, which correlation data already expresses, and mr-outcome says which publish ends the exchange. Nothing has to guess.

The consequence for a responder is that the response topic and content type must be held until the outcome, not until the first reply. Releasing them on the receipt sends every later answer to a derived topic in the responder's own encoding — so a caller that named its own reply topic gets its receipt where it asked and its actual answer somewhere it is not listening.

Event

topic            msgrpc/v2/evt/hmi
                 # no correlationData: unsolicited
userProperties
  mr-v           3
  mr-src         plantServer
  mr-kind        event
  mr-path        plant
  mr-event       alarm
  mr-seq         41                               # when the server counts this event
  mr-epoch       e-3f9c                           # the incarnation that count belongs to
  mr-nonce, mr-ts, mr-sig                         # signed frames only
payload          <msgpack of ["high pressure"]>   # the emit argument array

mr-seq and mr-epoch are what let a subscriber say "gapless" rather than only "saw nothing": consecutive counts within one epoch prove nothing fell between two emissions. The count runs whether or not anyone is subscribed, which is the point — a counter that only advanced while someone watched could never describe the gap. The epoch bounds the promise honestly, because a sequence orders within one server incarnation and says nothing across a restart, so a subscriber seeing a new epoch knows to treat its held cursor as unknowable rather than to subtract and get a plausible number.

Signing

The signature must cover everything that decides what a frame means and where it goes. Since the topic now carries the addressing, it is signed rather than a target field:

signedInput = utf8(JSON.stringify([
    v, topic, responseTopic, src, kind, path, methodOrEvent, correlation,
    contentType, code, contractVersion, ttl, idempotencyKey, fence,
    deferred, outcome, seq, epoch, ts, nonce
])) || payload

Fields are signed positionally by value, so the mr- property naming does not enter the canonical form and renaming a property later would not silently change what verifies. Absent fields are ""; correlation is "" for events. A JSON array fixes order and escapes values, so no combination of names can be made to look like a different frame. v is included so a later format revision cannot be made to verify under these rules.

Everything the receiver acts on is covered. Version 1 left out contentType, on the reasoning that it only says how to read bytes that are themselves signed — so altering it could make a payload fail to parse but never change what was authorised. That reasoning is wrong, and the counterexample is one byte long: 0x31 is the JSON text "1", which is the number 1, and a MsgPack positive fixint, which is 49. Both parse. Both verified. Flipping one unsigned property turned a signed writeSetpoint(1) into a signed writeSetpoint(49).

The same argument covers the rest of what version 2 added: code decides what a caller does about a failure, contractVersion decides whether the call is accepted at all, responseTopic decides where the answer is published, ttl decides whether a command that is already too late still runs, and mr-idem decides whether a command that has already run runs again.

Version 3 adds fence, and it is the sharpest case of the rule rather than an exception to it. Every other signed field can be changed to change the meaning of a frame; mr-fence only has to be removed. An unsigned fence would mean anyone on the path could turn a command that was meant to be refused under a new ownership into one that executes, by deleting a property — no key, no forgery, and nothing at either end to notice.

Version 3 also applies the rule on the answering side, where it had never been applied at all. mr-deferred decides whether a caller keeps waiting — clear it and the caller settles with the receipt in place of the answer, set it on an ordinary result and the caller hangs until its deadline for a ticket nobody will send. mr-outcome is the entire meaning of a ticket: rewriting resolved to progress strands the caller, and the reverse settles it with a value the work never produced. mr-seq and mr-epoch are the arithmetic behind a gaplessness claim, so rewriting them can close a real gap or open an imaginary one. All four are covered.

messageExpiryInterval is deliberately not signed, because the broker is required to decrement it in flight and a signature over it would break on the first queued message. Nothing is lost: it may only narrow the signed mr-ttl, so rewriting it can delay or drop a frame — which anyone able to rewrite it could do anyway — but cannot buy a stale command more time.

Replay protection is unchanged: mr-nonce plus the mr-ts freshness window, with messageExpiryInterval as defence in depth at the broker.

Session and delivery

	MQTT 3.1.1 (today)	MQTT 5
server session	clean: false, never expires	cleanStart: false + sessionExpiryInterval
client session	clean: true, no queueing	cleanStart: false + short expiry, so queueing without permanent broker litter
stale requests	delivered late, executed	dropped by the broker at messageExpiryInterval
server HA	not possible	shared subscription on req

messageExpiryInterval closes a real hole: a request queued for a persistent server session can arrive long after the caller timed out, and the server executes it. It is not a duplicate, so duplicate suppression does not help.

The expiry is taken from mr-ttl, which is the caller's own timeout. The two used to be set independently — a ten-second call timeout against a thirty-second expiry — so a request could be delivered and executed twenty seconds after the operator had already been told the call failed. For a read that is wasted work; for start pump or reset fault it is a machine moving when nobody expects it to.

A responder that is handed a request should therefore check the budget immediately before running the method, not only on arrival. The broker's expiry covers the queue in front of the broker; it says nothing about a request that arrived promptly and then waited on something slow inside the process serving it.

What a third party has to implement

A responder serving one namespace:

1. Subscribe <prefix>/req/<name>.
2. On a message, read mr-path and mr-method and decode the payload as an argument array using contentType.
3. If mr-ttl is present, stop and answer mr-code=Timeout when more than that many milliseconds have passed since the message arrived — less whatever the broker already deducted from messageExpiryInterval. Check it just before running the method, not on arrival. If mr-idem is present and the method is one a repeat would change something with, look the key up before running and answer from the recorded outcome if it is there. If mr-fence is present and you keep a record of who owns mr-path, compare the two and answer mr-code=OwnershipChanged on any difference — including when you hold no record at all, which fails closed rather than running a command whose fence you cannot check.
4. Publish the result to the packet's responseTopic, echoing correlationData, with mr-kind=result and the same contentType. If the work will finish later, mark that result mr-deferred=1 and send the answer afterwards as mr-kind=ticket with an mr-outcome — on the same correlation and the same response topic, which means keeping both until the outcome rather than releasing them after the receipt.

No msgrpc framing, no $ splitting, no nested envelope. A caller is the mirror image. A third party that prefers JSON simply sets contentType: application/json and gets JSON replies.

Known limits

• Shared subscriptions suit stateless calls, not event subscriptions. A client subscribing to events registers with whichever replica received the request, and only that replica will emit to it. Event fan-out across replicas needs shared state, and is out of scope here.
• A resumed session delivers its queue the instant it connects. Instances therefore have to be exposed before ready() is awaited; anything registered afterwards is too late for the requests that were queued while the server was down, and those callers get ClassNotFound.
• Replicas do not announce presence. One replica's will would declare the whole shared name offline while its siblings were still serving, so they observe presence without publishing it.
• Duplicate suppression stays per-replica. A QoS 1 redelivery that lands on a different replica after one dies would not be recognised as a repeat. Exactly-once across replicas needs a shared store.
• Interop and signing pull against each other. Full third-party participation on a signed topic means publishing this canonicalisation so outsiders can implement it. Cheaper to reserve signing for links crossing a trust boundary and rely on broker ACLs elsewhere.
• Requires an MQTT 5 broker. EMQX 5 and Mosquitto 2.x are fine; some embedded brokers are 3.1.1 only.

Implementation shape

The RPC handlers build a Message object that is msgpack-encoded and then framed with a $ header, so a transport only ever sees opaque bytes. MQTT 5 needs structured access to kind, path, method, correlation and arguments.

The transport-independent frame this section called for now exists, as RPC/Frame.ts. It is the flat form of the protocol — kind, correlation, path, method, event, and the fields a receiver acts on — and Transports/Mqtt5Frame.ts holds only what MQTT calls each of them. The connection-oriented transports - socket.io and SignalR - map to the same frame, as the flat layout, so the wire formats differ in their framing and share their vocabulary.

Splitting the two was not tidying. While one file held both the protocol and one transport's names for it, "add a field to the protocol" and "decide what MQTT calls it" were the same act — and a field could therefore be added to the payload, honoured by socket.io, and never noticed to be missing here. That is precisely how the owner fence, the deferred marker, the ticket kind and the event cursor all came to work on one transport and not the other. The rule that replaces it is in the file: anything a Message can carry has to be representable in the frame, and a payload field that a receiver acts on belongs there before it belongs in any transport.

Decisions

decision	choice	reasoning
topic split	three: req / rsp / evt	shared subscriptions require req alone; the rsp/evt split buys least-privilege ACLs
default encoding	msgpack, JSON accepted, reply mirrors request	between JSON and protobuf on size and parse cost with no schema toolchain, and implementable on tiny embedded devices carrying a lot of data
property names	mr- prefixed	collision-proof against broker-injected properties; short because every key rides on every packet
migration	default prefix → msgrpc/v2	v1 and v2 peers coexist on one broker; a bridge peer can run both