iris_x_hermes/docs/04-wire-protocol.md

04 — Wire Protocol

JSON frames over a single WebSocket. One connection per device. Text frames are JSON; media travels as binary frames (chunked) referenced by a header frame.

Envelope

Every frame:

{
  "v": 1,
  "id": 42,                 // optional; present on requests + their responses
  "type": "message",        // frame type (below)
  "chat_id": "default",   // optional; scope for chat-scoped frames
  "thread_id": "t_123",           // optional
  "payload": { }             // type-specific object
}

• v — protocol version (currently 1). Server rejects unknown major versions.
• id — request id (client-chosen). Responses/acks echo it. Events have no id.
• chat_id / thread_id — top-level for convenience; may also be in payload.
• cursor — outbox cursor the frame was parked under. Present only on frames replayed by sync (live frames carry none). The app compares it against last_pushed_cursor from hello.ack to skip re-notifying frames that already woke the device via push (08-push.md §8.7).
• Unknown types are ignored (forward-compat); unknown payload fields ignored.

Binary media frames are not JSON. A media transfer is: one JSON header frame (media.upload.start / media.pull ack) followed by raw binary frames, then a JSON media.upload.end / final ack. See 07-media.md.

Server → App (events / responses)

hello.ack

Pairing succeeded.

{"type":"hello.ack","payload":{
  "server_caps":{"streaming":true,"reasoning":true,"tools":true,"media":true,
                 "search":true,"push":"fcm","pickers":true},
  "sync_cursor":1042,
  "last_pushed_cursor":1040,
  "channels":[{"chat_id":"default","name":"Default","kind":"default","is_default":true}]
}}

last_pushed_cursor is the highest outbox cursor already delivered to THIS device via the push backend (0 = never). The app skips system notifications for sync-replayed frames with cursor <= last_pushed_cursor — they already woke the device via push (dedupe, 08-push.md §8.7).

message

A final / standalone message.

{"type":"message","chat_id":"default","thread_id":null,
 "payload":{
   "message_id":"m_9001","role":"assistant",
   "text":"Here is the answer…",
   "reasoning":"The user asked… so I will…",   // optional; render ABOVE text
   "media":[{"media_id":"md_5","kind":"video","mime":"video/mp4","size":123456,
             "filename":"clip.mp4"}],           // optional
   "reply_to":"m_8999",                          // optional
   "model":"qwen3-27b","tokens":11,"ts":1724000000000,
   "runtime":{"model":"qwen3-27b","context_pct":38,"cwd":"~",
              "latency":22.5,"cost":0.0012}      // optional; see below
 }}

role ∈ user | assistant | system | cron. reasoning present only when the agent produced reasoning and show_reasoning is on.

runtime (optional) is the structured runtime-metadata footer the app renders under final assistant messages (Telegram-style). The gateway ALWAYS sends it on final assistant messages; whether/what is shown is a per-app setting (Settings → Runtime footer), NOT a hermes config. Keys (all optional; absent when the data is unavailable, e.g. local models have no cost):

• model — bare model id, vendor prefix dropped (gpt-5.4)
• context_pct — last-call context occupancy, 0-100 (int)
• cwd — home-relative working dir (~)
• latency — wall-clock turn duration, seconds (float)
• cost — turn cost, USD (float)

message.start / message.update / message.stop

Streaming a bubble. update carries the full current text (app replaces).

{"type":"message.start","chat_id":"…","payload":{"message_id":"m_9002","role":"assistant"}}
{"type":"message.update","chat_id":"…","payload":{"message_id":"m_9002","text":"partial…"}}
{"type":"message.stop","chat_id":"…","payload":{"message_id":"m_9002","final_text":"full…",
    "reasoning":"…","model":"…","tokens":11,
    "runtime":{"model":"…","context_pct":38,"cwd":"~","latency":22.5}}}

message.deleted

The given message(s) were deleted from a chat/thread. Response to a message.delete request (id set) and broadcast to every device so all of them drop the message(s) from their cache. Also outboxed, so a device that was offline learns of the deletion on its next sync.

{"type":"message.deleted","id":30,"chat_id":"default","thread_id":null,
 "payload":{"message_ids":["m_9001","m_9002"]}}

commentary

Intermediate assistant beat (between tool iterations).

{"type":"commentary","chat_id":"…","payload":{"message_id":"m_9003","text":"Let me inspect the repo first."}}

tool.start / tool.progress / tool.end

Structured tool events. The app decides how much to show (everything / truncated / nothing).

{"type":"tool.start","chat_id":"…","payload":{
   "index":3,"name":"terminal","preview":"pytest -q","args":{"command":"pytest -q"},"emoji":"💻"}}
{"type":"tool.progress","chat_id":"…","payload":{"index":3,"name":"terminal","note":"running…"}}
{"type":"tool.end","chat_id":"…","payload":{"index":3,"name":"terminal","ok":true,"duration":12.4,
   "output_preview":"12 passed"}}

args may be large; the app truncates per its setting. output_preview is a short tail (full output is not streamed — it lives in agent history).

emoji is a cosmetic per-tool glyph resolved server-side via hermes' get_tool_emoji (active-skin overrides, then the tool registry's per-tool emoji — e.g. read_file 📖, write_file ✍️, terminal 💻). Omitted when the tool is unknown, so the app falls back to its own default glyph.

typing / typing.stop

{"type":"typing","chat_id":"…","payload":{"on":true}}

notification

In-app banner (foreground) and/or push mirror (background).

{"type":"notification","chat_id":"…","payload":{
   "kind":"channel_renamed","title":"ARIA","body":"Renamed topic to …","ts":1724000000000}}

kind ∈ channel_renamed | channel_created | cron | approval | clarify | generic.

picker.model / picker.choice / picker.clarify / picker.approval / picker.confirm

Interactive prompts. App renders a native picker; answers via picker.select.

picker.choice is implemented (the generic finite-choice menu used by /reasoning, /fast, and any future finite-choice slash command — hermes calls the adapter's send_choice_picker when the platform supports it). The server runs the command's selection callback on picker.select and delivers its reply as a normal message in the picker's chat. The frame is outboxed (a reconnecting device re-renders a still-pending picker); pending state is in-memory only, so a gateway restart expires it (a stale picker.select is a no-op). With no live device the adapter reports failure and hermes falls back to the text status card.

{"type":"picker.model","chat_id":"…","payload":{
   "picker_id":"pm_1","current_model":"qwen3-27b","current_provider":"local",
   "providers":[{"id":"local","label":"Local","models":[{"id":"qwen3-27b","label":"Qwen3 27B"}]}]}}
{"type":"picker.choice","chat_id":"…","payload":{
   "picker_id":"pc_1","title":"Reasoning effort","choices":[
     {"value":"low","label":"Low"},{"value":"high","label":"High","is_current":true}]}}

channel.list / channel.created / channel.renamed / channel.deleted

Channel directory updates. Broadcast to all connected devices (no explicit subscribe; the server pushes to every open WS).

{"type":"channel.created","payload":{"chat_id":"chan_7","name":"Cron Reports",
   "kind":"channel","parent_chat_id":null}}

channel.created may carry "auto":true for a thread the gateway minted itself for an incoming message (auto-threading): the name is an instant derived title, and a follow-up channel.renamed upgrades it to the model's title.

history

Response to a history request. Returns a page of messages for a chat/thread.

{"type":"history","id":20,"chat_id":"default","thread_id":null,
 "payload":{
   "messages":[
     {"message_id":"m_8990","role":"user","text":"Hi","ts":1723990000000},
     {"message_id":"m_8991","role":"assistant","text":"Hello!","reasoning":"…",
      "model":"qwen3-27b","tokens":8,"ts":1723990001000}
   ],
   "has_more":true,
   "oldest_message_id":"m_8990"
 }}

messages are ordered oldest → newest. Paginate with before_message_id in the request. The app uses this to populate the initial view when a channel is opened (complements sync, which only replays undelivered outbox frames).

commands.catalog

Request (app → server, empty payload) and response: the gateway's slash-command catalog for the app's / drawer. Derived from hermes' central COMMAND_REGISTRY (the same source the gateway help and the Telegram command menu use), restricted to commands available on gateway surfaces, plus plugin-registered commands. The app fuzzy-matches the typed prefix client-side (no commands.complete round-trip).

{"type":"commands.catalog","id":21,"payload":{
   "commands":[
     {"name":"/new","description":"Start a new session (fresh session ID + history)","args_hint":"[name]","category":"Session","aliases":["/reset"]},
     {"name":"/model","description":"Switch model","args_hint":"<provider/model>","category":"Configuration","aliases":[]},
     {"name":"/status","description":"Show session status","args_hint":"","category":"Info","aliases":[]}
   ]}}

name/aliases carry the leading slash; args_hint is the registry's argument placeholder (empty when the command takes none).

commands.complete

Response to a commands.complete request. Autocomplete matches for a typed prefix.

{"type":"commands.complete","id":22,"payload":{
   "prefix":"/mod",
   "matches":[
     {"name":"/model","description":"Switch model","args_hint":"<provider/model>"}
   ]}}

agent.busy / agent.idle

Agent lifecycle for a chat/thread. App shows a "thinking…" indicator on busy.

{"type":"agent.busy","chat_id":"default","thread_id":null,
 "payload":{"reason":"processing"}}
{"type":"agent.idle","chat_id":"default","thread_id":null,"payload":{}}

reason ∈ processing | tool | waiting_input | cron.

search.results

{"type":"search.results","id":7,"payload":{
   "query":"deploy","scope":"all","hits":[
     {"message_id":"m_123","chat_id":"chan_7","thread_id":null,
      "role":"assistant","snippet":"…deploy the service…","ts":1723900000000}]}}

media.offer

Agent-sent media is available; app pulls bytes.

{"type":"media.offer","chat_id":"…","payload":{
   "media_id":"md_5","kind":"video","mime":"video/mp4","size":123456,"filename":"clip.mp4"}}

read.receipt

The gateway acknowledges that the agent has received and started processing the user's message. The app uses it to show ✓✓ on user bubbles.

{"type":"read.receipt","chat_id":"default","payload":{"message_id":"m_9001"}}

Emitted to the originating connection when a message.send is accepted for processing (at the moment it is handed to the agent), for user-originated messages only.

status

Gateway health state. Broadcast to all connected clients at startup (state: "online") and to late joiners on hello.ack. The gateway also broadcasts state: "restarting" on its shutdown path (restart/stop), right before closing the sockets — the app posts the "Gateway restarting" chat notice immediately on that frame (the socket can take up to the ~20 s ping timeout to actually drop, so the notice must not wait for the disconnect); a plain network drop shows just the reconnect banner. degraded is reserved for future use.

{"type":"status","payload":{"state":"online"}}

state ∈ online | restarting | degraded.

error

{"type":"error","id":7,"payload":{"code":"not_found","message":"chat_id unknown"}}

code ∈ auth | not_found | rate_limited | media_too_large | unsupported | internal.

pong

Keepalive reply to ping.

App → Server (requests / actions)

hello

First frame; auth + caps.

{"type":"hello","payload":{
   "token":"<IRIS_TOKEN>","device_id":"dev_a1b2","device_name":"MIX 2S",
   "caps":{"min_protocol":1,"media":true,"push":"fcm"},
   "fcm_token":"<FCM token>","ntfy_topic":"<topic, if ntfy>"}}

message.send

Send text (or a /slash-command).

{"type":"message.send","id":10,"chat_id":"default","thread_id":null,
 "payload":{"text":"/model qwen3-27b","reply_to":"m_9001","media_refs":["mu_1"],
            "auto_thread":false}}

media_refs reference completed media.uploads to attach. auto_thread (optional, default false) asks the gateway to mint a fresh thread for the message (auto-threading, 06-channels-cron-search.md §6.3): honored only in a channel's flat lane (thread_id null) with non-empty text that is not a slash command. The gateway then broadcasts channel.created {auto:true} and the echo / agent turn carry the new thread_id.

media.upload.start / (binary) / media.upload.end

See 07-media.md.

{"type":"media.upload.start","id":11,"payload":{
   "media_ref":"mu_1","kind":"image","mime":"image/jpeg","size":204800,"filename":"a.jpg"}}
// … binary frames …
{"type":"media.upload.end","id":11,"payload":{"media_ref":"mu_1","sha256":"…"}}

media.upload.ack

Server → App response to media.upload.end: the ref is cached and may now be referenced in a message.send media_refs. Failures use error frames instead.

{"type":"media.upload.ack","id":11,"payload":{"ok":true,"media_ref":"mu_1"}}

media.pull

Request agent-sent media bytes.

{"type":"media.pull","id":12,"payload":{"media_id":"md_5"}}
// server replies: binary frames, then {"type":"media.pull.end","id":12,"payload":{"ok":true}}

picker.select

Answer an interactive picker.

{"type":"picker.select","id":13,"payload":{"picker_id":"pm_1","value":"local/qwen3-27b"}}

channel.create / channel.rename / channel.set_default / channel.delete

{"type":"channel.create","id":14,"payload":{"name":"Cron Reports","kind":"channel"}}
{"type":"channel.rename","id":15,"chat_id":"chan_7","payload":{"name":"Reports"}}
{"type":"channel.set_default","id":16,"chat_id":"chan_7","payload":{}}

channel.delete is a hard delete: the channel/thread row is removed from the directory and the lane's history is wiped from the outbox and the hermes session store (no search trace, not recoverable). Deleting a channel also removes its threads. The default channel cannot be deleted.

search

{"type":"search","id":17,"payload":{"query":"deploy","scope":"all"}}
{"type":"search","id":18,"payload":{"query":"deploy","scope":"chat","chat_id":"chan_7","thread_id":null}}

scope ∈ all | chat.

read.receipt

App → server: "user has viewed this message." Server stores the read state and broadcasts to other devices (for multi-device ✓✓ sync). The app uses it to mark messages as read locally (✓✓ on user bubbles).

{"type":"read.receipt","payload":{"chat_id":"default","message_id":"m_9001"}}

history

Load a page of messages for a chat/thread (initial open, scroll-up pagination).

{"type":"history","id":20,"chat_id":"default","thread_id":null,
 "payload":{"before_message_id":"m_8990","limit":50}}

before_message_id — return messages older than this (omit for newest page). limit — max messages (default 50, max 200).

message.delete

Completely delete the given message(s) from a chat/thread. The server removes them from the outbox (so history/sync no longer return them) and from the hermes session store (so no search trace survives and they are not recoverable), then broadcasts message.deleted to every device. Idempotent: a message already gone (pruned by retention) still yields a message.deleted broadcast so live caches drop it.

{"type":"message.delete","id":30,"chat_id":"default","thread_id":null,
 "payload":{"message_ids":["m_9001","m_9002"]}}

commands.catalog

Fetch the full slash-command catalog (for the Menü bottom sheet).

{"type":"commands.catalog","id":21,"payload":{}}

commands.complete

Autocomplete for a typed /prefix.

{"type":"commands.complete","id":22,"payload":{"prefix":"/mod"}}

agent.stop

Stop the current agent turn (abort generation / tool execution).

{"type":"agent.stop","id":23,"chat_id":"default","thread_id":null,"payload":{}}

agent.steer

Inject a steering message mid-turn (redirects the agent without a new turn).

{"type":"agent.steer","id":24,"chat_id":"default","thread_id":null,
 "payload":{"text":"Actually, focus on the error case."}}

sync

Reconnect catch-up. Replays undelivered outbox frames (frames sent while this device was offline). Does NOT load full history — use history for that.

{"type":"sync","id":19,"payload":{"cursor":1042}}
// server replays outbox frames with cursor > 1042, then {"type":"sync.done","id":19,"payload":{"cursor":1099}}

fcm.register

Update push token.

{"type":"fcm.register","payload":{"fcm_token":"<new>","ntfy_topic":"<topic>"}}

ping

Keepalive. {"type":"ping","payload":{"ts":1724000000000}} → pong.

Ordering & reliability

• Frames are ordered per connection (TCP/WS). Streaming message.update frames for a message_id are monotonic; the app may coalesce to the latest.
• Terminal frames (message, message.stop, message.deleted, tool.end, notification, picker.*, channel.*, agent.busy, agent.idle, history, commands.catalog, commands.complete) are never dropped under backpressure; only intermediate message.update/tool.progress are coalesced.
• Anything not delivered live goes to the outbox and is replayed by sync.
• Broadcast: channel directory events (channel.*) and read-receipts are pushed to all connected devices for that gateway (no subscribe step).
• Requests get exactly one response or error (matched by id).