iris_x_hermes/docs/05-streaming.md

05 — Streaming, Reasoning, Tools, Intermediate Messages

This doc covers the four "agent transparency" features and exactly how each is produced by the gateway and rendered by the app.

The main hermes gateway delivers via the legacy callback path (not the ACP-only event-native render_message_event path). We map the legacy send/edit_message/progress calls to our WS frames.

5.1 Text streaming

Gateway side. The agent's stream_delta_callback feeds a GatewayStreamConsumer (gateway/stream_consumer.py:156). The consumer accumulates text and, at intervals / thresholds, calls:

• adapter.send(chat_id, text) — first time a bubble is created.
• adapter.edit_message(chat_id, message_id, text) — subsequent updates (each carries the full accumulated text).

Adapter → frames.

• First send() of a turn segment → message.start {message_id, role}.
• Each edit_message() → message.update {message_id, text} (full text).
• Segment/turn finalization → message.stop {message_id, final_text, reasoning?, model?, tokens?} (or a final message frame when not streaming).

App side. Maintain a live bubble per message_id. On message.update, replace the bubble text (cheap: it's a full snapshot). On message.stop, finalize (attach reasoning/model/tokens footer, stop the cursor). Auto-scroll while the user is at the bottom.

Smooth streaming (app-side rendering). Re-parsing + re-laying-out the whole bubble on every update made streamed text unreadable (raw-markdown flashes, constant reflow). MarkdownText therefore renders streaming bubbles incrementally:

• Reveal (typewriter): the latest full snapshot is revealed at a steady rate instead of jumping per gateway update. The rate is user-adjustable: Settings → "Streaming speed" (streamSmoothness, 0.2–0.8 s between visible updates, lower = faster; streamCharsPerSecond maps it to chars/s: 60 / smoothness, so 0.2 → 300 chars/s, 0.8 → 75 chars/s). Applied on the fly; a backlog exceeding ~2 s of reveal time is revealed at once (fast model / reconnect catch-up).
• Wait for the reveal: the streaming renderer stays active until the reveal catches up, even after message.stop — a fast model that dumps the whole text in one or two frames still plays out the typewriter instead of jumping to the full message. Only then does the bubble switch to the static renderer (which also enables the HTML artifact card).
• Incremental parse: the revealed prefix is fed as append chunks (appendChunk) into the library's StreamingMarkdownState (rememberStreamingMarkdownState, mikepenz 0.44.0) — an append-only parser that re-parses only the unstable tail. Settled blocks keep AST identity, so Compose never re-lays them out; only the tail re-renders per tick. A non-extension snapshot (rewrite) recreates the parser state and re-seeds it.
• Prefix-preserving transform: preserveNewlinesAsHardBreaksStreaming() is like preserveNewlinesAsHardBreaks() but skips the still-incomplete last line, so the transform of a prefix is always a prefix of the transform of the whole (required for pure-append diffs). The static path keeps the original transform plus retainState = true (last formatted output stays visible during re-parses — no raw flash).
• Cursor: the ▉ is appended to the last text leaf by the annotator (not to the parse input, which would break the append diff).

The gateway cadence (edit_interval / buffer_threshold, default 0.8 s / 24 chars) is unchanged — smoothing happens entirely client-side, so it works with any gateway and per device.

Streaming on/off. Two levels:

• Gateway side: hermes display.platforms.iris.streaming (default follows global). When off, the app just gets one final message frame.
• App side (per device): Settings → "Streaming" toggle (default on). When off, the app ignores message.start/message.update frames and materializes each reply as a single final message on message.stop (ChatStore.onMessageStop already handles a stop without a live bubble). The gateway keeps streaming — frames are broadcast to all devices, so this is a display preference like tool verbosity, not a wire flag.

5.2 Reasoning (shown before the message)

Requirement: reasoning appears as a block above the answer (like the reference screenshot's "Reasoning:" panel with a copy button).

Gateway side. hermes prepends reasoning to the final response when show_reasoning is enabled (gateway/run.py:20089). The format is stable and chosen by reasoning_style (gateway/display_config.py:37):

• code (default): 💭 **Reasoning:**\n```\n<reasoning>\n```\n\n<response>
• blockquote: > 💭 **Reasoning:**\n> …\n\n<response>
• subtext: -# 💭 Reasoning\n-# …\n\n<response> (Discord-style)

Plugin config. Set for the iris platform:

display:
  platforms:
    iris:
      show_reasoning: true
      reasoning_style: code     # we split on the code-fence form

Adapter split. In send(), detect the code-style prefix and split:

prefix = "💭 **Reasoning:**\n```\n"
# find the closing "\n```\n\n" after the prefix
reasoning = text[len(prefix):close_idx]
body      = text[close_idx + len("\n```\n\n"):]

Emit message {reasoning: <reasoning>, text: <body>, …}. If no prefix is found (reasoning off / no reasoning), emit message {text: …} with no reasoning.

Robustness: the split is a best-effort parse of a stable, gateway-owned format. If the format ever changes, the fallback is "no reasoning field, full text" — the app still shows the answer. Verified in M2 against the live gateway.

App side. ReasoningBlock composable: collapsible, header "💭 Reasoning", monospace body, a copy button (matches reference). Rendered above the message body. Collapsed by default if long; expanded tap.

5.3 Tool output (app controls verbosity)

Requirement: hermes supports several tool-display modes; the gateway sends full structured data, and the app chooses how much to show (everything / truncated / nothing).

Gateway side. Tool activity flows through tool_progress_callback → the gateway progress queue → send_progress_messages (gateway/run.py:4603) → adapter.send(). The adapter receives tool lines during a turn.

Adapter → frames. The adapter classifies tool activity (via turn-state + line format) and emits structured frames — not pre-formatted strings:

• tool.start {index, name, preview, args} — a tool call began.
• tool.progress {index, name, note} — in-progress update (optional).
• tool.end {index, name, ok, duration, output_preview} — completed.

index is a monotonic per-turn counter so the app correlates start→end. args is the full argument dict (the app truncates). output_preview is a short tail; full tool output is not streamed (it lives in agent history and is reachable via search).

App side — the verbosity setting (Settings → "Tool detail"):

• Everything — show tool name, full args (collapsible), and output preview.
• Truncated (default) — show emoji name: "short preview" one-liner, collapsible to expand.
• Nothing — suppress tool.* frames entirely (clean chat).

Rendering: a ToolCard per tool, grouped under the message it belongs to, with a spinner while tool.end hasn't arrived, a ✓/✗ on completion, and duration.

Classification detail: the exact way to distinguish a tool-progress send() from a regular send()/commentary is confirmed empirically in M2 by running the real gateway with a test WS client and observing the calls. The adapter keeps a per-chat turn state machine (turn active, current streaming id, last tool index) to make the classification deterministic.

5.4 Intermediate messages (commentary)

Requirement: show the agent's interim beats (e.g. "I'll inspect the repo first.") as distinct messages.

Gateway side. interim_assistant_callback → consumer on_commentary (gateway/stream_consumer.py:518) → delivered as its own message.

Adapter → frames. commentary {message_id, text}.

App side. Render as a dimmed / smaller bubble, visually distinct from final answers (e.g. reduced opacity, no model footer). It reads as a "beat" in the conversation, not a full reply.

5.5 Typing indicator

send_typing → typing {on:true}; the app shows an animated indicator in the chat header / above the composer until typing.stop or the first message.start.

5.6 Frame sequence for a typical turn

app  → message.send {text:"summarize the repo"}
srv  → typing {on:true}
srv  → message.start {message_id:m1}
srv  → message.update {m1, "Let me look…"}        (streaming)
srv  → tool.start   {index:1, name:terminal, args:{command:"ls -la"}}
srv  → tool.end     {index:1, name:terminal, ok:true, duration:0.4}
srv  → commentary   {m2, "Found 12 files."}
srv  → message.start {message_id:m3}
srv  → message.update {m3, "The repo has…"}
srv  → message.stop {m3, final_text:"…", reasoning:"…", model:"…", tokens:42}
srv  → typing {on:false}