app: smooth streaming markdown via incremental parser + typewriter reveal
- MarkdownText: streaming branch on the library's StreamingMarkdownState (rememberStreamingMarkdownPipeline) — incremental append, no full re-parse per snapshot; static path keeps retainState=true - client-side reveal: buffer snapshots, reveal at a steady rate (streamCharsPerSecond = 60/smoothness, 300..75 c/s) with rate-relative catch-up (jump when backlog > 2s of reveal time) - keep the streaming renderer alive after message.stop until the reveal catches up (useStreaming gate); artifact card waits for the same - cursor drawn via annotator, never part of the parse input - new 'Streaming speed' setting (0.2-0.8s) in Settings, persisted per device via SecureStore, live-updatable - docs/05-streaming: rate mapping, catch-up, wait-for-reveal semantics
This commit is contained in:
1 parent
c7a16d51e3
commit
83a67f6fb1
10 files changed
+555
-100
No files matched your search
+46
-1
@@ -12,11 +12,13 @@ produced by the gateway and rendered by the app.
|
||||
**Gateway side.** The agent's `stream_delta_callback` feeds a
|
||||
`GatewayStreamConsumer` (`gateway/stream_consumer.py:156`). The consumer
|
||||
accumulates text and, at intervals / thresholds, calls:
|
||||
|
||||
- `adapter.send(chat_id, text)` — first time a bubble is created.
|
||||
- `adapter.edit_message(chat_id, message_id, text)` — subsequent updates
|
||||
(each carries the **full** accumulated text).
|
||||
|
||||
**Adapter → frames.**
|
||||
|
||||
- First `send()` of a turn segment → `message.start {message_id, role}`.
|
||||
- Each `edit_message()` → `message.update {message_id, text}` (full text).
|
||||
- Segment/turn finalization → `message.stop {message_id, final_text, reasoning?,
|
||||
@@ -27,7 +29,44 @@ replace the bubble text (cheap: it's a full snapshot). On `message.stop`,
|
||||
finalize (attach reasoning/model/tokens footer, stop the cursor). Auto-scroll
|
||||
while the user is at the bottom.
|
||||
|
||||
**Smooth streaming (app-side rendering).** Re-parsing + re-laying-out the
|
||||
whole bubble on every update made streamed text unreadable (raw-markdown
|
||||
flashes, constant reflow). `MarkdownText` therefore renders streaming bubbles
|
||||
incrementally:
|
||||
|
||||
- **Reveal (typewriter):** the latest full snapshot is revealed at a steady
|
||||
rate instead of jumping per gateway update. The rate is user-adjustable:
|
||||
Settings → "Streaming speed" (`streamSmoothness`, 0.2–0.8 s between visible
|
||||
updates, lower = faster; `streamCharsPerSecond` maps it to chars/s:
|
||||
`60 / smoothness`, so 0.2 → 300 chars/s, 0.8 → 75 chars/s). Applied on the
|
||||
fly; a backlog exceeding ~2 s of reveal time is revealed at once (fast
|
||||
model / reconnect catch-up).
|
||||
- **Wait for the reveal:** the streaming renderer stays active until the
|
||||
reveal catches up, even after `message.stop` — a fast model that dumps the
|
||||
whole text in one or two frames still plays out the typewriter instead of
|
||||
jumping to the full message. Only then does the bubble switch to the static
|
||||
renderer (which also enables the HTML artifact card).
|
||||
- **Incremental parse:** the revealed prefix is fed as append chunks
|
||||
(`appendChunk`) into the library's `StreamingMarkdownState`
|
||||
(`rememberStreamingMarkdownState`, mikepenz 0.44.0) — an append-only parser
|
||||
that re-parses only the unstable tail. Settled blocks keep AST identity, so
|
||||
Compose never re-lays them out; only the tail re-renders per tick. A
|
||||
non-extension snapshot (rewrite) recreates the parser state and re-seeds it.
|
||||
- **Prefix-preserving transform:** `preserveNewlinesAsHardBreaksStreaming()`
|
||||
is like `preserveNewlinesAsHardBreaks()` but skips the still-incomplete last
|
||||
line, so the transform of a prefix is always a prefix of the transform of
|
||||
the whole (required for pure-append diffs). The static path keeps the
|
||||
original transform plus `retainState = true` (last formatted output stays
|
||||
visible during re-parses — no raw flash).
|
||||
- **Cursor:** the ▉ is appended to the last text leaf by the annotator (not to
|
||||
the parse input, which would break the append diff).
|
||||
|
||||
The gateway cadence (`edit_interval` / `buffer_threshold`, default 0.8 s /
|
||||
24 chars) is unchanged — smoothing happens entirely client-side, so it works
|
||||
with any gateway and per device.
|
||||
|
||||
**Streaming on/off.** Two levels:
|
||||
|
||||
- **Gateway side:** hermes `display.platforms.iris.streaming` (default
|
||||
follows global). When off, the app just gets one final `message` frame.
|
||||
- **App side (per device):** Settings → "Streaming" toggle (default on). When
|
||||
@@ -45,11 +84,13 @@ reference screenshot's "Reasoning:" panel with a copy button).
|
||||
**Gateway side.** hermes prepends reasoning to the final response when
|
||||
`show_reasoning` is enabled (`gateway/run.py:20089`). The format is stable and
|
||||
chosen by `reasoning_style` (`gateway/display_config.py:37`):
|
||||
|
||||
- `code` (default): `💭 **Reasoning:**\n```\n<reasoning>\n```\n\n<response>`
|
||||
- `blockquote`: `> 💭 **Reasoning:**\n> …\n\n<response>`
|
||||
- `subtext`: `-# 💭 Reasoning\n-# …\n\n<response>` (Discord-style)
|
||||
|
||||
**Plugin config.** Set for the `iris` platform:
|
||||
|
||||
```yaml
|
||||
display:
|
||||
platforms:
|
||||
@@ -59,12 +100,14 @@ display:
|
||||
```
|
||||
|
||||
**Adapter split.** In `send()`, detect the `code`-style prefix and split:
|
||||
|
||||
```
|
||||
prefix = "💭 **Reasoning:**\n```\n"
|
||||
# find the closing "\n```\n\n" after the prefix
|
||||
reasoning = text[len(prefix):close_idx]
|
||||
body = text[close_idx + len("\n```\n\n"):]
|
||||
```
|
||||
|
||||
Emit `message {reasoning: <reasoning>, text: <body>, …}`. If no prefix is found
|
||||
(reasoning off / no reasoning), emit `message {text: …}` with no `reasoning`.
|
||||
|
||||
@@ -89,6 +132,7 @@ gateway progress queue → `send_progress_messages` (`gateway/run.py:4603`) →
|
||||
|
||||
**Adapter → frames.** The adapter classifies tool activity (via turn-state +
|
||||
line format) and emits **structured** frames — not pre-formatted strings:
|
||||
|
||||
- `tool.start {index, name, preview, args}` — a tool call began.
|
||||
- `tool.progress {index, name, note}` — in-progress update (optional).
|
||||
- `tool.end {index, name, ok, duration, output_preview}` — completed.
|
||||
@@ -99,6 +143,7 @@ short tail; full tool output is **not** streamed (it lives in agent history and
|
||||
is reachable via search).
|
||||
|
||||
**App side — the verbosity setting** (Settings → "Tool detail"):
|
||||
|
||||
- **Everything** — show tool name, full args (collapsible), and output preview.
|
||||
- **Truncated** (default) — show `emoji name: "short preview"` one-liner,
|
||||
collapsible to expand.
|
||||
@@ -147,4 +192,4 @@ srv → message.start {message_id:m3}
|
||||
srv → message.update {m3, "The repo has…"}
|
||||
srv → message.stop {m3, final_text:"…", reasoning:"…", model:"…", tokens:42}
|
||||
srv → typing {on:false}
|
||||
```
|
||||
```
|
||||
Reference in new issue
Block a user