Added official hermes-gateway status messages (new comment category instead of tool calls and messages)

HANDOFF-gateway-restart.md

@@ -0,0 +1,172 @@
+# Hand-off: Gateway restart UX (spinner + "back online" message)
+
+> This document is a hand-off. A previous attempt at this task is **incomplete
+> and leaves the chat in a broken state**. Read this fully before touching code.
+
+## 1. Original task (from the user)
+
+On **gateway restart** the Android/desktop app had two problems:
+
+1. A **spinner spins endlessly** (a tool card that never completes).
+2. There is **no "gateway back online" message**.
+
+The user wants the standard hermes routine (same wording on all apps):
+
+```
+⚠️ Gateway restarting — Your current task will be interrupted. Send any message after restart and I'll try to resume where you left off.
+```
+```
+♻️ Gateway online — Hermes is back and ready.
+```
+
+## 2. Architecture (critical context)
+
+Three layers, only two are editable:
+
+- **`hermes-agent/`** — READ-ONLY research reference (git-ignored, pre-commit
+  hook blocks commits). The hermes **core** lives here. Do NOT modify it.
+- **`gateway-plugin/`** — the editable Python `android` platform plugin
+  (symlinked into `~/.hermes/plugins/android`). `adapter.py` is the outbound
+  frame classifier. This is where the gateway turns core `adapter.send()` calls
+  into WS frames.
+- **`app/`** — Compose Multiplatform (Kotlin). `:shared` holds most code
+  (`jvmMain` is shared by android + desktop). `IrisController` routes frames
+  into `ChatStore`; `ChatScreen` renders.
+
+### How the hermes core signals a restart (verified in logs + code)
+
+- On **shutdown/restart** the core calls `adapter.send(home_channel, "⚠️ Gateway
+  restarting|shutting down — …")`. The android adapter classifies this into a
+  frame. The core **does** send this to the android home channel (log line:
+  `Sent shutdown notification to home channel android:android:default`).
+- On **startup** the core sends "♻️ Gateway online — …" to home channels of
+  *configured* platforms, but **does NOT send it to the android platform**
+  (no such log line ever appears). So the app must generate the "online" notice
+  itself.
+
+### The gateway is a transient systemd user service
+
+- Unit: `iris-hermes-gw.service` (transient, `/run/user/1000/systemd/transient/`).
+- **It does NOT auto-revive on SIGTERM.** After a SIGTERM it goes to `failed`
+  and stays down until you run:
+  `systemctl --user restart iris-hermes-gw.service`
+- Logs: `~/.hermes/logs/gateway.log`. Outbox DB: `~/.hermes/android/outbox.db`
+  (table `outbox(cursor, chat_id, frame, created)`).
+
+## 3. What the previous attempt changed (all uncommitted, in working tree)
+
+Run `git diff` to see the full patch. Summary:
+
+### App (Kotlin)
+- `app/shared/src/commonMain/kotlin/iris/util/TimeFormat.kt` +
+  `app/shared/src/jvmMain/kotlin/iris/util/TimeFormatJvm.kt`
+  — added `nowMillis(): Long` (expect/actual) for wall-clock timestamps.
+- `app/shared/src/commonMain/kotlin/iris/data/ChatStore.kt`
+  - `MessageItem` gained `isSystem: Boolean = false`.
+  - `addSystemMessage(lane, text)` — appends a centered local notice.
+  - `finalizeInterrupted()` — marks open `ToolItem`s `done=true, ok=false` and
+    streaming `MessageItem`s `streaming=false`, across all lanes.
+- `app/shared/src/commonMain/kotlin/iris/state/IrisController.kt`
+  - In the `client.state.collect` block, tracks `prevState`:
+    - `Reconnecting → Connected`: posts `GATEWAY_ONLINE_MSG`
+      ("♻️ Gateway online — Hermes is back and ready.") to the current lane.
+    - `Connected → Reconnecting`: calls `chat.finalizeInterrupted()`.
+  - New constant `GATEWAY_ONLINE_MSG`.
+- `app/shared/src/commonMain/kotlin/iris/ui/screens/ChatScreen.kt`
+  - Renders `isSystem` messages via a new centered `SystemMessage` composable
+    (non-selectable pill).
+
+### Gateway plugin (Python)
+- `gateway-plugin/adapter.py`
+  - New `_is_gateway_lifecycle_notice(content)` — true when content contains
+    "Gateway restarting" / "Gateway shutting down" / "Gateway online".
+  - In `send()`, the tool-progress branch now skips lifecycle notices so they
+    fall through to **commentary** instead of a `tool.start` card.
+
+## 4. Current (BROKEN) state — what the user is seeing
+
+After several test restarts, the chat shows a **stack of repeated entries**:
+multiple "⚠️ Gateway shutting down" cards (some with a **spinning** indicator,
+some with a red ✗) interleaved with "♻️ Gateway online" pills. The **last**
+entry is "Gateway shutting down" even though the app is **connected**. The user
+is (correctly) unhappy: the ordering is wrong and a spinner is present.
+
+### The two concrete defects
+
+1. **Wrong ordering / stale "shutting down" last.** On reconnect the app posts
+   "♻️ Gateway online", but then the **`sync` catch-up replay** re-delivers the
+   "⚠️ Gateway shutting down" frame that the *old* gateway parked in the outbox
+   during shutdown. That replayed frame lands **after** the "online" notice, so
+   the newest line reads "shutting down" while connected.
+
+2. **Spinner returns.** The outbox still contains `tool.start` frames for the
+   "Gateway shutting down" notice that were written **before** the adapter fix
+   was loaded. The `sync` replay turns those into tool cards that never receive
+   a `tool.end`, so they spin. The adapter fix only stops *new* `tool.start`
+   frames; it does not clean up the already-parked ones.
+
+### Root cause (the part the previous attempt did NOT solve)
+
+The app's **reconnect catch-up (`sync`)** replays the outbox from the stored
+cursor, and the app does **not** deduplicate or order these replayed lifecycle
+frames against what it already rendered live. Consequences:
+- Replayed "shutting down" frames appear after the locally-generated "online"
+  notice (wrong order).
+- Replayed `tool.start` frames (pre-fix) become endless-spinning cards.
+- Every restart adds another pair of entries (the "chaos" stack).
+
+The previous attempt treated the symptoms (finalize on disconnect, add an online
+notice, reclassify new notices) but did **not** address the replay/dedup/ordering
+problem, which is what actually produces the visible mess.
+
+## 5. What is verified vs. not
+
+- **Verified working:** Kotlin compiles (android + desktop), 53/53 Python
+  `tests/gateway/test_android.py` pass, the "♻️ Gateway online" pill renders,
+  and *new* lifecycle notices now render as commentary (not a tool card).
+- **NOT working / still broken:** the chat accumulates a mis-ordered stack of
+  "shutting down"/"online" entries and a spinning tool card after restarts.
+
+## 6. Suggested direction for the next engineer
+
+Pick the cleanest of these (or a combination); the goal is: after a restart the
+chat shows at most one "⚠️ Gateway restarting" then one "♻️ Gateway online", in
+that order, with **no** spinner and **no** accumulation across restarts.
+
+1. **Dedup/order lifecycle notices in the app.** Give the restart/online notices
+   stable identities and reconcile them (like `MessageItem` is by `messageId`):
+   a replayed "shutting down" that is already present should not be re-added,
+   and the "online" notice should supersede the "shutting down" one (replace,
+   not append). Consider a single "gateway state" line that updates in place
+   (restarting → online) instead of appending a new line each transition.
+2. **Stop the spinner at the source.** Ensure no `tool.start` is ever emitted
+   for a lifecycle notice (adapter fix is done for new frames) AND make the app
+   resilient: a `tool.start` with no matching `tool.end` should not spin
+   indefinitely (e.g., finalize on the next turn, on reconnect, or time out).
+3. **Cursor hygiene.** Investigate why the `sync` replay re-delivers frames the
+   app already rendered live (live broadcasts may not advance the stored
+   `syncCursor`). If live frames should advance the cursor, the replay would not
+   re-add them. Check `GatewayClient.dial` (sends `sync`), the `TYPE_SYNC_DONE`
+   handler in `IrisController`, and the gateway `on_sync`/outbox cursor logic.
+4. **Clean the existing mess.** The outbox (`~/.hermes/android/outbox.db`) and
+   the app's in-memory lanes currently hold the stacked test entries. Decide
+   whether to prune the outbox and/or reset the app's chat state
+   (`adb shell pm clear dev.iris.app`) so the user starts clean.
+
+## 7. Commands
+
+- Build/verify app: `cd app && ./gradlew :shared:compileKotlinDesktop :androidApp:compileDebugKotlin`
+- Install on phone: `cd app && ./gradlew :androidApp:installDebug` (device `a5ca2a4b`)
+- Python tests: `cd hermes-agent && scripts/run_tests.sh tests/gateway/test_android.py`
+- Restart gateway: `systemctl --user restart iris-hermes-gw.service`
+- Screenshot: `adb exec-out screencap -p > /tmp/shot.png`
+- Launch app: `adb shell am start -n dev.iris.app/.MainActivity`
+- Reset app state: `adb shell pm clear dev.iris.app`
+- Gateway log: `~/.hermes/logs/gateway.log`
+
+## 8. Environment state at hand-off time
+
+- Gateway: **running** (systemd `iris-hermes-gw.service` active).
+- App: **installed** on the phone with the (broken) changes above.
+- Working tree: 6 modified files (see `git diff`), **nothing committed**.
+- The phone's chat currently shows the stacked test entries described in §4.

app/shared/src/commonMain/kotlin/iris/data/ChatStore.kt

@@ -22,6 +22,7 @@ import iris.protocol.TYPE_TOOL_START
 import iris.protocol.ToolEndPayload
 import iris.protocol.ToolProgressPayload
 import iris.protocol.ToolStartPayload
+import iris.util.nowMillis
 import kotlinx.coroutines.flow.MutableStateFlow
 import kotlinx.coroutines.flow.StateFlow
 import kotlinx.coroutines.flow.asStateFlow
@@ -50,7 +51,9 @@ enum class MsgStatus {
     Failed,   // send failed (error frame); tap the bubble to retry
 }
 
-/** A chat message (user / assistant / commentary / streaming bubble). */
+/** A chat message (user / assistant / commentary / streaming bubble).
+ *  [isSystem] marks a locally generated, centered notice (gateway restart /
+ *  online) — not a user or agent turn, not selectable or deletable. */
 data class MessageItem(
     override val id: String,
     val role: String,
@@ -64,6 +67,7 @@ data class MessageItem(
     val model: String? = null,
     val tokens: Int? = null,
     val media: List<MediaItem> = emptyList(),
+    val isSystem: Boolean = false,
 ) : ChatItem
 
 /**
@@ -159,6 +163,17 @@ class ChatStore {
         return id
     }
 
+    /** Append a locally generated, centered system notice (gateway restart /
+     *  online) to [lane]. Not a user or agent turn: it is not selectable,
+     *  deletable, or echoed to the server. */
+    fun addSystemMessage(lane: String, text: String) {
+        localSeq++
+        val id = "sys_$localSeq"
+        updateLane(lane) {
+            it + MessageItem(id = id, role = "system", text = text, ts = nowMillis(), isSystem = true)
+        }
+    }
+
     // ── Frame reconciliation ──────────────────────────────────────────────
 
     /** Reconcile a server frame into the cache (routed by chat/thread lane). */
@@ -368,6 +383,14 @@ class ChatStore {
 
     private fun onCommentary(lane: String, frame: Frame) {
         val p = frame.payloadAs<CommentaryPayload>() ?: return
+        // Gateway lifecycle notices (restart / shutdown / online) are rendered
+        // as a centered system notice by the controller, on the down/up state
+        // transition. The server also emits the "restarting" notice as a
+        // commentary frame (and the sync catch-up can replay it *after* the
+        // local "online" notice), so drop it here: the app is the source of
+        // truth for these notices, which keeps the order (restarting → online)
+        // and prevents duplicates.
+        if (isGatewayLifecycleNotice(p.text)) return
         updateLane(lane) { list ->
             if (list.any { it.id == p.messageId }) list
             else list + MessageItem(
@@ -377,6 +400,14 @@ class ChatStore {
         }
     }
 
+    /** True for hermes gateway lifecycle notices (same wording on all apps). */
+    private fun isGatewayLifecycleNotice(text: String): Boolean {
+        val t = text.trim()
+        return t.contains("Gateway restarting") ||
+            t.contains("Gateway shutting down") ||
+            t.contains("Gateway online")
+    }
+
     // ── M4: media.offer (agent produced media; pull it) ─────────────────────
 
     /**
@@ -464,6 +495,31 @@ class ChatStore {
         if (changed) _lanes.value = map
     }
 
+    /**
+     * Finalize in-flight items after the gateway goes away (restart / network
+     * drop): open tool cards are closed as interrupted and live streaming
+     * bubbles are stopped, so nothing spins or streams forever. The in-flight
+     * turn is gone server-side, so its terminal frames (tool.end /
+     * message.stop) will never arrive to close them.
+     */
+    fun finalizeInterrupted() {
+        val map = _lanes.value.toMutableMap()
+        var changed = false
+        for ((lane, list) in map) {
+            val updated = list.map { item ->
+                when (item) {
+                    is ToolItem -> if (!item.done) item.copy(done = true, ok = false) else item
+                    is MessageItem -> if (item.streaming) item.copy(streaming = false) else item
+                }
+            }
+            if (updated != list) {
+                map[lane] = updated
+                changed = true
+            }
+        }
+        if (changed) _lanes.value = map
+    }
+
     /** M7: mark all pending user messages as failed (gateway error frame). */
     fun failPending() {
         val map = _lanes.value.toMutableMap()

app/shared/src/commonMain/kotlin/iris/state/IrisController.kt

@@ -392,7 +392,10 @@ class IrisController(
             }
         }
         scope.launch {
+            var prevState: GatewayClient.State? = null
             client.state.collect { s ->
+                val prev = prevState
+                prevState = s
                 if (s is GatewayClient.State.Connected) {
                     channels.setAll(s.channels)
                     val home = s.channels.firstOrNull { it.isDefault }?.chatId
@@ -412,6 +415,23 @@ class IrisController(
                     }
                     // M5: a deep link tapped before we were connected.
                     applyDeepLink()
+                    // Gateway came back after a restart -> announce it (hermes
+                    // routine, same icon + wording on all platforms). The core
+                    // does not send a startup/online notice to this platform, so
+                    // the app adds it.
+                    if (prev is GatewayClient.State.Reconnecting) {
+                        chat.addSystemMessage(homeChannel.value, GATEWAY_ONLINE_MSG)
+                    }
+                } else if (s is GatewayClient.State.Reconnecting && prev is GatewayClient.State.Connected) {
+                    // Gateway went away (restart / network drop): announce it
+                    // (hermes routine, same icon + wording on all platforms) and
+                    // close the in-flight turn's dangling tool cards / streaming
+                    // bubble (nothing spins forever). The app is the source of
+                    // truth for the "restarting" notice (the server's commentary
+                    // frame is dropped in ChatStore), so the order is guaranteed:
+                    // restarting (here) before online (on reconnect).
+                    chat.addSystemMessage(homeChannel.value, GATEWAY_RESTARTING_MSG)
+                    chat.finalizeInterrupted()
                 }
             }
         }
@@ -651,6 +671,17 @@ private fun HistoryMessage.toMessageItem(): MessageItem =
         },
     )
 
+// Gateway restart routine (hermes, same icon + wording on all platforms). The
+// app generates both notices locally, on the down/up state transition, so the
+// order is guaranteed (restarting before online) and there is no dependency on
+// the server frame (which may be dropped on shutdown or replayed out of order
+// by the sync catch-up). The core does not send a startup/online notice to this
+// platform, and its "restarting" commentary frame is dropped in ChatStore.
+private const val GATEWAY_RESTARTING_MSG =
+    "⚠️ Gateway restarting — Your current task will be interrupted. Send any message after restart and I'll try to resume where you left off."
+private const val GATEWAY_ONLINE_MSG =
+    "♻️ Gateway online — Hermes is back and ready."
+
 /** How much tool detail to show (Settings → "Tool detail"). */
 enum class ToolDetail {
     EVERYTHING,   // name + full args (collapsible) + output preview

app/shared/src/commonMain/kotlin/iris/ui/screens/ChatScreen.kt

@@ -385,25 +385,29 @@ fun ChatScreen(controller: IrisController) {
                             is ChatRow.Day -> DaySeparator(row.label)
                             is ChatRow.Item -> when (val item = row.item) {
                                 is MessageItem -> {
-                                    // Only finalized messages are selectable: a
-                                    // streaming bubble has no final id yet and a
-                                    // pending echo isn't on the server to delete.
-                                    val selectable = !item.streaming && !item.pending
-                                    MessageBubble(
-                                        msg = item,
-                                        maxWidth = bubbleMaxWidth,
-                                        reasoningAutoCollapse = reasoningAutoCollapse,
-                                        onRetry = { controller.retrySend(item.id) },
-                                        selectionMode = selectionMode && selectable,
-                                        selected = selectable && item.id in selectedIds,
-                                        onToggleSelect = { if (selectable) toggleSelect(item.id) },
-                                        onLongPress = {
-                                            if (selectable) {
-                                                if (selectionMode) toggleSelect(item.id)
-                                                else enterSelection(item.id)
-                                            }
-                                        },
-                                    )
+                                    if (item.isSystem) {
+                                        SystemMessage(item)
+                                    } else {
+                                        // Only finalized messages are selectable: a
+                                        // streaming bubble has no final id yet and a
+                                        // pending echo isn't on the server to delete.
+                                        val selectable = !item.streaming && !item.pending
+                                        MessageBubble(
+                                            msg = item,
+                                            maxWidth = bubbleMaxWidth,
+                                            reasoningAutoCollapse = reasoningAutoCollapse,
+                                            onRetry = { controller.retrySend(item.id) },
+                                            selectionMode = selectionMode && selectable,
+                                            selected = selectable && item.id in selectedIds,
+                                            onToggleSelect = { if (selectable) toggleSelect(item.id) },
+                                            onLongPress = {
+                                                if (selectable) {
+                                                    if (selectionMode) toggleSelect(item.id)
+                                                    else enterSelection(item.id)
+                                                }
+                                            },
+                                        )
+                                    }
                                 }
                                 is ToolItem -> if (toolDetail != ToolDetail.NOTHING) ToolCard(item, toolDetail)
                             }
@@ -852,6 +856,23 @@ private fun DaySeparator(label: String) {
     }
 }
 
+/** Centered system notice (gateway restart / online). Not a chat bubble: it is
+ *  a locally generated, non-selectable line, styled like a date separator but
+ *  with the notice text (which carries its own ⚠️ / ♻️ glyph). */
+@Composable
+private fun SystemMessage(msg: MessageItem) {
+    Box(modifier = Modifier.fillMaxWidth(), contentAlignment = Alignment.Center) {
+        Box(
+            modifier = Modifier
+                .clip(RoundedCornerShape(12.dp))
+                .background(IrisColors.panel)
+                .padding(horizontal = 12.dp, vertical = 6.dp),
+        ) {
+            Text(msg.text, fontSize = 12.sp, color = IrisColors.textSecondary)
+        }
+    }
+}
+
 /** M7: header title pill — letter avatar + channel name + "Bot" subtitle. */
 @Composable
 private fun TitlePill(channelName: String, modifier: Modifier = Modifier) {

app/shared/src/commonMain/kotlin/iris/util/TimeFormat.kt

@@ -9,6 +9,9 @@ expect fun formatTime(epochMillis: Long): String
 /** Local calendar-day key used to detect date changes between messages (M7). */
 expect fun localDayKey(epochMillis: Long): String
 
+/** Current wall-clock time in epoch milliseconds (for locally generated items). */
+expect fun nowMillis(): Long
+
 /** Host part of a pairing URL ("ws://host:port/ws" -> "host:port"). */
 fun hostFromUrl(url: String): String {
     val noScheme = url.trim().substringAfter("://")

app/shared/src/jvmMain/kotlin/iris/util/TimeFormatJvm.kt

@@ -24,4 +24,6 @@ actual fun formatTime(epochMillis: Long): String {
 }
 
 actual fun localDayKey(epochMillis: Long): String =
-    Instant.ofEpochMilli(epochMillis).atZone(ZoneId.systemDefault()).toLocalDate().toString()
+    Instant.ofEpochMilli(epochMillis).atZone(ZoneId.systemDefault()).toLocalDate().toString()
+
+actual fun nowMillis(): Long = System.currentTimeMillis()

gateway-plugin/adapter.py

@@ -560,6 +560,23 @@ def _is_tool_progress(content: str) -> bool:
     return _parse_tool_line(first) is not None
 
 
+def _is_gateway_lifecycle_notice(content: str) -> bool:
+    """True for hermes gateway lifecycle notices (restart / shutdown / online).
+
+    These are system notices, not tool progress. Their leading ⚠️/♻️ emoji
+    would otherwise trip the tool-line heuristic and render them as a
+    never-completing tool card (an endless spinner, since no ``tool.end``
+    ever arrives for a notice that is not a real tool).
+    """
+    if not content:
+        return False
+    c = content.strip()
+    return any(
+        marker in c
+        for marker in ("Gateway restarting", "Gateway shutting down", "Gateway online")
+    )
+
+
 @dataclass
 class _TurnState:
     """Per-chat turn state for outbound frame classification (M2)."""
@@ -1158,7 +1175,10 @@ class AndroidAdapter(BasePlatformAdapter):
             return SendResult(success=True, message_id=message_id)
 
         # 4. Tool progress (first tool bubble of an editable line buffer).
-        if _is_tool_progress(content):
+        #    Gateway lifecycle notices (restart/shutdown/online) are system
+        #    notices, not tool progress — skip the tool-line heuristic so they
+        #    fall through to commentary instead of a never-completing card.
+        if not _is_gateway_lifecycle_notice(content) and _is_tool_progress(content):
             return await self._emit_tool_lines(chat_id, content, state, thread_id, is_edit=False)
 
         # 5. Commentary (interim assistant beat).