106 lines
8.8 KiB
Markdown
106 lines
8.8 KiB
Markdown
# 01 — Architecture
|
|
|
|
## System diagram
|
|
|
|
```
|
|
┌────────────────────────────── User's machine (home server / PC) ─────────────────────────────┐
|
|
│ │
|
|
│ hermes gateway (ONE process) │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Agent core (run_agent.py) ── sessions (SQLite + FTS5) ── cron scheduler │ │
|
|
│ │ │ │ │
|
|
│ │ ▼ legacy stream callbacks (delta / tool-progress / commentary) │ │
|
|
│ │ ┌──────────────────────────────┐ ┌────────────────────────────────┐ │ │
|
|
│ │ │ android PLATFORM PLUGIN │◄───────►│ WebSocket SERVER (websockets) │ │ │
|
|
│ │ │ IrisAdapter │ JSON │ ws://host:8790/ws │ │ │
|
|
│ │ │ (BasePlatformAdapter) │ frames │ + media + FCM-token + pairing │ │ │
|
|
│ │ │ • send / edit / stream │ └───────────────┬────────────────┘ │ │
|
|
│ │ │ • media cache │ │ WSS │ │
|
|
│ │ │ • outbox (SQLite) │ │ │ │
|
|
│ │ │ • push (FCM / ntfy) │─────────────────────────┼──────────┐ │ │
|
|
│ │ └──────────────────────────────┘ │ │ │ │
|
|
│ └───────────────────────────────────────────────────────────┼──────────┼───────────┘ │
|
|
└───────────────────────────────────────────────────────────────┼──────────┼───────────────────────┘
|
|
│ │ push (FCM HTTP v1 / ntfy)
|
|
┌──────────────────┼──────────▼─────────┐
|
|
│ │ Google FCM cloud │
|
|
▼ │ │ │
|
|
┌────────────────────────┐ │ ▼ │
|
|
│ ANDROID APP │◄──┴── (wake) ┌──────────┐
|
|
│ (Kotlin / Compose) │ WSS │ PHONE │
|
|
│ • WS client (OkHttp) │◄───────────►│ MIX 2S │
|
|
│ • ExoPlayer │ │ (API 29) │
|
|
│ • FCM service │ └──────────┘
|
|
└────────────────────────┘
|
|
DESKTOP APP (Compose Multiplatform)
|
|
• same shared code, WSS to same server
|
|
• tray + OS notifications (no FCM), big-screen two-pane
|
|
```
|
|
|
|
## Process model
|
|
|
|
- **One `hermes gateway` process** hosts the agent core, the session store, the
|
|
cron scheduler, *and* our `iris` platform plugin. The plugin's WebSocket
|
|
server runs on the gateway's asyncio loop (started in `IrisAdapter.connect()`).
|
|
- **The app is a client.** It *initiates* the WS connection to the gateway
|
|
(outbound), so no inbound port is needed on the phone. For LAN/remote access
|
|
the user points the app at the gateway's LAN IP / Tailscale name / a WSS
|
|
tunnel (see `09-pairing-security.md`).
|
|
- **Push is the only inbound path to a sleeping phone**, and it goes through a
|
|
cloud relay (FCM or ntfy), not a direct connection.
|
|
|
|
## Why the *messaging gateway* is the connection point (not `tui_gateway`)
|
|
|
|
hermes has two "gateways": the **messaging gateway** (`hermes gateway`, which
|
|
serves Telegram/Discord/… and cron) and the **`tui_gateway`** JSON-RPC backend
|
|
(used by the TUI and the existing Electron desktop app). We deliberately use the
|
|
**messaging gateway** because:
|
|
|
|
1. **Cron delivery is native.** Cron jobs resolve `deliver=iris:<chat>[:<thread>]`
|
|
through the platform registry and call our adapter's `send()`. No bridging.
|
|
2. **`send_message` tool routing** works out of the box (plugin
|
|
`parse_target_ref_fn`).
|
|
3. **Slash commands** are dispatched by the gateway's command pipeline — the app
|
|
just sends `/cmd args` as a message.
|
|
4. **Coexistence.** The same agent is reachable via Telegram *and* the app at
|
|
once; sessions/channels are shared.
|
|
|
|
The `tui_gateway` WS protocol is *not* reused; we define a clean, purpose-built
|
|
protocol (`04-wire-protocol.md`) that borrows familiar names (`message.*`,
|
|
`tool.*`, `reasoning`) but is owned by our plugin.
|
|
|
|
## Key architectural decisions + rationale
|
|
|
|
| Decision | Rationale |
|
|
|---|---|
|
|
| **Community-style platform plugin** (`register(ctx)` → `ctx.register_platform`) | Zero hermes-core changes; survives hermes updates; follows `ADDING_A_PLATFORM.md` "Plugin Path". |
|
|
| **Single WS transport** for chat, streaming, tools, media, pairing, FCM-token | One connection, one auth, one dependency (`websockets`, already core). Media as chunked binary frames avoids adding an HTTP server. |
|
|
| **`websockets` + `httpx` only** | Both are hermes *core* deps → the plugin adds **zero** new Python dependencies (respects hermes supply-chain pinning policy). |
|
|
| **Structured tool events, app-side verbosity** | Per the requirement: the gateway sends full tool data; the *app* decides everything/truncated/nothing. |
|
|
| **Reasoning split in the adapter** | The gateway prepends reasoning to the final text when `show_reasoning` is on; the adapter splits the stable prefix into a `reasoning` field so the app renders a clean collapsible block. |
|
|
| **Channels/threads = `chat_id`/`thread_id`** | The gateway's `SessionSource` already models this; cron delivery already targets `platform:chat_id:thread_id`. We map app concepts onto existing primitives. |
|
|
| **SQLite outbox + sync cursor** | Offline delivery + reconnect catch-up without re-reading full history. |
|
|
| **Compose Multiplatform** | Desktop is "the Android app, tweaked" → share protocol/state/UI; only platform services + layout differ. |
|
|
|
|
## Data flow (one turn)
|
|
|
|
1. App sends `message.send {text}` (or `/cmd`).
|
|
2. Plugin builds a `MessageEvent` (+ `media_urls` if attachments) →
|
|
`IrisAdapter.handle_message(event)`.
|
|
3. Gateway resolves the session (`chat_id`/`thread_id`), runs the agent.
|
|
4. Agent streams: `stream_delta_callback` → `GatewayStreamConsumer` →
|
|
`adapter.send()` (first) / `adapter.edit_message()` (updates) →
|
|
`message.start` / `message.update` frames.
|
|
5. Tool calls: `tool_progress_callback` → progress queue → `adapter.send()` →
|
|
`tool.*` frames (structured).
|
|
6. Intermediate beats: `interim_assistant_callback` → consumer → `commentary` frames.
|
|
7. Final answer: consumer finalizes → `adapter.send()` → `message` frame
|
|
(reasoning split into its own field).
|
|
8. If the app is disconnected at any point: frame is dropped to the **outbox**
|
|
and a **push** is fired; on reconnect the app `sync`s the delta.
|
|
|
|
> Implementation note: the main gateway uses the **legacy callback path**
|
|
> (not the ACP-only event-native `render_message_event` path). We therefore map
|
|
> the legacy `send`/`edit_message`/progress calls to our frames. The exact
|
|
> tool-progress vs commentary classification is verified empirically in M2 by
|
|
> running the real gateway with a test WS client (see `13-testing.md`). |