Add QR pairing (terminal QR, in-app scanner, iris://pair deep link)
This commit is contained in:
ARIA committed
1 parent
27dc7917f2
commit
7a6d922d12
63 files changed
+2073
-630
No files matched your search
+5
-5
@@ -46,7 +46,7 @@ Everything in the feature checklist below.
|
||||
| Tool output (app decides how much) | Emits **structured** tool events | App setting: everything / truncated / nothing |
|
||||
| Reasoning shown before message | Captures + splits reasoning | Collapsible "Reasoning" block above message |
|
||||
| Intermediate messages | Forwards `Commentary` events | Distinct dimmed bubble |
|
||||
| Threading + channels; default chat; user channels for cron | `chat_id`/`thread_id` model; cron `deliver=android:<chat>[:<thread>]` | Channel list, thread toggle, "new channel" |
|
||||
| Threading + channels; default chat; user channels for cron | `chat_id`/`thread_id` model; cron `deliver=iris:<chat>[:<thread>]` | Channel list, thread toggle, "new channel" |
|
||||
| Search ("everywhere" / "this chat/channel") | FTS5 session search bridge | Search UI + scope toggle |
|
||||
| Attach media (music/video/images/docs) | Inbound cache; outbound `send_*` | Pickers + chunked upload + preview |
|
||||
| Push notifications | FCM (primary) / ntfy (fallback) | FCM token / ntfy topic + notification service |
|
||||
@@ -57,7 +57,7 @@ Everything in the feature checklist below.
|
||||
| Decision | Choice |
|
||||
|---|---|
|
||||
| Desktop app tech | **Compose Multiplatform** (shares Android code; "tweaked" for big screen) |
|
||||
| Push backend | **Both** — FCM primary, ntfy fallback (`ANDROID_PUSH_BACKEND`) |
|
||||
| Push backend | **Both** — FCM primary, ntfy fallback (`IRIS_PUSH_BACKEND`) |
|
||||
| Media transport | **Over the WebSocket** (chunked binary frames; no extra Python deps) |
|
||||
| Phone default layout | **User-toggleable, single-pane default** (auto two-pane on large screens) |
|
||||
|
||||
@@ -66,7 +66,7 @@ Everything in the feature checklist below.
|
||||
1. **`hermes-agent/` is a read-only research reference.** It lives next to this
|
||||
folder for study only. It is **git-ignored** and must **never** be committed,
|
||||
pushed, or included in any artifact. Our plugin is *installed* into a live
|
||||
hermes home (`~/.hermes/plugins/android`); we never edit hermes core files.
|
||||
hermes home (`~/.hermes/plugins/iris`); we never edit hermes core files.
|
||||
2. **ADB is available and a device is connected** (`a5ca2a4b`, Xiaomi MIX 2S,
|
||||
Android 10 / API 29). Use `adb install` / `adb logcat` / `adb shell am start`
|
||||
to install, launch, and debug the app on-device throughout the build.
|
||||
@@ -88,6 +88,6 @@ Everything in the feature checklist below.
|
||||
## Naming
|
||||
|
||||
- Product/effort name: **Iris × Hermes** (folder `iris_x_hermes`).
|
||||
- hermes platform name: **`android`** (the plugin registers `Platform("android")`).
|
||||
- hermes platform name: **`iris`** (the plugin registers `Platform("iris")`).
|
||||
- WS default port: **8790** (configurable).
|
||||
- Default chat id: **`android:default`** (the home channel).
|
||||
- Default chat id: **`default`** (the home channel).
|
||||
@@ -12,7 +12,7 @@
|
||||
│ │ ▼ legacy stream callbacks (delta / tool-progress / commentary) │ │
|
||||
│ │ ┌──────────────────────────────┐ ┌────────────────────────────────┐ │ │
|
||||
│ │ │ android PLATFORM PLUGIN │◄───────►│ WebSocket SERVER (websockets) │ │ │
|
||||
│ │ │ AndroidAdapter │ JSON │ ws://host:8790/ws │ │ │
|
||||
│ │ │ IrisAdapter │ JSON │ ws://host:8790/ws │ │ │
|
||||
│ │ │ (BasePlatformAdapter) │ frames │ + media + FCM-token + pairing │ │ │
|
||||
│ │ │ • send / edit / stream │ └───────────────┬────────────────┘ │ │
|
||||
│ │ │ • media cache │ │ WSS │ │
|
||||
@@ -40,8 +40,8 @@
|
||||
## Process model
|
||||
|
||||
- **One `hermes gateway` process** hosts the agent core, the session store, the
|
||||
cron scheduler, *and* our `android` platform plugin. The plugin's WebSocket
|
||||
server runs on the gateway's asyncio loop (started in `AndroidAdapter.connect()`).
|
||||
cron scheduler, *and* our `iris` platform plugin. The plugin's WebSocket
|
||||
server runs on the gateway's asyncio loop (started in `IrisAdapter.connect()`).
|
||||
- **The app is a client.** It *initiates* the WS connection to the gateway
|
||||
(outbound), so no inbound port is needed on the phone. For LAN/remote access
|
||||
the user points the app at the gateway's LAN IP / Tailscale name / a WSS
|
||||
@@ -56,7 +56,7 @@ serves Telegram/Discord/… and cron) and the **`tui_gateway`** JSON-RPC backend
|
||||
(used by the TUI and the existing Electron desktop app). We deliberately use the
|
||||
**messaging gateway** because:
|
||||
|
||||
1. **Cron delivery is native.** Cron jobs resolve `deliver=android:<chat>[:<thread>]`
|
||||
1. **Cron delivery is native.** Cron jobs resolve `deliver=iris:<chat>[:<thread>]`
|
||||
through the platform registry and call our adapter's `send()`. No bridging.
|
||||
2. **`send_message` tool routing** works out of the box (plugin
|
||||
`parse_target_ref_fn`).
|
||||
@@ -86,7 +86,7 @@ protocol (`04-wire-protocol.md`) that borrows familiar names (`message.*`,
|
||||
|
||||
1. App sends `message.send {text}` (or `/cmd`).
|
||||
2. Plugin builds a `MessageEvent` (+ `media_urls` if attachments) →
|
||||
`AndroidAdapter.handle_message(event)`.
|
||||
`IrisAdapter.handle_message(event)`.
|
||||
3. Gateway resolves the session (`chat_id`/`thread_id`), runs the agent.
|
||||
4. Agent streams: `stream_delta_callback` → `GatewayStreamConsumer` →
|
||||
`adapter.send()` (first) / `adapter.edit_message()` (updates) →
|
||||
|
||||
+7
-7
@@ -13,10 +13,10 @@ iris_x_hermes/
|
||||
│
|
||||
├── hermes-agent/ # ⚠️ READ-ONLY REFERENCE — NEVER PUSHED (git-ignored)
|
||||
│
|
||||
├── gateway-plugin/ # ① Python plugin → installed to ~/.hermes/plugins/android
|
||||
├── gateway-plugin/ # ① Python plugin → installed to ~/.hermes/plugins/iris
|
||||
│ ├── plugin.yaml # manifest (kind: platform, env vars, home channel)
|
||||
│ ├── __init__.py
|
||||
│ ├── adapter.py # AndroidAdapter(BasePlatformAdapter) + register(ctx)
|
||||
│ ├── adapter.py # IrisAdapter(BasePlatformAdapter) + register(ctx)
|
||||
│ ├── ws_server.py # websockets server, connection registry, framing
|
||||
│ ├── protocol.py # frame schemas (source of truth, mirrored in Kotlin)
|
||||
│ ├── media.py # inbound cache + outbound chunked streaming
|
||||
@@ -50,9 +50,9 @@ iris_x_hermes/
|
||||
## Module responsibilities
|
||||
|
||||
### `gateway-plugin/` (Python)
|
||||
- **`plugin.yaml`** — manifest: `name: android-platform`, `kind: platform`,
|
||||
- **`plugin.yaml`** — manifest: `name: iris-platform`, `kind: platform`,
|
||||
`requires_env` / `optional_env` (surfaced in `hermes config`/setup).
|
||||
- **`adapter.py`** — `AndroidAdapter(BasePlatformAdapter)` + `register(ctx)`.
|
||||
- **`adapter.py`** — `IrisAdapter(BasePlatformAdapter)` + `register(ctx)`.
|
||||
The heart of the plugin. See `03-gateway-plugin.md`.
|
||||
- **`ws_server.py`** — `websockets` server, per-device connection registry,
|
||||
frame encode/decode, heartbeat, broadcast routing to all connected devices.
|
||||
@@ -62,7 +62,7 @@ iris_x_hermes/
|
||||
`media.offer`/`media.pull` chunked streaming.
|
||||
- **`outbox.py`** — SQLite outbox per `chat_id` + monotonic sync cursor.
|
||||
- **`push.py`** — `PushBackend` interface; `FcmBackend` (httpx, FCM HTTP v1) and
|
||||
`NtfyBackend` (reuses hermes ntfy publish). Selected by `ANDROID_PUSH_BACKEND`.
|
||||
`NtfyBackend` (reuses hermes ntfy publish). Selected by `IRIS_PUSH_BACKEND`.
|
||||
- **`pairing.py`** — token generation/verification (constant-time), device
|
||||
registry (SQLite), QR payload.
|
||||
- **`search.py`** — FTS5 query bridge over the hermes session store.
|
||||
@@ -83,7 +83,7 @@ Thin shells: `Application`/`MainActivity` (Android) and `main()`/window
|
||||
## Build systems
|
||||
|
||||
- **Python plugin:** no build step (pure Python, stdlib + hermes core deps).
|
||||
Installed by copying/symlinking into `~/.hermes/plugins/android`. Tested with
|
||||
Installed by copying/symlinking into `~/.hermes/plugins/iris`. Tested with
|
||||
hermes's `scripts/run_tests.sh`.
|
||||
- **Kotlin/CMP:** Gradle (Kotlin DSL) with the Compose Multiplatform plugin.
|
||||
`./gradlew :androidApp:installDebug`, `./gradlew :desktopApp:run`,
|
||||
@@ -124,7 +124,7 @@ keystore.jks
|
||||
|
||||
## Install layout (runtime)
|
||||
|
||||
- **Plugin:** `~/.hermes/plugins/android/` ← copy of `gateway-plugin/`
|
||||
- **Plugin:** `~/.hermes/plugins/iris/` ← copy of `gateway-plugin/`
|
||||
(or a symlink for dev). Discovered by hermes's `PluginManager`.
|
||||
- **App (dev):** installed on-device via `./gradlew :androidApp:installDebug`.
|
||||
- **App (desktop, dev):** `./gradlew :desktopApp:run`.
|
||||
+42
-35
@@ -1,6 +1,6 @@
|
||||
# 03 — Gateway Plugin (Python)
|
||||
|
||||
The plugin is a **community-style hermes platform plugin** named `android`.
|
||||
The plugin is a **community-style hermes platform plugin** named `iris`.
|
||||
It follows the "Plugin Path" in `hermes-agent/gateway/platforms/ADDING_A_PLATFORM.md`
|
||||
and the canonical example `hermes-agent/plugins/platforms/irc/adapter.py`.
|
||||
**Zero hermes-core changes. Zero new Python dependencies** (`websockets` and
|
||||
@@ -11,7 +11,7 @@ and the canonical example `hermes-agent/plugins/platforms/irc/adapter.py`.
|
||||
## 3.1 `plugin.yaml` (manifest)
|
||||
|
||||
```yaml
|
||||
name: android-platform
|
||||
name: iris-platform
|
||||
label: Android
|
||||
kind: platform
|
||||
version: 0.1.0
|
||||
@@ -22,63 +22,63 @@ description: >
|
||||
channels/threads, media, FTS5 search, and FCM/ntfy push.
|
||||
author: <you>
|
||||
requires_env:
|
||||
- name: ANDROID_TOKEN
|
||||
- name: IRIS_TOKEN
|
||||
description: "Shared pairing token the app presents on connect"
|
||||
prompt: "Android pairing token"
|
||||
password: true
|
||||
optional_env:
|
||||
- name: ANDROID_WS_HOST
|
||||
- name: IRIS_WS_HOST
|
||||
description: "WS bind host (default 127.0.0.1; use 0.0.0.0 for LAN)"
|
||||
prompt: "WS host"
|
||||
password: false
|
||||
- name: ANDROID_WS_PORT
|
||||
- name: IRIS_WS_PORT
|
||||
description: "WS port (default 8790)"
|
||||
prompt: "WS port"
|
||||
password: false
|
||||
- name: ANDROID_HOME_CHANNEL
|
||||
description: "Default chat id for cron/notification delivery (default android:default)"
|
||||
description: "Default chat id for cron/notification delivery (default default)"
|
||||
prompt: "Home channel"
|
||||
password: false
|
||||
- name: ANDROID_ALLOWED_USERS
|
||||
- name: IRIS_ALLOWED_USERS
|
||||
description: "Comma-separated allowed device_ids (empty = token-only auth)"
|
||||
prompt: "Allowed device ids"
|
||||
password: false
|
||||
- name: ANDROID_ALLOW_ALL_USERS
|
||||
- name: IRIS_ALLOW_ALL_USERS
|
||||
description: "Allow any paired device (dev only)"
|
||||
prompt: "Allow all devices? (true/false)"
|
||||
password: false
|
||||
- name: ANDROID_PUSH_BACKEND
|
||||
- name: IRIS_PUSH_BACKEND
|
||||
description: "Push backend: fcm (default) or ntfy"
|
||||
prompt: "Push backend"
|
||||
password: false
|
||||
- name: ANDROID_FCM_SERVICE_ACCOUNT
|
||||
- name: IRIS_FCM_SERVICE_ACCOUNT
|
||||
description: "Path to Firebase service-account JSON (FCM HTTP v1)"
|
||||
prompt: "FCM service account path"
|
||||
password: true
|
||||
- name: ANDROID_FCM_SERVER_KEY
|
||||
- name: IRIS_FCM_SERVER_KEY
|
||||
description: "Legacy FCM server key (fallback if no service account)"
|
||||
prompt: "FCM server key"
|
||||
password: true
|
||||
- name: NTFY_TOPIC
|
||||
description: "ntfy topic for push (when ANDROID_PUSH_BACKEND=ntfy)"
|
||||
description: "ntfy topic for push (when IRIS_PUSH_BACKEND=ntfy)"
|
||||
prompt: "ntfy topic"
|
||||
password: false
|
||||
- name: NTFY_SERVER_URL
|
||||
description: "ntfy server URL (default https://ntfy.sh)"
|
||||
prompt: "ntfy server URL"
|
||||
password: false
|
||||
- name: ANDROID_WS_CERT
|
||||
- name: IRIS_WS_CERT
|
||||
description: "TLS cert path for WSS (optional)"
|
||||
prompt: "WSS cert"
|
||||
password: false
|
||||
- name: ANDROID_WS_KEY
|
||||
- name: IRIS_WS_KEY
|
||||
description: "TLS key path for WSS (optional)"
|
||||
prompt: "WSS key"
|
||||
password: false
|
||||
```
|
||||
|
||||
Behavioral (non-secret) settings live in `config.yaml` under
|
||||
`gateway.platforms.android.extra` (host, port, home_channel, outbox retention,
|
||||
`gateway.platforms.iris.extra` (host, port, home_channel, outbox retention,
|
||||
max upload bytes, tls). Secrets live in `.env`. (hermes policy: `.env` = secrets
|
||||
only.)
|
||||
|
||||
@@ -87,21 +87,21 @@ only.)
|
||||
```python
|
||||
def register(ctx):
|
||||
ctx.register_platform(
|
||||
name="android",
|
||||
label="Android",
|
||||
adapter_factory=lambda cfg: AndroidAdapter(cfg),
|
||||
name="iris",
|
||||
label="Iris",
|
||||
adapter_factory=lambda cfg: IrisAdapter(cfg),
|
||||
check_fn=check_requirements, # passive: websockets importable + token set
|
||||
validate_config=validate_config, # host/port/token present
|
||||
is_connected=is_connected,
|
||||
required_env=["ANDROID_TOKEN"],
|
||||
required_env=["IRIS_TOKEN"],
|
||||
install_hint="No extra packages needed (websockets + httpx are core deps)",
|
||||
setup_fn=interactive_setup, # hermes gateway setup flow
|
||||
env_enablement_fn=_env_enablement, # seed extra + home_channel from env
|
||||
cron_deliver_env_var="ANDROID_HOME_CHANNEL",
|
||||
standalone_sender_fn=_standalone_send, # best-effort out-of-proc cron (stretch)
|
||||
parse_target_ref_fn=_parse_target_ref, # "android:<chat>[:<thread>]"
|
||||
allowed_users_env="ANDROID_ALLOWED_USERS",
|
||||
allow_all_env="ANDROID_ALLOW_ALL_USERS",
|
||||
parse_target_ref_fn=_parse_target_ref, # "iris:<chat>[:<thread>]"
|
||||
allowed_users_env="IRIS_ALLOWED_USERS",
|
||||
allow_all_env="IRIS_ALLOW_ALL_USERS",
|
||||
max_message_length=0, # 0 = no limit (WS has none)
|
||||
emoji="📱",
|
||||
pii_safe=False,
|
||||
@@ -123,26 +123,29 @@ Field reference (all from `PlatformEntry`, `gateway/platform_registry.py:63`):
|
||||
`ensure_deps_fn`.
|
||||
|
||||
- **`check_requirements()`** — passive probe: `import websockets` succeeds and
|
||||
`ANDROID_TOKEN` is set. Never installs.
|
||||
`IRIS_TOKEN` is set. Never installs.
|
||||
- **`_env_enablement()`** — returns a dict seeding `PlatformConfig.extra`
|
||||
(host/port/home_channel/push_backend) + a `home_channel` key
|
||||
`{"chat_id": "android:default", "name": "Default"}` so `hermes gateway status`
|
||||
`{"chat_id": "default", "name": "Default"}` so `hermes gateway status`
|
||||
and cron home-channel resolution work without instantiating the adapter.
|
||||
- **`_parse_target_ref(ref)`** — if `ref` starts with `android:`, return
|
||||
`(chat_id, thread_id)` parsed from `android:<chat>[:<thread>]`; else `None`.
|
||||
- **`_parse_target_ref(ref)`** — the core strips the platform prefix first, so
|
||||
`ref` is the direct chat id (e.g. `chan_7`, `default`) with an optional
|
||||
`:t_<n>` thread suffix; friendly names resolve via the channel directory.
|
||||
Returns `(chat_id, thread_id)` or `None`.
|
||||
- **`interactive_setup()`** — prompts for token (or generates one), host/port,
|
||||
push backend + credentials, prints a QR code (pairing) and the app URL.
|
||||
|
||||
## 3.3 `AndroidAdapter(BasePlatformAdapter)`
|
||||
## 3.3 `IrisAdapter(BasePlatformAdapter)`
|
||||
|
||||
Constructor: `super().__init__(config=config, platform=Platform("android"))`.
|
||||
Constructor: `super().__init__(config=config, platform=Platform("iris"))`.
|
||||
Reads `config.extra` (env overrides win). Initializes: WS server (not started
|
||||
until `connect()`), connection registry, outbox (SQLite under
|
||||
`get_hermes_home()/"android"`), push backend, pairing store, channel directory.
|
||||
`get_hermes_home()/"iris"`), push backend, pairing store, channel directory.
|
||||
|
||||
### Lifecycle
|
||||
|
||||
- **`connect(*, is_reconnect=False) -> bool`**
|
||||
- Acquire scoped lock (`gateway.status.acquire_scoped_lock("android", key)`)
|
||||
- Acquire scoped lock (`gateway.status.acquire_scoped_lock("iris", key)`)
|
||||
so two profiles can't bind the same port/identity.
|
||||
- Start the `websockets` server on `host:port` (TLS if cert/key set).
|
||||
- `_mark_connected()`; return True.
|
||||
@@ -150,6 +153,7 @@ until `connect()`), connection registry, outbox (SQLite under
|
||||
- Stop server, close all device sockets, release lock, `_mark_disconnected()`.
|
||||
|
||||
### Inbound (app → agent)
|
||||
|
||||
- WS `message.send {text, reply_to?, media_refs?}` → build `SessionSource` via
|
||||
`self.build_source(chat_id, chat_name, chat_type, user_id, user_name,
|
||||
thread_id)` → build `MessageEvent(text=…, message_type=TEXT, source=…,
|
||||
@@ -171,6 +175,7 @@ until `connect()`), connection registry, outbox (SQLite under
|
||||
- `sync {cursor}` → `outbox.py` → replay frames since cursor.
|
||||
|
||||
### Outbound (agent → app)
|
||||
|
||||
- **`send(chat_id, content, reply_to=None, metadata=None) -> SendResult`**
|
||||
- Split reasoning prefix (see `05-streaming.md`) → `reasoning` field.
|
||||
- If **any** device is connected: broadcast `message` frame to all.
|
||||
@@ -195,7 +200,9 @@ until `connect()`), connection registry, outbox (SQLite under
|
||||
channel directory, return it (used by cron "continuable" threads).
|
||||
|
||||
### Streaming hooks
|
||||
|
||||
The main gateway drives delivery through the **legacy callback path**:
|
||||
|
||||
- `stream_delta_callback` → `GatewayStreamConsumer` → `send()` (first) +
|
||||
`edit_message()` (updates) → `message.start` / `message.update`.
|
||||
- `tool_progress_callback` → progress queue → `send_progress_messages` →
|
||||
@@ -230,7 +237,7 @@ verified empirically in M2 (see `13-testing.md`).
|
||||
`message.update` (coalesce to latest) under pressure, never drop
|
||||
`message`/`tool.end`/`notification`.
|
||||
|
||||
## 3.5 State & storage (all under `get_hermes_home()/"android"`)
|
||||
## 3.5 State & storage (all under `get_hermes_home()/"iris"`)
|
||||
|
||||
> Use `get_hermes_home()` from `hermes_constants` for **all** paths (profile-safe).
|
||||
> Never hardcode `~/.hermes`.
|
||||
@@ -245,9 +252,9 @@ verified empirically in M2 (see `13-testing.md`).
|
||||
|
||||
## 3.6 Config resolution
|
||||
|
||||
- **Secrets (`.env`):** `ANDROID_TOKEN`, `ANDROID_FCM_SERVICE_ACCOUNT`,
|
||||
`ANDROID_FCM_SERVER_KEY`, `ANDROID_WS_CERT/KEY`, `NTFY_TOPIC` (if secret).
|
||||
- **Behavioral (`config.yaml` → `gateway.platforms.android.extra`):** `host`,
|
||||
- **Secrets (`.env`):** `IRIS_TOKEN`, `IRIS_FCM_SERVICE_ACCOUNT`,
|
||||
`IRIS_FCM_SERVER_KEY`, `IRIS_WS_CERT/KEY`, `NTFY_TOPIC` (if secret).
|
||||
- **Behavioral (`config.yaml` → `gateway.platforms.iris.extra`):** `host`,
|
||||
`port`, `home_channel`, `allowed_users`, `push_backend`, `outbox_retention_hours`,
|
||||
`max_upload_bytes`, `tls`.
|
||||
- Env vars override `config.yaml` (hermes convention). Read secrets with the
|
||||
@@ -260,4 +267,4 @@ verified empirically in M2 (see `13-testing.md`).
|
||||
- All outbound sends are best-effort; a dead socket latches and the frame falls
|
||||
to the outbox.
|
||||
- `disconnect()` cancels the server task and closes sockets cleanly.
|
||||
- Token/PII redaction in all logs (hermes PII policy).
|
||||
- Token/PII redaction in all logs (hermes PII policy).
|
||||
+20
-20
@@ -12,7 +12,7 @@ Every frame:
|
||||
"v": 1,
|
||||
"id": 42, // optional; present on requests + their responses
|
||||
"type": "message", // frame type (below)
|
||||
"chat_id": "android:default", // optional; scope for chat-scoped frames
|
||||
"chat_id": "default", // optional; scope for chat-scoped frames
|
||||
"thread_id": "t_123", // optional
|
||||
"payload": { } // type-specific object
|
||||
}
|
||||
@@ -43,7 +43,7 @@ Pairing succeeded.
|
||||
"search":true,"push":"fcm","pickers":true},
|
||||
"sync_cursor":1042,
|
||||
"last_pushed_cursor":1040,
|
||||
"channels":[{"chat_id":"android:default","name":"Default","kind":"default","is_default":true}]
|
||||
"channels":[{"chat_id":"default","name":"Default","kind":"default","is_default":true}]
|
||||
}}
|
||||
```
|
||||
|
||||
@@ -57,7 +57,7 @@ woke the device via push (dedupe, `08-push.md` §8.7).
|
||||
A final / standalone message.
|
||||
|
||||
```json
|
||||
{"type":"message","chat_id":"android:default","thread_id":null,
|
||||
{"type":"message","chat_id":"default","thread_id":null,
|
||||
"payload":{
|
||||
"message_id":"m_9001","role":"assistant",
|
||||
"text":"Here is the answer…",
|
||||
@@ -107,7 +107,7 @@ them drop the message(s) from their cache. Also outboxed, so a device that was
|
||||
offline learns of the deletion on its next `sync`.
|
||||
|
||||
```json
|
||||
{"type":"message.deleted","id":30,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"message.deleted","id":30,"chat_id":"default","thread_id":null,
|
||||
"payload":{"message_ids":["m_9001","m_9002"]}}
|
||||
```
|
||||
|
||||
@@ -176,7 +176,7 @@ Channel directory updates. **Broadcast to all connected devices** (no explicit
|
||||
subscribe; the server pushes to every open WS).
|
||||
|
||||
```json
|
||||
{"type":"channel.created","payload":{"chat_id":"android:chan_7","name":"Cron Reports",
|
||||
{"type":"channel.created","payload":{"chat_id":"chan_7","name":"Cron Reports",
|
||||
"kind":"channel","parent_chat_id":null}}
|
||||
```
|
||||
|
||||
@@ -190,7 +190,7 @@ title.
|
||||
Response to a `history` request. Returns a page of messages for a chat/thread.
|
||||
|
||||
```json
|
||||
{"type":"history","id":20,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"history","id":20,"chat_id":"default","thread_id":null,
|
||||
"payload":{
|
||||
"messages":[
|
||||
{"message_id":"m_8990","role":"user","text":"Hi","ts":1723990000000},
|
||||
@@ -244,9 +244,9 @@ Response to a `commands.complete` request. Autocomplete matches for a typed pref
|
||||
Agent lifecycle for a chat/thread. App shows a "thinking…" indicator on `busy`.
|
||||
|
||||
```json
|
||||
{"type":"agent.busy","chat_id":"android:default","thread_id":null,
|
||||
{"type":"agent.busy","chat_id":"default","thread_id":null,
|
||||
"payload":{"reason":"processing"}}
|
||||
{"type":"agent.idle","chat_id":"android:default","thread_id":null,"payload":{}}
|
||||
{"type":"agent.idle","chat_id":"default","thread_id":null,"payload":{}}
|
||||
```
|
||||
|
||||
`reason` ∈ `processing | tool | waiting_input | cron`.
|
||||
@@ -256,7 +256,7 @@ Agent lifecycle for a chat/thread. App shows a "thinking…" indicator on `busy`
|
||||
```json
|
||||
{"type":"search.results","id":7,"payload":{
|
||||
"query":"deploy","scope":"all","hits":[
|
||||
{"message_id":"m_123","chat_id":"android:chan_7","thread_id":null,
|
||||
{"message_id":"m_123","chat_id":"chan_7","thread_id":null,
|
||||
"role":"assistant","snippet":"…deploy the service…","ts":1723900000000}]}}
|
||||
```
|
||||
|
||||
@@ -275,7 +275,7 @@ The gateway acknowledges that the agent has received and started processing
|
||||
the user's message. The app uses it to show ✓✓ on user bubbles.
|
||||
|
||||
```json
|
||||
{"type":"read.receipt","chat_id":"android:default","payload":{"message_id":"m_9001"}}
|
||||
{"type":"read.receipt","chat_id":"default","payload":{"message_id":"m_9001"}}
|
||||
```
|
||||
|
||||
Emitted to the originating connection when a `message.send` is accepted for
|
||||
@@ -319,7 +319,7 @@ First frame; auth + caps.
|
||||
|
||||
```json
|
||||
{"type":"hello","payload":{
|
||||
"token":"<ANDROID_TOKEN>","device_id":"dev_a1b2","device_name":"MIX 2S",
|
||||
"token":"<IRIS_TOKEN>","device_id":"dev_a1b2","device_name":"MIX 2S",
|
||||
"caps":{"min_protocol":1,"media":true,"push":"fcm"},
|
||||
"fcm_token":"<FCM token>","ntfy_topic":"<topic, if ntfy>"}}
|
||||
```
|
||||
@@ -329,7 +329,7 @@ First frame; auth + caps.
|
||||
Send text (or a `/slash-command`).
|
||||
|
||||
```json
|
||||
{"type":"message.send","id":10,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"message.send","id":10,"chat_id":"default","thread_id":null,
|
||||
"payload":{"text":"/model qwen3-27b","reply_to":"m_9001","media_refs":["mu_1"],
|
||||
"auto_thread":false}}
|
||||
```
|
||||
@@ -383,8 +383,8 @@ Answer an interactive picker.
|
||||
|
||||
```json
|
||||
{"type":"channel.create","id":14,"payload":{"name":"Cron Reports","kind":"channel"}}
|
||||
{"type":"channel.rename","id":15,"chat_id":"android:chan_7","payload":{"name":"Reports"}}
|
||||
{"type":"channel.set_default","id":16,"chat_id":"android:chan_7","payload":{}}
|
||||
{"type":"channel.rename","id":15,"chat_id":"chan_7","payload":{"name":"Reports"}}
|
||||
{"type":"channel.set_default","id":16,"chat_id":"chan_7","payload":{}}
|
||||
```
|
||||
|
||||
`channel.delete` is a **hard delete**: the channel/thread row is removed from
|
||||
@@ -396,7 +396,7 @@ removes its threads. The default channel cannot be deleted.
|
||||
|
||||
```json
|
||||
{"type":"search","id":17,"payload":{"query":"deploy","scope":"all"}}
|
||||
{"type":"search","id":18,"payload":{"query":"deploy","scope":"chat","chat_id":"android:chan_7","thread_id":null}}
|
||||
{"type":"search","id":18,"payload":{"query":"deploy","scope":"chat","chat_id":"chan_7","thread_id":null}}
|
||||
```
|
||||
|
||||
`scope` ∈ `all | chat`.
|
||||
@@ -408,7 +408,7 @@ broadcasts to other devices (for multi-device ✓✓ sync). The app uses it to
|
||||
mark messages as read locally (✓✓ on user bubbles).
|
||||
|
||||
```json
|
||||
{"type":"read.receipt","payload":{"chat_id":"android:default","message_id":"m_9001"}}
|
||||
{"type":"read.receipt","payload":{"chat_id":"default","message_id":"m_9001"}}
|
||||
```
|
||||
|
||||
### `history`
|
||||
@@ -416,7 +416,7 @@ mark messages as read locally (✓✓ on user bubbles).
|
||||
Load a page of messages for a chat/thread (initial open, scroll-up pagination).
|
||||
|
||||
```json
|
||||
{"type":"history","id":20,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"history","id":20,"chat_id":"default","thread_id":null,
|
||||
"payload":{"before_message_id":"m_8990","limit":50}}
|
||||
```
|
||||
|
||||
@@ -433,7 +433,7 @@ message already gone (pruned by retention) still yields a `message.deleted`
|
||||
broadcast so live caches drop it.
|
||||
|
||||
```json
|
||||
{"type":"message.delete","id":30,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"message.delete","id":30,"chat_id":"default","thread_id":null,
|
||||
"payload":{"message_ids":["m_9001","m_9002"]}}
|
||||
```
|
||||
|
||||
@@ -458,7 +458,7 @@ Autocomplete for a typed `/prefix`.
|
||||
Stop the current agent turn (abort generation / tool execution).
|
||||
|
||||
```json
|
||||
{"type":"agent.stop","id":23,"chat_id":"android:default","thread_id":null,"payload":{}}
|
||||
{"type":"agent.stop","id":23,"chat_id":"default","thread_id":null,"payload":{}}
|
||||
```
|
||||
|
||||
### `agent.steer`
|
||||
@@ -466,7 +466,7 @@ Stop the current agent turn (abort generation / tool execution).
|
||||
Inject a steering message mid-turn (redirects the agent without a new turn).
|
||||
|
||||
```json
|
||||
{"type":"agent.steer","id":24,"chat_id":"android:default","thread_id":null,
|
||||
{"type":"agent.steer","id":24,"chat_id":"default","thread_id":null,
|
||||
"payload":{"text":"Actually, focus on the error case."}}
|
||||
```
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ finalize (attach reasoning/model/tokens footer, stop the cursor). Auto-scroll
|
||||
while the user is at the bottom.
|
||||
|
||||
**Streaming on/off.** Two levels:
|
||||
- **Gateway side:** hermes `display.platforms.android.streaming` (default
|
||||
- **Gateway side:** hermes `display.platforms.iris.streaming` (default
|
||||
follows global). When off, the app just gets one final `message` frame.
|
||||
- **App side (per device):** Settings → "Streaming" toggle (default on). When
|
||||
off, the app ignores `message.start`/`message.update` frames and
|
||||
@@ -49,11 +49,11 @@ chosen by `reasoning_style` (`gateway/display_config.py:37`):
|
||||
- `blockquote`: `> 💭 **Reasoning:**\n> …\n\n<response>`
|
||||
- `subtext`: `-# 💭 Reasoning\n-# …\n\n<response>` (Discord-style)
|
||||
|
||||
**Plugin config.** Set for the `android` platform:
|
||||
**Plugin config.** Set for the `iris` platform:
|
||||
```yaml
|
||||
display:
|
||||
platforms:
|
||||
android:
|
||||
iris:
|
||||
show_reasoning: true
|
||||
reasoning_style: code # we split on the code-fence form
|
||||
```
|
||||
|
||||
@@ -9,10 +9,10 @@ gateway identity concepts**.
|
||||
|
||||
| App concept | hermes primitive | Example |
|
||||
| --- | --- | --- |
|
||||
| Default chat | home channel `chat_id` | `android:default` |
|
||||
| A thread (inside default chat) | `thread_id` under the default `chat_id` | `chat_id=android:default, thread_id=t_12` |
|
||||
| A user-created channel | a new `chat_id` | `android:chan_7` |
|
||||
| A thread inside a channel | `thread_id` under that `chat_id` | `chat_id=android:chan_7, thread_id=t_31` |
|
||||
| Default chat | home channel `chat_id` | `default` |
|
||||
| A thread (inside default chat) | `thread_id` under the default `chat_id` | `chat_id=default, thread_id=t_12` |
|
||||
| A user-created channel | a new `chat_id` | `chan_7` |
|
||||
| A thread inside a channel | `thread_id` under that `chat_id` | `chat_id=chan_7, thread_id=t_31` |
|
||||
|
||||
- **`chat_id`** = the conversation lane (a channel or the default chat).
|
||||
- **`thread_id`** = an optional sub-lane within a `chat_id` (topic-like).
|
||||
@@ -23,7 +23,7 @@ gateway identity concepts**.
|
||||
## 6.2 Default chat
|
||||
|
||||
- On first connect, the plugin ensures a **default channel** exists:
|
||||
`chat_id = ANDROID_HOME_CHANNEL` (default `android:default`), `kind=default`,
|
||||
`chat_id = ANDROID_HOME_CHANNEL` (default `default`), `kind=default`,
|
||||
`is_default=true`, name "Default".
|
||||
- It is also the **cron home channel** (`cron_deliver_env_var=
|
||||
ANDROID_HOME_CHANNEL`), so `deliver=android` (bare) routes here.
|
||||
@@ -37,7 +37,7 @@ gateway identity concepts**.
|
||||
- **Threads OFF** — flat conversation; all messages use `thread_id=null`.
|
||||
- **Threads ON** — the app groups the conversation into topic-like lanes.
|
||||
Each new "topic" mints a `thread_id` (via `channel.create {kind:thread,
|
||||
parent_chat_id:android:default}` or an implicit thread). The UI shows a
|
||||
parent_chat_id:default}` or an implicit thread). The UI shows a
|
||||
topic switcher (like Telegram topics) above the message list.
|
||||
- Threads are **app-organized** but **gateway-real**: each `thread_id` is a
|
||||
distinct hermes session lane, so context is isolated per thread and cron can
|
||||
@@ -82,7 +82,7 @@ lane (nothing to title / session-scoped, not conversation starters).
|
||||
- **Requirement:** the user creates new channels so **cron job outputs can be
|
||||
delegated to them** instead of the default chat.
|
||||
- **`channel.create {name, kind:"channel"}`** → plugin mints
|
||||
`chat_id = android:chan_<n>`, stores in directory, broadcasts
|
||||
`chat_id = chan_<n>`, stores in directory, broadcasts
|
||||
`channel.created` to all devices. The new channel appears in the channel list.
|
||||
- **`channel.rename` / `channel.set_default` / `channel.delete`** manage the
|
||||
directory (rename broadcasts `channel.renamed`; delete is a **hard delete**
|
||||
@@ -104,9 +104,9 @@ lane (nothing to title / session-scoped, not conversation starters).
|
||||
- **Cron targeting** (the key payoff): because the plugin registers
|
||||
`parse_target_ref_fn` and `cron_deliver_env_var`, cron jobs and the
|
||||
`send_message` tool can target any channel/thread:
|
||||
- `deliver="android"` → home (default) channel.
|
||||
- `deliver="android:android:chan_7"` → that channel.
|
||||
- `deliver="android:android:chan_7:t_31"` → that channel's thread.
|
||||
- `deliver="iris"` → home (default) channel.
|
||||
- `deliver="iris:chan_7"` → that channel.
|
||||
- `deliver="iris:chan_7:t_31"` → that channel's thread.
|
||||
- In-chat: the agent's `cronjob` tool can be told "deliver to the *Cron
|
||||
Reports* channel"; the gateway resolves the name via the channel directory.
|
||||
- **In-app affordance:** each channel's menu has "Set as cron target" / shows a
|
||||
@@ -118,7 +118,7 @@ lane (nothing to title / session-scoped, not conversation starters).
|
||||
- Cron resolves delivery targets in `cron/scheduler.py:2148`
|
||||
(`_resolve_single_delivery_target`). For `platform:chat_id[:thread_id]` it
|
||||
calls `tools.send_message_tool.resolve_send_target`, which uses our
|
||||
`parse_target_ref_fn` to parse `android:<chat>[:<thread>]`.
|
||||
`parse_target_ref_fn` to parse `iris:<chat>[:<thread>]`.
|
||||
- Delivery then calls the **live adapter's `send(chat_id, text, …)`** (gateway
|
||||
running) → our WS `message` frame (or outbox+push if the app is offline).
|
||||
- Cron deliveries are framed with a `[Cron delivery: <name>]` header by hermes;
|
||||
|
||||
+4
-4
@@ -1,7 +1,7 @@
|
||||
# 08 — Push Notifications, Outbox & Sync
|
||||
|
||||
The gateway can't reach a sleeping phone directly. Push goes through a cloud
|
||||
relay. **Decision: FCM primary, ntfy fallback** (`ANDROID_PUSH_BACKEND`).
|
||||
relay. **Decision: FCM primary, ntfy fallback** (`IRIS_PUSH_BACKEND`).
|
||||
|
||||
## 8.1 When push fires
|
||||
|
||||
@@ -23,13 +23,13 @@ class PushBackend(Protocol):
|
||||
def configured(self) -> bool: ...
|
||||
```
|
||||
|
||||
Selected at adapter init by `ANDROID_PUSH_BACKEND` (`fcm` default, `ntfy`).
|
||||
Selected at adapter init by `IRIS_PUSH_BACKEND` (`fcm` default, `ntfy`).
|
||||
|
||||
### 8.2.1 `FcmBackend` (primary)
|
||||
- **FCM HTTP v1 API** via `httpx` (core dep). Auth = Firebase **service
|
||||
account** (`ANDROID_FCM_SERVICE_ACCOUNT` JSON path) → mint a short-lived
|
||||
account** (`IRIS_FCM_SERVICE_ACCOUNT` JSON path) → mint a short-lived
|
||||
OAuth2 access token (cached, refreshed before expiry).
|
||||
- Fallback: legacy **server key** (`ANDROID_FCM_SERVER_KEY`) if no service
|
||||
- Fallback: legacy **server key** (`IRIS_FCM_SERVER_KEY`) if no service
|
||||
account (simpler, but legacy).
|
||||
- Target = the device's **FCM token** (registered via `hello` /
|
||||
`fcm.register`, stored in `devices.db`).
|
||||
|
||||
+20
-19
@@ -13,19 +13,20 @@
|
||||
## 9.2 Pairing flow
|
||||
|
||||
1. **Generate a token.** `hermes gateway setup` (our `interactive_setup`) either
|
||||
uses an existing `ANDROID_TOKEN` or generates a fresh high-entropy token
|
||||
uses an existing `IRIS_TOKEN` or generates a fresh high-entropy token
|
||||
(e.g. 32 bytes → 64 hex chars) and stores it in `.env`.
|
||||
2. **Present to the app.** Two options:
|
||||
- **QR code:** the setup prints a QR encoding
|
||||
`iris://pair?host=<lan-ip>&port=8790&token=<token>` (or a WSS URL). The
|
||||
phone scans it with the app's camera (or a system scanner) → pre-fills
|
||||
`iris://pair?host=<lan-ip>&port=8791&secure=0&token=<token>` (or a WSS
|
||||
URL when `secure=1`). The phone scans it with the app's **Scan QR**
|
||||
button (or a system scanner → `iris://pair` deep link) → pre-fills
|
||||
settings.
|
||||
- **Manual:** user types the server URL + token in the app's Connect screen.
|
||||
3. **App connects.** First WS frame is `hello {token, device_id, device_name,
|
||||
caps, fcm_token?}`.
|
||||
4. **Server verifies.** Constant-time compare of `token` vs `ANDROID_TOKEN`
|
||||
4. **Server verifies.** Constant-time compare of `token` vs `IRIS_TOKEN`
|
||||
(`hmac.compare_digest`). Optionally check `device_id` against
|
||||
`ANDROID_ALLOWED_USERS` (if set) or `ANDROID_ALLOW_ALL_USERS`.
|
||||
`IRIS_ALLOWED_USERS` (if set) or `IRIS_ALLOW_ALL_USERS`.
|
||||
5. **On success:** register the device in `devices.db`, send `hello.ack`.
|
||||
**On failure:** send `error {code:"auth"}` and close.
|
||||
|
||||
@@ -36,24 +37,24 @@ security principal (the token is).
|
||||
## 9.3 Auth model
|
||||
|
||||
- **Token = the security principal.** Any connection presenting the valid
|
||||
`ANDROID_TOKEN` is authorized (it's the user's own token).
|
||||
- **Allowlist (optional):** `ANDROID_ALLOWED_USERS` (comma-separated
|
||||
`IRIS_TOKEN` is authorized (it's the user's own token).
|
||||
- **Allowlist (optional):** `IRIS_ALLOWED_USERS` (comma-separated
|
||||
`device_id`s) restricts which *devices* may connect even with the token —
|
||||
useful if the token is shared. `ANDROID_ALLOW_ALL_USERS=true` disables the
|
||||
useful if the token is shared. `IRIS_ALLOW_ALL_USERS=true` disables the
|
||||
allowlist (dev only).
|
||||
- **Per-device tokens (stretch):** mint a unique token per device at pairing
|
||||
(revocable) instead of one shared token. v1 uses the shared token + optional
|
||||
device allowlist.
|
||||
- **Re-pairing:** rotating `ANDROID_TOKEN` invalidates all devices; they must
|
||||
- **Re-pairing:** rotating `IRIS_TOKEN` invalidates all devices; they must
|
||||
re-pair. `hermes android pair` (stretch CLI) re-issues + prints a new QR.
|
||||
|
||||
## 9.4 Transport security
|
||||
|
||||
- **Default (LAN/dev):** plain `ws://` on the trusted LAN. Fine for a home
|
||||
network.
|
||||
- **WSS (recommended for remote):** set `ANDROID_WS_CERT` / `ANDROID_WS_KEY`
|
||||
- **WSS (recommended for remote):** set `IRIS_WS_CERT` / `IRIS_WS_KEY`
|
||||
(self-signed or CA-signed). The app pins/accepts the cert (self-signed → user
|
||||
confirms fingerprint on first pair, like a SSH host key).
|
||||
confirms fingerprint on first pair, like a SSH host key).
|
||||
- **Remote reachability options** (documented, user's choice):
|
||||
- **Tailscale / WireGuard** (recommended): gateway gets a stable tailnet IP;
|
||||
app connects over the private mesh. No public exposure.
|
||||
@@ -61,11 +62,11 @@ security principal (the token is).
|
||||
at the edge, forward WS to `127.0.0.1:8790`.
|
||||
- **Public bind** (`0.0.0.0`) + WSS + strong token — last resort.
|
||||
- **HTTP fallback leg (docs/19):** the gateway also serves the same frames
|
||||
over plain HTTP (`ANDROID_HTTP_PORT`, default 8791) for the app's
|
||||
over plain HTTP (`IRIS_HTTP_PORT`, default 8791) for the app's
|
||||
fallback transport. It is a *second door with the same lock*: the same
|
||||
Bearer token (constant-time `verify_token`) + the same device allowlist
|
||||
(`X-Iris-Device`), the same 64 KiB body cap and per-device rate limit as
|
||||
the WS. Optional TLS via `ANDROID_HTTP_CERT` / `ANDROID_HTTP_KEY`.
|
||||
the WS. Optional TLS via `IRIS_HTTP_CERT` / `IRIS_HTTP_KEY`.
|
||||
`GET /v1/health` is unauthenticated by design (liveness only — it must
|
||||
not reflect tokens, device ids, or versions).
|
||||
- The app stores the server URL + (for self-signed) the pinned cert fingerprint
|
||||
@@ -73,7 +74,7 @@ security principal (the token is).
|
||||
|
||||
## 9.5 Secret & PII handling
|
||||
|
||||
- **Tokens/keys never logged.** Redact `ANDROID_TOKEN`, FCM tokens/keys, ntfy
|
||||
- **Tokens/keys never logged.** Redact `IRIS_TOKEN`, FCM tokens/keys, ntfy
|
||||
tokens in all log output (hermes PII policy; `agent/redact.py` patterns).
|
||||
- **`device_id`** is a random UUID (not PII). `device_name` is user-chosen.
|
||||
- **Media pull** is gated by hermes `validate_media_delivery_path` + delivery
|
||||
@@ -84,7 +85,7 @@ security principal (the token is).
|
||||
|
||||
## 9.6 Profile safety
|
||||
|
||||
- All plugin state lives under `get_hermes_home()/"android"` (profile-aware).
|
||||
- All plugin state lives under `get_hermes_home()/"iris"` (profile-aware).
|
||||
- Secrets are read with the scope-aware `_get_scoped_secret` pattern (see
|
||||
`plugins/platforms/irc/adapter.py:42`) so multiplexed profiles don't leak
|
||||
each other's tokens (fail-closed under `gateway.multiplex_profiles`).
|
||||
@@ -110,16 +111,16 @@ M7 research pass. "verified" = implemented and covered by
|
||||
"gap" = known limitation with the planned mitigation.
|
||||
|
||||
| # | Item | Status | Evidence / mitigation |
|
||||
|---|------|--------|-----------------------|
|
||||
| --- | ------ | -------- | ----------------------- |
|
||||
| 1 | Constant-time token compare | verified | `gateway-plugin/pairing.py:34` (`hmac.compare_digest`); `test_wrong_token_rejected` |
|
||||
| 2 | Bounded per-connection send buffer + rate limit on inbound frames | verified | Send: `SEND_TIMEOUT_S` bounds every outbound send (`ws_server.py:47`, `broadcast`/`send_to`). Inbound: per-connection token bucket on JSON frames (20/s, burst 40) → `error {code:"rate_limited"}` + close on exceed (`ws_server.py:55`, `_TokenBucket`, `_on_frame`); binary upload chunks exempt (see gap 1) |
|
||||
| 3 | Reject oversized frames / uploads (`max_upload_bytes`) | verified | `serve(max_size=adapter.max_upload_bytes)` (`ws_server.py:139`); per-upload total cap in `media.py` (`create_upload`/`feed`); `test_upload_declared_over_limit_rejected`, `test_upload_midstream_over_limit_rejected` |
|
||||
| 4 | Verify media sha256 + re-sniff MIME (don't trust client) | verified | `media.py:317` (`complete_upload` digest check), `media.py:147` (`reclassify_kind`); `test_upload_sha256_mismatch_rejected`, `test_reclassify_kind_does_not_trust_client` |
|
||||
| 5 | Redact all secrets in logs | gap | No mechanical redaction; the token is printed to stdout by design during `hermes gateway setup` (`gateway-plugin/adapter.py:632,650`). Mitigation: stdout is operator-only, not a log file; a redaction pass over gateway logs is planned |
|
||||
| 6 | WSS + cert pinning for remote | gap (partial) | WSS supported server-side (`ANDROID_WS_CERT`/`ANDROID_WS_KEY`, `ws_server.py:122`); the app builds a default `OkHttpClient` with no `CertificatePinner` (`app/shared/src/commonMain/kotlin/iris/net/GatewayClient.kt:87`). Mitigation: remote access requires CA-signed WSS until pinning lands; LAN `ws://` stays the default |
|
||||
| 6 | WSS + cert pinning for remote | gap (partial) | WSS supported server-side (`IRIS_WS_CERT`/`IRIS_WS_KEY`, `ws_server.py:122`); the app builds a default `OkHttpClient` with no `CertificatePinner` (`app/shared/src/commonMain/kotlin/iris/net/GatewayClient.kt:87`). Mitigation: remote access requires CA-signed WSS until pinning lands; LAN `ws://` stays the default |
|
||||
| 7 | Outbox retention cap + prune | verified | `gateway-plugin/outbox.py:48` (`retention_hours` default 72h, `max_rows` cap, `take_overflow_pruned`); `test_outbox_row_cap_prunes_oldest` |
|
||||
| 8 | Fail-closed secret reads under multiplexing | verified | `_get_scoped_secret` (`gateway-plugin/adapter.py:74`) for `ANDROID_TOKEN`/`ANDROID_WS_CERT`/`ANDROID_WS_KEY`/FCM/ntfy secrets; scoped bind lock in `connect()` (`adapter.py:779`) |
|
||||
| 8 | Fail-closed secret reads under multiplexing | verified | `_get_scoped_secret` (`gateway-plugin/adapter.py:74`) for `IRIS_TOKEN`/`IRIS_WS_CERT`/`IRIS_WS_KEY`/FCM/ntfy secrets; scoped bind lock in `connect()` (`adapter.py:779`) |
|
||||
| 9 | Gap: inbound frame rate limiting | implemented | Closes item 2: token bucket in `ws_server.py` (JSON frames only). Binary upload chunks are exempt — a 100 MB upload is 400 × 256 KiB frames in a tight loop and would exhaust any sane bucket; uploads are already bounded by per-frame `max_size` + the per-upload total cap |
|
||||
| 10 | Gap: Android token storage | implemented | `AndroidSecureStore` → `EncryptedSharedPreferences` (MasterKey AES256_GCM) with one-time migration of the plain `iris` prefs (read old key → write encrypted → delete old key); dep in `app/shared/build.gradle.kts` (`app/shared/src/androidMain/kotlin/iris/platform/AndroidSecureStore.kt`) |
|
||||
| 11 | Gap: guard not committed | implemented | `.pre-commit-config.yaml` (local hook → `scripts/guard_hermes_agent.sh --staged`); a fresh clone gets the guard after `pre-commit install` |
|
||||
| 12 | Gap: in-app QR scanner | gap | Pairing is manual URL+token only; the server prints a QR (`gateway-plugin/adapter.py:648-654`) that any system scanner can read. Plan: in-app camera scan later |
|
||||
| 12 | Gap: in-app QR scanner | implemented | `hermes gateway setup` renders a terminal QR (`gateway-plugin/qr.py`, pure-stdlib encoder) and the app's Connect screen has a **Scan QR** button (CameraX + ML Kit, `QrScanActivity`) plus an `iris://pair` deep link (`PairLink.parse`); `docs/20` |
|
||||
@@ -115,7 +115,7 @@ app/shared/src/
|
||||
|
||||
- `ToolCard` renders `tool.start/progress/end` frames.
|
||||
- The gateway always supplies the **full** tool data: it forces
|
||||
`display.platforms.android.tool_progress: verbose` (so the progress line
|
||||
`display.platforms.iris.tool_progress: verbose` (so the progress line
|
||||
carries the full args JSON → `tool.start.args`) and captures each completed
|
||||
call via the `post_tool_call` hook (→ `tool.end` `output_preview` /
|
||||
`duration` / `ok`). The app decides how much to show.
|
||||
@@ -317,5 +317,13 @@ Storage: `AndroidSqliteDriver` (app database dir) on Android,
|
||||
- First launch → **Connect**: server URL + token (or scan QR). "Test connection"
|
||||
does a real `hello` (not just a TCP probe — per hermes desktop guidance, the
|
||||
auth leg must be exercised). On success → save (secure storage) → main.
|
||||
- **Scan QR** (Android only, `docs/20`): a button below the token field opens
|
||||
`QrScanActivity` (CameraX + ML Kit, on-device, no Play services), requests the
|
||||
`CAMERA` permission, and pre-fills URL + token from the decoded
|
||||
`iris://pair…` payload via `PairLink.parse`. It never auto-connects — the
|
||||
user still taps "Test & Connect". A non-pairing QR sets an error and leaves
|
||||
the fields untouched. The same payload also arrives as an `iris://pair` deep
|
||||
link (system scanner / other phones) and pre-fills the screen the same way.
|
||||
Hidden on desktop (no camera).
|
||||
- States: connecting / connected / reconnecting / degraded / auth-failed — each
|
||||
with honest copy and a way out.
|
||||
+14
-14
@@ -57,8 +57,8 @@ hermes --version # sanity
|
||||
```bash
|
||||
# install the plugin (dev: symlink)
|
||||
mkdir -p ~/.hermes/plugins
|
||||
ln -s "$PWD/../gateway-plugin" ~/.hermes/plugins/android
|
||||
hermes gateway status # should list "android"
|
||||
ln -s "$PWD/../gateway-plugin" ~/.hermes/plugins/iris
|
||||
hermes gateway status # should list "iris"
|
||||
hermes gateway # run
|
||||
```
|
||||
- Tests use hermes's hermetic runner (never bare `pytest`):
|
||||
@@ -73,43 +73,43 @@ hermes --version # sanity
|
||||
`dev.iris.app`). Download `google-services.json` → `app/androidApp/`.
|
||||
3. Create a **service account** (Project settings → Service accounts → Generate
|
||||
new private key) → download the JSON. Store its path in
|
||||
`ANDROID_FCM_SERVICE_ACCOUNT` (in `~/.hermes/.env`).
|
||||
`IRIS_FCM_SERVICE_ACCOUNT` (in `~/.hermes/.env`).
|
||||
4. The app's `FirebaseMessagingService` obtains the FCM token at runtime and
|
||||
registers it via `hello` / `fcm.register`.
|
||||
|
||||
> Skip Firebase → set `ANDROID_PUSH_BACKEND=ntfy` and configure `NTFY_TOPIC` /
|
||||
> Skip Firebase → set `IRIS_PUSH_BACKEND=ntfy` and configure `NTFY_TOPIC` /
|
||||
> `NTFY_SERVER_URL` (self-host ntfy or use ntfy.sh). See `08-push.md`.
|
||||
|
||||
## 12.6 Environment variables (summary)
|
||||
|
||||
**Secrets (`~/.hermes/.env`):**
|
||||
```
|
||||
ANDROID_TOKEN=<64-hex>
|
||||
ANDROID_PUSH_BACKEND=fcm # or ntfy
|
||||
ANDROID_FCM_SERVICE_ACCOUNT=/path/to/service-account.json
|
||||
# ANDROID_FCM_SERVER_KEY=<legacy key> # fallback if no service account
|
||||
IRIS_TOKEN=<64-hex>
|
||||
IRIS_PUSH_BACKEND=fcm # or ntfy
|
||||
IRIS_FCM_SERVICE_ACCOUNT=/path/to/service-account.json
|
||||
# IRIS_FCM_SERVER_KEY=<legacy key> # fallback if no service account
|
||||
# NTFY_TOPIC=iris-push # when ntfy
|
||||
# NTFY_SERVER_URL=https://ntfy.sh
|
||||
# ANDROID_WS_CERT=/path/cert.pem # WSS
|
||||
# ANDROID_WS_KEY=/path/key.pem
|
||||
# IRIS_WS_CERT=/path/cert.pem # WSS
|
||||
# IRIS_WS_KEY=/path/key.pem
|
||||
```
|
||||
|
||||
**Behavioral (`~/.hermes/config.yaml`):**
|
||||
```yaml
|
||||
gateway:
|
||||
platforms:
|
||||
android:
|
||||
iris:
|
||||
enabled: true
|
||||
extra:
|
||||
host: 127.0.0.1 # 0.0.0.0 for LAN
|
||||
port: 8790
|
||||
home_channel: android:default
|
||||
home_channel: default
|
||||
push_backend: fcm
|
||||
outbox_retention_hours: 72
|
||||
max_upload_bytes: 104857600 # 100 MB
|
||||
display:
|
||||
platforms:
|
||||
android:
|
||||
iris:
|
||||
show_reasoning: true
|
||||
reasoning_style: code
|
||||
streaming: true
|
||||
@@ -128,7 +128,7 @@ import asyncio, json, websockets
|
||||
async def main():
|
||||
async with websockets.connect("ws://127.0.0.1:8790/ws") as ws:
|
||||
await ws.send(json.dumps({"v":1,"type":"hello","payload":{
|
||||
"token":"<ANDROID_TOKEN>","device_id":"test","device_name":"probe",
|
||||
"token":"<IRIS_TOKEN>","device_id":"test","device_name":"probe",
|
||||
"caps":{"min_protocol":1}}}))
|
||||
print("recv:", await ws.recv())
|
||||
asyncio.run(main())
|
||||
|
||||
+5
-5
@@ -19,7 +19,7 @@ without the app (critical for verifying frame shapes early).
|
||||
- `register(ctx)` produces a valid `PlatformEntry` (name, cron env var,
|
||||
parse_target_ref).
|
||||
- `check_requirements` / `validate_config` / `is_connected` truth table.
|
||||
- `_parse_target_ref`: `android:<chat>`, `android:<chat>:<thread>`, non-android
|
||||
- `_parse_target_ref`: `iris:<chat>`, `iris:<chat>:<thread>`, non-android
|
||||
→ None.
|
||||
- **Reasoning split:** given a `show_reasoning`-style final text, `send()`
|
||||
emits `message {reasoning, text}` correctly; no-prefix → no reasoning field.
|
||||
@@ -54,7 +54,7 @@ Kotlin client.
|
||||
|
||||
```bash
|
||||
hermes gateway & # with the android plugin
|
||||
python gateway-plugin/tests/ws_probe.py --token <ANDROID_TOKEN> \
|
||||
python gateway-plugin/tests/ws_probe.py --token <IRIS_TOKEN> \
|
||||
--send "list the files and summarize"
|
||||
# prints: hello.ack, typing, message.start, message.update…, tool.start, tool.end,
|
||||
# commentary, message.stop {reasoning,…}, …
|
||||
@@ -112,7 +112,7 @@ adb logcat -d > /tmp/logcat.txt
|
||||
verbosity in Settings → rendering changes.
|
||||
5. **Intermediate:** a multi-step prompt → commentary bubble appears dimmed.
|
||||
6. **Channels:** create "Cron Reports" → appears in list; set as cron target.
|
||||
7. **Cron delivery:** create a cron job `deliver=android:android:chan_<n>` → it
|
||||
7. **Cron delivery:** create a cron job `deliver=iris:chan_<n>` → it
|
||||
fires → lands in that channel (not default).
|
||||
8. **Search:** "search everywhere" vs "this chat" → correct scoping; tap → jump.
|
||||
9. **Media (in):** attach a photo + a video → agent receives (vision) → reply.
|
||||
@@ -144,11 +144,11 @@ adb logcat -d > /tmp/logcat.txt
|
||||
## 13.5 Debugging tips
|
||||
|
||||
- **Gateway side:** `~/.hermes/logs/gateway.log` (and `hermes logs --follow`).
|
||||
Our plugin logs under the `android` adapter name; secrets redacted.
|
||||
Our plugin logs under the `iris` adapter name; secrets redacted.
|
||||
- **WS framing bugs:** use the `ws_probe.py` harness — it isolates the protocol
|
||||
from the app.
|
||||
- **Streaming jitter:** the consumer edits at intervals; if updates look chunky,
|
||||
check `display.platforms.android.streaming` and the consumer's edit interval.
|
||||
check `display.platforms.iris.streaming` and the consumer's edit interval.
|
||||
- **Media pull stalls:** check chunk size + backpressure; confirm the file is
|
||||
within hermes delivery roots (`validate_media_delivery_path`).
|
||||
- **FCM not arriving:** confirm the token registered (`devices.db`), the service
|
||||
|
||||
+56
-9
@@ -1,4 +1,4 @@
|
||||
# 14 — Milestones (M0–M7)
|
||||
# 14 — Milestones (M0–M8)
|
||||
|
||||
Phased delivery. Each milestone ends with a **demo** (on-device where noted) and
|
||||
has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
@@ -6,7 +6,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
---
|
||||
|
||||
## M0 — Toolchain & scaffolding
|
||||
|
||||
**Goal:** everything builds; the plugin is discoverable; the repo is safe.
|
||||
|
||||
- [X] Install JDK 17, Android SDK, set `ANDROID_HOME` (`12-toolchain.md`).
|
||||
- [X] `cd hermes-agent && uv sync` (hermes venv works).
|
||||
- [X] Create monorepo scaffold (`02-monorepo.md`): `gateway-plugin/`, `app/`
|
||||
@@ -16,20 +18,22 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
- [X] CMP project builds empty: `./gradlew :androidApp:assembleDebug`,
|
||||
`./gradlew :desktopApp:run` (blank window).
|
||||
- [X] Plugin skeleton: `plugin.yaml` + `adapter.py` with `register(ctx)` + a
|
||||
no-op `AndroidAdapter` → `hermes gateway status` lists **android**.
|
||||
- **Demo:** `hermes gateway status` shows `android`; `./gradlew
|
||||
no-op `IrisAdapter` → `hermes gateway status` lists **iris**.
|
||||
- **Demo:** `hermes gateway status` shows `iris`; `./gradlew
|
||||
:androidApp:installDebug` installs a blank app on the MIX 2S.
|
||||
- **Accept:** blank app installs + launches on-device; plugin visible in
|
||||
`hermes gateway status`; `hermes-agent/` is git-ignored (verify with
|
||||
`git status --ignored`).
|
||||
|
||||
## M1 — Gateway core loop (text round-trip)
|
||||
|
||||
**Goal:** pair + send a text message + get a (non-streaming) reply.
|
||||
|
||||
- [X] WS server (`ws_server.py`): bind, `hello` auth (constant-time),
|
||||
`hello.ack`, heartbeat, connection registry.
|
||||
- [X] `AndroidAdapter.send()` → `message` frame; inbound `message.send` →
|
||||
- [X] `IrisAdapter.send()` → `message` frame; inbound `message.send` →
|
||||
`MessageEvent` → `handle_message`.
|
||||
- [X] Pairing store + `ANDROID_TOKEN`; QR payload in `interactive_setup`.
|
||||
- [X] Pairing store + `IRIS_TOKEN`; QR payload in `interactive_setup`.
|
||||
- [X] App: Connect screen (URL+token, real `hello` test), `GatewayClient`
|
||||
(connect + reconnect), ChatScreen sends + renders `message`.
|
||||
- [X] `ws_probe.py` harness drives a real turn.
|
||||
@@ -38,9 +42,11 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
reconnect after gateway restart re-pairs.
|
||||
|
||||
## M2 — Streaming + reasoning + tools + commentary
|
||||
|
||||
**Goal:** the "agent transparency" features.
|
||||
|
||||
- [X] Map consumer `send`/`edit_message` → `message.start/update/stop`.
|
||||
- [X] Reasoning: set `show_reasoning` for android; adapter splits prefix →
|
||||
- [X] Reasoning: set `show_reasoning` for iris; adapter splits prefix →
|
||||
`reasoning` field. **Verify format with `ws_probe.py`.** (The model
|
||||
returns a separate `reasoning_content` field. In the *streaming* case the
|
||||
gateway drops it — the stream consumer only forwards `content` and the
|
||||
@@ -64,12 +70,14 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
rendering; reasoning copy button works; frame shapes match `04-wire-protocol`.
|
||||
|
||||
## M3 — Channels/threads + cron + search
|
||||
|
||||
**Goal:** organization + cron delegation + search.
|
||||
|
||||
- [X] Channel directory (SQLite): default channel ensured; `channel.create/
|
||||
rename/set_default/delete` + `channel.*` frames.
|
||||
- [X] Threads: toggle in default chat; `thread_id` lanes; `create_handoff_thread`.
|
||||
- [X] `parse_target_ref_fn` + `cron_deliver_env_var` → cron
|
||||
`deliver=android:<chat>[:<thread>]` works.
|
||||
`deliver=iris:<chat>[:<thread>]` works.
|
||||
- [X] `search.py` FTS5 bridge; `search` frame (all / this-chat) → results.
|
||||
- [X] App: channel list (drawer/rail), thread toggle + topic switcher, "new
|
||||
channel" + "set as cron target", SearchScreen with scope toggle + jump.
|
||||
@@ -79,7 +87,7 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
isolate context; search scoping correct; channel list reconciles on events.
|
||||
- **Status (complete):** channel directory + threads + search + outbox sync
|
||||
verified end-to-end via `ws_probe.py` (create/rename/set_default/delete,
|
||||
thread lanes, FTS5 search, sync); cron `deliver=android:<chat>[:<thread>]`
|
||||
thread lanes, FTS5 search, sync); cron `deliver=iris:<chat>[:<thread>]`
|
||||
target resolution verified via `resolve_send_target`. App on-device: channel
|
||||
drawer, thread toggle + topic switcher, new channel, search overlay with
|
||||
jump. Minor UI gaps deferred to M7 polish: "set as cron target" is set via
|
||||
@@ -89,7 +97,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
not e2e-tested (it shares the verified resolution path).
|
||||
|
||||
## M4 — Media
|
||||
|
||||
**Goal:** attach + receive + play media.
|
||||
|
||||
- [X] Inbound: `media.upload` chunked → `cache_*_from_bytes` → `media_urls`;
|
||||
size limit + sha256 + MIME re-sniff.
|
||||
- [X] Outbound: `send_*` → `media.offer`; `media.pull` chunked; delivery-path
|
||||
@@ -117,12 +127,14 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
`04-wire-protocol.md` + `frames.schema.json`.
|
||||
|
||||
## M5 — Push + offline (FCM + ntfy)
|
||||
|
||||
**Goal:** reach the phone when backgrounded; catch up on reconnect.
|
||||
|
||||
- [x] Outbox (SQLite) + sync cursor; `sync`/`sync.done`; retention prune
|
||||
(row cap 5000 + prune banner, throttled 1/h).
|
||||
- [x] `push.py`: `FcmBackend` (HTTP v1 + service account, httpx; JWT via
|
||||
PyJWT+cryptography) + `NtfyBackend` (X-Data header); selected by
|
||||
`ANDROID_PUSH_BACKEND`. ntfy server exposed in `server_caps.push_ntfy_server`.
|
||||
`IRIS_PUSH_BACKEND`. ntfy server exposed in `server_caps.push_ntfy_server`.
|
||||
- [x] Fire push on no-live-subscriber; data payload for silent sync.
|
||||
High-priority kinds (approval/clarify/cron) push even when live.
|
||||
- [x] App: FCM service (`onNewToken` → `fcm.register`; inert without a
|
||||
@@ -143,7 +155,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
loss/dup (verified); banners show for foreground events (implemented).
|
||||
|
||||
## M6 — Desktop app
|
||||
|
||||
**Goal:** the same app on a big screen.
|
||||
|
||||
- [x] `desktopMain`: tray + OS notifications; `MediaPlayer` actual (mpv/WebView);
|
||||
`MediaPicker` actual (file dialog); `SecureStore` actual; window mgmt.
|
||||
- [x] Two-pane default layout; keyboard shortcuts; optional inspector pane.
|
||||
@@ -178,7 +192,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
macOS/Windows packaging are deferred to M7.
|
||||
|
||||
## M7 — Polish + E2E + docs
|
||||
|
||||
**Goal:** ship-quality.
|
||||
|
||||
- [x] Telegram-style layout pass (per reference image): header, bubbles, date
|
||||
separators, ✓✓, model/token footer, banner, bottom bar.
|
||||
- [x] Theming (dark default, accent), onboarding/pairing UX, empty/loading/
|
||||
@@ -222,7 +238,38 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
|
||||
---
|
||||
|
||||
## M8 — QR pairing
|
||||
|
||||
**Goal:** QR-based pairing — a scannable QR at `hermes gateway setup` plus an
|
||||
in-app scanner and `iris://pair` deep link (closes gap #12, `docs/20`).
|
||||
|
||||
- [x] Pure-stdlib QR encoder + terminal renderer (`gateway-plugin/qr.py`):
|
||||
byte mode, EC M with L fallback, versions 1–10, ISO penalty masking,
|
||||
**zero new Python deps**.
|
||||
- [x] `interactive_setup` renders the QR after the pairing URL (text lines
|
||||
stay as the primary path).
|
||||
- [x] `PairLink` parser (`iris/util/PairLink.kt`) + jvmTest (valid/missing
|
||||
token/bad port/wrong scheme/wrong host/percent-encoded/secure/default
|
||||
port).
|
||||
- [x] CameraX + ML Kit scanner (`QrScanActivity`, on-device, no Play
|
||||
services) + Connect-screen **Scan QR** button (Android only; hidden on
|
||||
desktop) + `CAMERA` permission.
|
||||
- [x] `iris://pair` deep link (system-scanner / other-phone fallback) reusing
|
||||
the same parser.
|
||||
- **Accept:** `docs/20` §20.6; gap #12 in `09-pairing-security.md` closed.
|
||||
- **Status (2026-08-22):** Encoder cross-checked byte-for-byte against an
|
||||
independent reference and decoded by an independent decoder (zbarimg); fixed
|
||||
v1-M and v7-M matrix vectors lock the algorithm. `hermes gateway setup`
|
||||
prints a scannable QR (v7-M, 45 modules) for the 64-hex-token payload. App:
|
||||
Connect screen shows **Scan QR** (Android), which opens `QrScanActivity`
|
||||
(CameraX camera2 + ML Kit barcode), requests `CAMERA`, and pre-fills URL +
|
||||
token via `PairLink.parse` without auto-connecting; `iris://pair` deep link
|
||||
pre-fills the same way. Docs updated per `docs/20` Part C.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing notes
|
||||
|
||||
- **M1/M2 depend on the `ws_probe.py` harness** to lock frame shapes early —
|
||||
build it in M1.
|
||||
- **M3 (cron) and M5 (push) both touch the outbox** — build the outbox in M3,
|
||||
|
||||
@@ -5,7 +5,7 @@ integration point. Paths are relative to `hermes-agent/` (the read-only
|
||||
reference). This lets a coder jump straight to the right code instead of
|
||||
re-deriving the architecture.
|
||||
|
||||
> ⚠️ Read-only. We **install** our plugin into `~/.hermes/plugins/android`; we
|
||||
> ⚠️ Read-only. We **install** our plugin into `~/.hermes/plugins/iris`; we
|
||||
> never edit these files.
|
||||
|
||||
## Plugin / platform registration
|
||||
|
||||
@@ -3,24 +3,24 @@
|
||||
## Locked decisions (from planning, 2026-08-19)
|
||||
|
||||
| # | Decision | Choice | Rationale |
|
||||
|---|---|---|---|
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | Desktop app tech | **Compose Multiplatform** | Desktop = "the Android app, tweaked"; share protocol/state/UI. |
|
||||
| 2 | Push backend | **Both — FCM primary, ntfy fallback** | FCM is standard/reliable; ntfy for self-hosters with no Firebase. `ANDROID_PUSH_BACKEND`. |
|
||||
| 2 | Push backend | **Both — FCM primary, ntfy fallback** | FCM is standard/reliable; ntfy for self-hosters with no Firebase. `IRIS_PUSH_BACKEND`. |
|
||||
| 3 | Media transport | **Over the WebSocket** | One transport, zero new Python deps; chunked binary frames. |
|
||||
| 4 | Phone default layout | **User-toggleable, single-pane default** | App-like on phones; auto two-pane on large screens; desktop defaults two-pane. |
|
||||
|
||||
## Additional decisions made during planning
|
||||
|
||||
| Decision | Choice | Note |
|
||||
|---|---|---|
|
||||
| Connection point | **Messaging gateway** (platform plugin `android`), not `tui_gateway` | Makes cron/`send_message`/slash/coexistence native. |
|
||||
| --- | --- | --- |
|
||||
| Connection point | **Messaging gateway** (platform plugin `iris`), not `tui_gateway` | Makes cron/`send_message`/slash/coexistence native. |
|
||||
| Plugin style | **Community plugin** (`register(ctx)`) | Zero hermes-core changes. |
|
||||
| Python deps | **None new** (`websockets` + `httpx` are core) | Respects hermes pinning policy. |
|
||||
| Tool events | **Structured frames; app controls verbosity** | Gateway sends full data; app = everything/truncated/nothing. |
|
||||
| Reasoning | **Adapter splits the `show_reasoning` prefix** | Clean `reasoning` field → collapsible block above message. |
|
||||
| Channels/threads | **Map onto `chat_id`/`thread_id`** | Existing gateway primitives; cron targets them. |
|
||||
| Offline | **SQLite outbox + sync cursor** | Catch-up on reconnect; push on disconnect. |
|
||||
| Default chat id | `android:default` | Home channel + cron default. |
|
||||
| Default chat id | `default` | Home channel + cron default. |
|
||||
| WS port | `8790` (default) | Configurable. |
|
||||
| minSdk | 26 (test device API 29) | Broad coverage. |
|
||||
| Frame routing | **Broadcast to all connected devices** (no per-chat subscribe) | Single-user model; simpler. |
|
||||
@@ -29,6 +29,8 @@
|
||||
| Agent lifecycle | **`agent.busy`/`agent.idle`** events + **`agent.stop`/`agent.steer`** requests | App shows thinking indicator; user can abort or steer mid-turn. |
|
||||
| Local DB (KMP) | **SQLDelight** (not Room) | Room is Android-only; SQLDelight works in commonMain for both platforms. |
|
||||
| Voice input | **Record → upload as audio media** (no client-side STT) | Agent's STT (if configured) handles transcription. |
|
||||
| QR encoder | **Pure-stdlib** (`gateway-plugin/qr.py`) | No `qrcode`/`segno`/Pillow; byte mode, EC M→L, v1–10, ISO penalty masking. Keeps the plugin's zero-new-dep rule. |
|
||||
| QR scanner | **ML Kit barcode** (not zxing-android-embedded) | On-device, no Google Play services; better accuracy/latency, Google-maintained. Android-only (desktop has no camera). |
|
||||
|
||||
## Open questions (resolve during implementation)
|
||||
|
||||
@@ -45,7 +47,7 @@ will proceed with unless you say otherwise.
|
||||
`reasoning_style: code` for android and split on that; fallback = no
|
||||
reasoning field (full text) if the prefix isn't found. Verify in M2.
|
||||
3. **Per-device tokens vs shared token (M1/M5).** *Default (v1):* shared
|
||||
`ANDROID_TOKEN` + optional `ANDROID_ALLOWED_USERS` device allowlist.
|
||||
`IRIS_TOKEN` + optional `IRIS_ALLOWED_USERS` device allowlist.
|
||||
Per-device revocable tokens are a stretch.
|
||||
4. **Desktop video backend (M6).** *Default:* `libmpv`/`mpv`-backed Compose
|
||||
surface, WebView fallback. Confirm `mpv` availability on target OSes during
|
||||
@@ -54,7 +56,7 @@ will proceed with unless you say otherwise.
|
||||
recommended remote path; WSS + reverse proxy as alternatives. No public bind
|
||||
by default.
|
||||
6. **Streaming cadence (M2).** If live updates look chunky, tune
|
||||
`display.platforms.android.streaming` / consumer edit interval. *Default:*
|
||||
`display.platforms.iris.streaming` / consumer edit interval. *Default:*
|
||||
follow global streaming config.
|
||||
7. **App package name / branding.** *Default:* applicationId `dev.iris.app`,
|
||||
app name "Iris". Confirm final product name + package + icon.
|
||||
@@ -72,4 +74,4 @@ will proceed with unless you say otherwise.
|
||||
- End-to-end encryption (transport WSS only).
|
||||
- Standalone-cron delivery while the gateway process is fully down (best-effort
|
||||
push only).
|
||||
- iOS.
|
||||
- iOS.
|
||||
@@ -153,7 +153,7 @@ Config + setup per provider — `web_server.py` `/api/memory/providers/*`.
|
||||
`app/shared/.../protocol/Protocol.kt` → `docs/protocol/frames.schema.json`).
|
||||
2. **Security is the real gate.** The single-user model in
|
||||
`09-pairing-security.md` still holds, but control frames widen the blast
|
||||
radius of a leaked `ANDROID_TOKEN`. Sensitive operations (env/secrets,
|
||||
radius of a leaked `IRIS_TOKEN`. Sensitive operations (env/secrets,
|
||||
config writes, gateway restart, profile deletion) should get either an
|
||||
in-app confirmation step or a capability flag negotiated at pairing.
|
||||
3. **Read-heavy first.** Most of the value is in list/view frames (cheap,
|
||||
|
||||
@@ -71,7 +71,7 @@ acceptable alternative if preferred).
|
||||
|
||||
```
|
||||
┌──────────────────────── hermes gateway process ───────────────────────┐
|
||||
│ AndroidAdapter │
|
||||
│ IrisAdapter │
|
||||
│ │ frames (same protocol.Frame objects) │
|
||||
│ ▼ │
|
||||
│ _broadcast_or_log ──► outbox.append(cursor) ──► push (if no live) │
|
||||
@@ -105,7 +105,7 @@ over HTTP whenever the WS is down.
|
||||
|
||||
## 19.4 Gateway: `gateway-plugin/http_server.py`
|
||||
|
||||
New module, started/stopped by `AndroidAdapter.connect()`/`disconnect()` next
|
||||
New module, started/stopped by `IrisAdapter.connect()`/`disconnect()` next
|
||||
to the WS server.
|
||||
|
||||
- **Server:** `http.server.ThreadingHTTPServer` + `BaseHTTPRequestHandler`,
|
||||
@@ -114,8 +114,8 @@ to the WS server.
|
||||
into the gateway's asyncio loop with
|
||||
`asyncio.run_coroutine_threadsafe(coro, loop)` (the loop is captured at
|
||||
start, same loop the WS server runs on).
|
||||
- **Config:** `ANDROID_HTTP_PORT` (default **8791**), same bind host as the WS
|
||||
(`ANDROID_WS_HOST`). Optional TLS via `ANDROID_HTTP_CERT`/`ANDROID_HTTP_KEY`
|
||||
- **Config:** `IRIS_HTTP_PORT` (default **8791**), same bind host as the WS
|
||||
(`IRIS_WS_HOST`). Optional TLS via `IRIS_HTTP_CERT`/`IRIS_HTTP_KEY`
|
||||
(`ssl.SSLContext` on the server) — same posture as the WS: plaintext on a
|
||||
trusted LAN by default, TLS for remote/Tailscale setups.
|
||||
- **Bind failure is NON-fatal** (unlike the WS): log a warning, disable the
|
||||
@@ -151,7 +151,7 @@ Wire format (standard SSE, three fields):
|
||||
```
|
||||
id: 1043
|
||||
event: frame
|
||||
data: {"v":1,"type":"message","chat_id":"android:default",...}
|
||||
data: {"v":1,"type":"message","chat_id":"default",...}
|
||||
|
||||
: hb ← comment heartbeat every 15 s (keeps proxies alive)
|
||||
```
|
||||
|
||||
@@ -0,0 +1,322 @@
|
||||
# 20 — QR Pairing (terminal QR + in-app scanner)
|
||||
|
||||
**Status: implemented (M8, 2026-08-22).**
|
||||
|
||||
Closes gap #12 in `09-pairing-security.md` ("in-app QR scanner") and implements
|
||||
the QR branch of the §9.2 pairing flow, which the docs already promise but the
|
||||
code never delivered: today `interactive_setup` prints the pairing URL as
|
||||
plain text only, and the app has no `iris://pair` parser at all.
|
||||
|
||||
Two halves, independent and shippable separately:
|
||||
|
||||
- **A — Gateway:** `hermes gateway setup` renders a scannable QR in the
|
||||
terminal encoding `iris://pair?host=…&port=…&token=…`. **Zero new Python
|
||||
dependencies** (pure stdlib encoder).
|
||||
- **B — App:** a "Scan QR" button on the Android Connect screen (CameraX +
|
||||
ML Kit, on-device, no Play services) that pre-fills URL + token. Not added
|
||||
to desktop. Plus an `iris://pair` deep link so *any* scanner (system camera
|
||||
app, other phones) can route the QR into the app.
|
||||
|
||||
---
|
||||
|
||||
## 20.1 Current state (what exists today)
|
||||
|
||||
| Piece | State | Location |
|
||||
| ------- | ------- | ---------- |
|
||||
| QR payload format | ✅ implemented | `gateway-plugin/pairing.py` → `qr_payload(host, port, token, secure)` → `iris://pair?host=<lan-ip>&port=8791&secure=0&token=<64-hex>` |
|
||||
| Terminal QR rendering | ❌ missing | `gateway-plugin/adapter.py` → `interactive_setup()` prints the URL text only |
|
||||
| App `iris://pair` parser | ❌ missing | app has manual URL + token entry only (`ConnectScreen`) |
|
||||
| In-app camera scan | ❌ missing | no camera deps anywhere in `app/` |
|
||||
| `iris://` deep link | ⚠️ partial | manifest handles `iris://chat/<id>` only (`androidApp/.../AndroidManifest.xml`, `MainActivity.handleDeepLink`) |
|
||||
| QR libs in hermes venv | ❌ absent | `qrcode`/`segno` not installed; `Pillow` is a hermes core dep but only renders images — the QR *matrix* algorithm is still needed either way |
|
||||
|
||||
Payload size: `iris://pair?host=192.168.x.x&port=8791&secure=0&token=<64 hex>`
|
||||
≈ **118 bytes** → QR version **7 at EC level M** (capacity 122 bytes) or v6 at
|
||||
L (134). The encoder must therefore support at least versions 1–8; we target
|
||||
1–10.
|
||||
|
||||
---
|
||||
|
||||
## 20.2 Part A — terminal QR in `interactive_setup`
|
||||
|
||||
### A1. Pure-stdlib QR encoder — `gateway-plugin/qr.py` (new file)
|
||||
|
||||
A self-contained ISO/IEC 18004 encoder, **stdlib only** (no `qrcode`, no
|
||||
`segno`, no Pillow). Scope is deliberately minimal — we only ever encode
|
||||
ASCII pairing URLs:
|
||||
|
||||
- **Mode:** byte mode only (no alphanumeric/numeric/kanji paths).
|
||||
- **Error correction:** level **M** (15 %); auto-fallback to **L** if the
|
||||
payload doesn't fit at M within the version cap.
|
||||
- **Versions:** 1–10, auto-selected (smallest version whose capacity fits).
|
||||
Payloads that don't fit v10-L raise `QrTooLongError` (caller falls back to
|
||||
text-only output — see A3).
|
||||
- **Components** (all well-known, spec-stable algorithms):
|
||||
1. Data encoding: mode indicator `0100`, 8-bit char count (8 bits for
|
||||
v1–9, 16 bits for v10), payload bytes, terminator, padding
|
||||
(`0xEC`/`0x11` alternation).
|
||||
2. Reed–Solomon error correction over GF(256), generator polynomial
|
||||
`0x11D`, per (version, EC level) block structure from the spec tables.
|
||||
3. Matrix placement: finder patterns + separators, timing patterns,
|
||||
alignment patterns (v2+), dark module, format info (BCH(15,5)),
|
||||
version info (v7+, BCH(18,6)), zig-zag data placement.
|
||||
4. Masking: all 8 masks, ISO penalty scoring (N1–N4), pick lowest.
|
||||
- **Public API:**
|
||||
|
||||
```python
|
||||
def qr_matrix(data: str) -> list[list[bool]]:
|
||||
"""Encode *data* (ASCII) into a module matrix (True = dark).
|
||||
Includes the 4-module quiet zone. Raises QrTooLongError."""
|
||||
```
|
||||
|
||||
~250–350 lines including the spec tables. No I/O, no globals, fully
|
||||
unit-testable.
|
||||
|
||||
### A2. Terminal renderer — `qr.py`
|
||||
|
||||
```python
|
||||
def render_qr(data: str) -> str:
|
||||
"""Render *data* as a terminal QR using Unicode half-blocks (▀).
|
||||
Returns '' (not an exception) when the payload is too long."""
|
||||
```
|
||||
|
||||
- Pair consecutive module rows into one character row: both dark → `█`,
|
||||
top dark → `▀`, bottom dark → `▄`, both light → space. (Matrix height
|
||||
including quiet zone is always even: `2·(17+4v)+8`.)
|
||||
- Output is a single string of `\n`-joined lines; the caller prints it.
|
||||
- No ANSI colors, no cursor tricks — must survive `less`, log files, and
|
||||
copy-paste.
|
||||
|
||||
### A3. Integration — `adapter.py:interactive_setup()`
|
||||
|
||||
After the existing "Pairing URL / Server URL" lines:
|
||||
|
||||
```python
|
||||
qr = render_qr(qr_payload(host, port, token))
|
||||
if qr:
|
||||
print_info("Scan with the Iris app (Connect → Scan QR) or any camera app:")
|
||||
print(qr)
|
||||
else:
|
||||
print_warning("QR too large to render; use the pairing URL above.")
|
||||
```
|
||||
|
||||
- The **URL text lines stay** — the QR is a convenience, not a replacement
|
||||
(terminals without UTF-8 still work, and the text is copy-pasteable).
|
||||
- Printed on every setup run (new *and* existing token), consistent with the
|
||||
URL lines which already print the token in cleartext.
|
||||
- **Security note:** no new exposure — the token is already printed in the
|
||||
pairing URL line today; the QR is the same bytes in a different encoding,
|
||||
on the same operator-only stdout. (Gap #5 in the §9.7 table already
|
||||
documents the stdout token print.)
|
||||
|
||||
### A4. Tests — `hermes-agent/tests/gateway/test_android.py`
|
||||
|
||||
The test file is a thin mirror importing the **live `gateway-plugin/`
|
||||
package**, so new tests land there:
|
||||
|
||||
1. **Fixed test vectors** (guard against silent algorithm drift): at least
|
||||
two known-good (data → matrix) pairs from public QR test vectors
|
||||
(e.g. the ISO 18004 annex examples / the classic `KARAT` v2-L vector).
|
||||
Assert the full matrix, not just dimensions.
|
||||
2. **Round-trip via payload:** `qr_matrix(qr_payload(h, p, t))` has the
|
||||
expected version/size for a 64-hex token (`17 + 4·7 = 45` modules at
|
||||
v7-M, +8 quiet zone).
|
||||
3. **Renderer shape:** every line equal length, height = half of matrix
|
||||
height, quiet zone renders as blank border, only the 4 block chars +
|
||||
space appear.
|
||||
4. **`QrTooLongError` / `render_qr` → `""`** for a payload beyond v10-L.
|
||||
5. **`interactive_setup` smoke:** with sandboxed HERMES_HOME (conftest
|
||||
already does this), capture stdout and assert the QR block appears after
|
||||
the pairing URL line.
|
||||
|
||||
Run: `cd hermes-agent && scripts/run_tests.sh tests/gateway/test_android.py`.
|
||||
|
||||
---
|
||||
|
||||
## 20.3 Part B — in-app scanner (Android only)
|
||||
|
||||
### B1. Dependencies — `app/shared/build.gradle.kts`, `androidMain.dependencies` only
|
||||
|
||||
| Dependency | Why |
|
||||
| ------------ | ----- |
|
||||
| `androidx.camera:camera-camera2` | Camera access (CameraX) |
|
||||
| `androidx.camera:camera-lifecycle` | Lifecycle-aware binding |
|
||||
| `androidx.camera:camera-view` | `PreviewView` for the scan surface |
|
||||
| `com.google.mlkit:barcode-scanning` | On-device QR decode; **no Google Play services required** (self-contained model) |
|
||||
|
||||
- Chosen over `zxing-android-embedded` (decided 2026-07-20): ML Kit has
|
||||
better accuracy/latency, is maintained by Google, and works fully
|
||||
on-device without GMS.
|
||||
- These go in **`androidMain`** only — the no-new-dep rule covers
|
||||
`gateway-plugin/`, and the app already carries OkHttp/SQLDelight/KCEF/etc.
|
||||
Desktop is untouched.
|
||||
- `minSdk 29` is fine for all four (ML Kit barcode needs 21+).
|
||||
- Versions go in the existing version catalog / `composeVersion`-style
|
||||
constants at the top of the build file (follow the current pattern).
|
||||
|
||||
### B2. Scanner activity — `shared/src/androidMain/kotlin/iris/platform/QrScanActivity.kt` (new)
|
||||
|
||||
A minimal `ComponentActivity` (not a Fragment, no nav graph):
|
||||
|
||||
- Layout: full-screen `PreviewView` + overlay hint text ("Point at the QR
|
||||
code") + close button.
|
||||
- `ImageAnalysis` (STRATEGY_LATEST, YUV_420_888) →
|
||||
`BarcodeScannerOptions(FORMAT_QR_CODE)` → first result →
|
||||
`setResult(RESULT_OK, Intent().putExtra("iris.qr.text", raw))` → finish.
|
||||
- **Runtime permission:** request `CAMERA` on launch; on denial show a
|
||||
message + close (the Connect screen still has manual entry).
|
||||
- Registered in `shared/src/androidMain/AndroidManifest.xml` (or the
|
||||
androidApp manifest — follow where `NtfyListenerService` is declared)
|
||||
with `android:exported="false"`, `android:theme` reusing the app theme.
|
||||
- Manifest additions (androidApp manifest):
|
||||
|
||||
```xml
|
||||
<uses-permission android:name="android.permission.CAMERA" />
|
||||
<uses-feature android:name="android.hardware.camera" android:required="false" />
|
||||
```
|
||||
|
||||
`required="false"` so the app stays installable on camera-less devices
|
||||
(the button then just reports "no camera").
|
||||
|
||||
### B3. Platform hook — `expect`/`actual`
|
||||
|
||||
`shared/src/commonMain/kotlin/iris/platform/PlatformQr.kt` (new):
|
||||
|
||||
```kotlin
|
||||
/** Launch the QR scanner. [onResult] gets the decoded text, or null when
|
||||
* the user cancelled / no camera / permission denied. Desktop: no-op. */
|
||||
expect fun scanQrCode(onResult: (String?) -> Unit)
|
||||
```
|
||||
|
||||
- **androidMain actual:** `ActivityResultLauncher` (from the Compose
|
||||
`LocalContext`) starting `QrScanActivity`; maps `RESULT_OK` → text,
|
||||
everything else → `null`.
|
||||
- **desktopMain actual:** `onResult(null)` immediately (the button is
|
||||
hidden on desktop anyway — see B4; the no-op keeps the `expect` total).
|
||||
|
||||
### B4. Connect screen button — `ConnectScreen.kt`
|
||||
|
||||
- New **"Scan QR"** `Button` below the token field, rendered only when
|
||||
`!isDesktop` (`iris.platform.isDesktop` already exists).
|
||||
- On tap: `scanQrCode { raw -> … }`; on non-null `raw`:
|
||||
- `PairLink.parse(raw)` (B5) → pre-fill `url` and `token` state, clear
|
||||
error, and **do not auto-connect** — the user still taps
|
||||
"Test & Connect" (pairing stays an explicit act, per §10.8).
|
||||
- Parse failure → set `error` to "Not a pairing QR code" (don't echo the
|
||||
raw payload — it may contain someone else's token).
|
||||
- On `null` (cancel/denied): no-op, no error.
|
||||
|
||||
### B5. Pair-link parser — `shared/src/commonMain/kotlin/iris/util/PairLink.kt` (new)
|
||||
|
||||
```kotlin
|
||||
data class PairLink(val url: String, val token: String)
|
||||
|
||||
object PairLink {
|
||||
/** Parse `iris://pair?host=…&port=…&secure=…&token=…` → PairLink.
|
||||
* Returns null on any malformation. */
|
||||
fun parse(raw: String): PairLink?
|
||||
}
|
||||
```
|
||||
|
||||
- Accepts exactly scheme `iris`, host `pair` (case-insensitive scheme).
|
||||
- Required: `host` (non-empty), `token` (non-empty). `port` defaults to
|
||||
`8791` (the HTTP default, `docs/19`); `secure` defaults to `0`.
|
||||
- Builds `url` as `http(s)://<host>:<port>`; validates port 1–65535.
|
||||
- URL-decodes `host`/`token` (the Python side `quote()`s them).
|
||||
- Pure function, no platform imports → **unit-tested in `jvmTest`**
|
||||
(`:shared:testAndroidHostTest` / `:shared:desktopTest` both run it):
|
||||
valid link, missing token, bad port, wrong scheme, wrong host,
|
||||
percent-encoded host, secure=1 → https, default port.
|
||||
|
||||
### B6. `iris://pair` deep link (system-scanner fallback)
|
||||
|
||||
So a QR scanned by *any* app (phone's built-in scanner, a friend's phone)
|
||||
lands in Iris:
|
||||
|
||||
- Manifest: extend the existing `VIEW` intent-filter block (or add a
|
||||
sibling) with `<data android:scheme="iris" android:host="pair" />`.
|
||||
- `MainActivity.handleDeepLink`: on `iris://pair` → `PairLink.parse(uri)` →
|
||||
stash into a `mutableStateOf<PairLink?>` passed into `IrisApp` →
|
||||
`ConnectScreen` receives it as `prefillUrl`/`prefillToken` (the params
|
||||
already exist). If the app is already connected, ignore (or surface in
|
||||
Settings later — out of scope).
|
||||
- This reuses B5's parser; add one test for the URI shape Android delivers.
|
||||
|
||||
---
|
||||
|
||||
## 20.4 Part C — doc updates (with the implementation)
|
||||
|
||||
| Doc | Change |
|
||||
| ----- | -------- |
|
||||
| `09-pairing-security.md` | Gap #12 → **implemented** (fix the stale `adapter.py:648-654` reference while at it); §9.2 QR branch no longer aspirational |
|
||||
| `10-android-app.md` §10.8 | "or scan QR" becomes real: scanner button + deep link, camera permission |
|
||||
| `14-milestones.md` | New **M8 — QR pairing** section (acceptance criteria below) |
|
||||
| `16-open-questions.md` | Record decision: ML Kit over zxing; pure-stdlib encoder over vendoring `segno` |
|
||||
| `README.md` | Reading-order table: add row 20 |
|
||||
|
||||
---
|
||||
|
||||
## 20.5 Work breakdown & sequencing
|
||||
|
||||
Ordered so each step is independently verifiable; A and B can be
|
||||
interleaved (different languages, no shared surface).
|
||||
|
||||
| # | Task | Verify |
|
||||
| --- | ------ | -------- |
|
||||
| 1 | `qr.py` encoder + renderer (A1/A2) | new unit tests green (A4.1–4.4) |
|
||||
| 2 | `interactive_setup` integration (A3) | A4.5 + manual: `hermes gateway setup` in a real terminal shows a scannable QR (scan with the phone's *system* camera app as the decoder oracle) |
|
||||
| 3 | `PairLink` parser + jvmTest (B5) | `./gradlew :shared:testAndroidHostTest` |
|
||||
| 4 | Deps + manifest + `QrScanActivity` (B1/B2) | `:androidApp:assembleDebug` |
|
||||
| 5 | `PlatformQr` expect/actual + Connect button (B3/B4) | `:androidApp:assembleDebug` + `:desktopApp:run` (button absent, no crash) |
|
||||
| 6 | `iris://pair` deep link (B6) | ADB: `adb shell am start -a android.intent.action.VIEW -d "iris://pair?host=…&port=…&token=…"` → Connect screen pre-filled |
|
||||
| 7 | Doc updates (Part C) | — |
|
||||
|
||||
**On-device E2E (final gate, per `13-testing.md` ADB workflow):**
|
||||
|
||||
1. `hermes gateway setup` on the gateway host → QR in terminal.
|
||||
2. Phone: `adb shell am start -n dev.iris.app/.MainActivity` → Connect →
|
||||
**Scan QR** → grant camera → point at the terminal (screenshot the QR
|
||||
onto a second screen if needed; the reference device is API 29 —
|
||||
verify CameraX works on the MIX 2S in step 4 before building the rest).
|
||||
3. Fields pre-filled → **Test & Connect** → chat screen.
|
||||
4. Repeat via deep link (step 6 command) with a *different* token.
|
||||
5. Negative: scan a non-pairing QR (e.g. a website) → "Not a pairing QR
|
||||
code", fields untouched.
|
||||
|
||||
---
|
||||
|
||||
## 20.6 Acceptance criteria (M8)
|
||||
|
||||
- [ ] `hermes gateway setup` prints a QR that a stock Android camera app
|
||||
decodes to exactly `qr_payload(host, port, token)`.
|
||||
- [ ] QR encoder: fixed test vectors + size/round-trip tests green;
|
||||
**zero** new entries in the plugin's import surface (stdlib only —
|
||||
verifiable by `ruff`/import scan).
|
||||
- [ ] Android: Connect screen shows **Scan QR** (hidden on desktop);
|
||||
scanning the setup QR pre-fills URL + token; "Test & Connect" pairs.
|
||||
- [ ] Camera permission denied → graceful message, manual entry still works.
|
||||
- [ ] `iris://pair` deep link pre-fills the Connect screen (ADB-verified).
|
||||
- [ ] `PairLink.parse` unit tests cover the matrix in B5.
|
||||
- [ ] Full Python suite green: `scripts/run_tests.sh` (no args).
|
||||
- [ ] Docs updated per Part C; gap #12 closed.
|
||||
|
||||
## 20.7 Risks & mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
| ------ | ------------ |
|
||||
| Hand-rolled QR encoder has a subtle bug | Fixed spec test vectors (A4.1) + the system-camera-app oracle in the E2E gate; scope locked to byte mode / v1–10 so the surface stays small |
|
||||
| Terminal without UTF-8 mangles the QR | URL text lines remain the primary path; QR is additive |
|
||||
| CameraX quirks on API 29 (MIX 2S) | Build the scanner activity first (task 4) and verify on-device before wiring the UI |
|
||||
| ML Kit model size (~4 MB) | Bundled in the APK, on-device, no runtime download — acceptable for this app's footprint |
|
||||
| Token in QR scanned by a bystander's phone | Same trust domain as the token already printed in the terminal; LAN pairing is operator-supervised by design (§9.2). Deep link only pre-fills — it never auto-connects |
|
||||
| `secure=1` (WSS) URLs | Parser already handles `secure` → `https://`; QR payload unchanged |
|
||||
|
||||
## 20.8 Explicit non-goals
|
||||
|
||||
- **QR display in the app** (showing a QR for other devices to scan) —
|
||||
single-device pairing today; revisit if multi-device lands.
|
||||
- **`hermes android pair` stretch CLI** (re-issue token + new QR,
|
||||
`09-pairing-security.md` §9.2 line 48) — separate backlog item.
|
||||
- **WSS cert pinning** (gap #6) — orthogonal; QR carries `secure=1`
|
||||
already, pinning is app-side.
|
||||
- **iOS scanner** — no iOS target (per `00-overview.md`).
|
||||
+2
-1
@@ -41,6 +41,7 @@ top-to-bottom once, then use the numbered docs as a lookup while implementing.
|
||||
| 16 | [`16-open-questions.md`](16-open-questions.md) | Decisions made + open items. |
|
||||
| 17 | [`17-future-control-surface.md`](17-future-control-surface.md) | **Backlog** — what the app could control beyond chat (cron, kanban, models, …). |
|
||||
| 19 | [`19-http-fallback-transport.md`](19-http-fallback-transport.md) | **Design** — HTTP fallback leg (POST + SSE/long-poll) so the app can send/receive when the WS is down. |
|
||||
| 20 | [`20-qr-pairing.md`](20-qr-pairing.md) | Terminal QR at `gateway setup` + in-app QR scanner (Android) + `iris://pair` deep link. |
|
||||
|
||||
Machine-readable / diagrams:
|
||||
|
||||
@@ -51,7 +52,7 @@ Machine-readable / diagrams:
|
||||
|
||||
## The three deliverables (one monorepo)
|
||||
|
||||
1. **`gateway-plugin/`** — a Python hermes **platform plugin** named `android`.
|
||||
1. **`gateway-plugin/`** — a Python hermes **platform plugin** named `iris`.
|
||||
Runs inside the `hermes gateway` process. Opens a WebSocket server the apps
|
||||
connect to. Implements the full `BasePlatformAdapter` contract. **Zero new
|
||||
Python dependencies, zero hermes-core changes.**
|
||||
|
||||
@@ -17,7 +17,7 @@ flowchart TB
|
||||
WSS["WebSocket SERVER<br/>(websockets) ws://host:8790/ws"]
|
||||
end
|
||||
AGENT -->|legacy stream callbacks| ADAPTER
|
||||
CRON -->|deliver=android:chat:thread| ADAPTER
|
||||
CRON -->|deliver=iris:chat:thread| ADAPTER
|
||||
ADAPTER <--> WSS
|
||||
ADAPTER <--> OUTBOX
|
||||
ADAPTER <--> PUSH
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"v": { "type": "integer", "const": 1, "description": "Protocol version." },
|
||||
"id": { "type": "integer", "description": "Request id; present on requests and their responses/acks. Absent on pure events." },
|
||||
"type": { "type": "string", "description": "Frame type (see frame_types)." },
|
||||
"chat_id": { "type": "string", "description": "Optional chat scope (e.g. android:default, android:chan_7)." },
|
||||
"chat_id": { "type": "string", "description": "Optional chat scope (e.g. default, chan_7)." },
|
||||
"thread_id": { "type": "string", "description": "Optional thread scope within a chat_id." },
|
||||
"cursor": { "type": "integer", "description": "Outbox cursor the frame was parked under. Present ONLY on frames replayed by sync (live frames carry none). The app skips re-notifying replayed frames with cursor <= last_pushed_cursor (docs/08 §8.7)." },
|
||||
"payload": { "type": "object", "description": "Type-specific payload." }
|
||||
|
||||
+20
-17
@@ -9,7 +9,7 @@ just the steps.
|
||||
## Prerequisites
|
||||
|
||||
| Where | You need |
|
||||
|---|---|
|
||||
| --- | --- |
|
||||
| Gateway host | hermes installed with its venv (`cd hermes-agent && uv sync`, see [`12-toolchain.md` §12.4](12-toolchain.md)) |
|
||||
| Android build machine | JDK 17, Android SDK with `ANDROID_HOME` set (or `app/local.properties`), ADB with a connected device |
|
||||
| Desktop build machine | JDK 17 only |
|
||||
@@ -24,8 +24,8 @@ root):
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.hermes/plugins
|
||||
ln -s "$PWD/gateway-plugin" ~/.hermes/plugins/android
|
||||
hermes gateway status # should list "android"
|
||||
ln -s "$PWD/gateway-plugin" ~/.hermes/plugins/iris
|
||||
hermes gateway status # should list "iris"
|
||||
```
|
||||
|
||||
Run the interactive setup:
|
||||
@@ -36,12 +36,13 @@ hermes gateway setup
|
||||
|
||||
What it does:
|
||||
|
||||
- Generates `ANDROID_TOKEN` (64 hex chars) if none exists and stores it in
|
||||
- Generates `IRIS_TOKEN` (64 hex chars) if none exists and stores it in
|
||||
`~/.hermes/.env` (it prints the token once, at generation).
|
||||
- Prompts for the WS bind host (default `127.0.0.1`), port (default `8790`),
|
||||
and push backend (`fcm` or `ntfy`, default `fcm`).
|
||||
- Prints the pairing payload (a QR-encodable `iris://pair?host=…&port=…&token=…`
|
||||
string) and the server URL (`ws://<host>:8790/ws`).
|
||||
string), a scannable QR of that payload, and the server URL
|
||||
(`ws://<host>:8790/ws`).
|
||||
|
||||
Then start the gateway:
|
||||
|
||||
@@ -52,7 +53,7 @@ hermes gateway # or: hermes gateway restart after config changes
|
||||
> **Note:** the default bind host `127.0.0.1` only accepts connections from the
|
||||
> gateway host itself (e.g. a desktop app on the same machine). For a phone on
|
||||
> the LAN, re-run `hermes gateway setup` (or edit `~/.hermes/.env`) and set
|
||||
> `ANDROID_WS_HOST` to the host's LAN IP (e.g. `192.168.1.10`).
|
||||
> `IRIS_WS_HOST` to the host's LAN IP (e.g. `192.168.1.10`).
|
||||
|
||||
## 2. Android app
|
||||
|
||||
@@ -68,12 +69,14 @@ First run opens the **Connect** screen:
|
||||
1. **Server URL** — `ws://<gateway-ip>:8790/ws` (the URL printed by
|
||||
`hermes gateway setup`; use the LAN IP, not `127.0.0.1`, from a phone).
|
||||
2. **Pairing token** — from the `hermes gateway setup` output, or
|
||||
`grep ANDROID_TOKEN ~/.hermes/.env` on the gateway host.
|
||||
`grep IRIS_TOKEN ~/.hermes/.env` on the gateway host.
|
||||
3. **Test & Connect** — performs a real `hello` (the auth leg), then saves the
|
||||
pairing and connects.
|
||||
|
||||
> **Honest limitation:** QR scanning is **not** supported in the app yet. The
|
||||
> server prints a QR payload, but pairing is manual URL + token entry only.
|
||||
> **Scan QR (Android):** the Connect screen has a **Scan QR** button (CameraX +
|
||||
> ML Kit) that reads the QR printed by `hermes gateway setup` and pre-fills the
|
||||
> URL + token. Desktop has no camera, so it uses manual entry. An `iris://pair`
|
||||
> deep link (from any scanner) pre-fills the same way.
|
||||
|
||||
## 3. Desktop app
|
||||
|
||||
@@ -104,8 +107,8 @@ high-priority events (approvals, clarifies, cron) even when a device is live.
|
||||
`app/androidApp/`.
|
||||
2. Create a service account (Project settings → Service accounts → Generate new
|
||||
private key) and store the JSON path in `~/.hermes/.env`:
|
||||
`ANDROID_FCM_SERVICE_ACCOUNT=/path/to/service-account.json`.
|
||||
3. Keep `ANDROID_PUSH_BACKEND=fcm` (the default).
|
||||
`IRIS_FCM_SERVICE_ACCOUNT=/path/to/service-account.json`.
|
||||
3. Keep `IRIS_PUSH_BACKEND=fcm` (the default).
|
||||
|
||||
Without a Firebase project the FCM path is **inert** (the app's FCM service
|
||||
does nothing) — use ntfy below, or add Firebase later.
|
||||
@@ -116,7 +119,7 @@ backgrounded; tapping one deep-links to the chat.
|
||||
### ntfy (zero-config fallback)
|
||||
|
||||
```
|
||||
ANDROID_PUSH_BACKEND=ntfy
|
||||
IRIS_PUSH_BACKEND=ntfy
|
||||
```
|
||||
|
||||
- The device **generates its own topic** automatically (no `NTFY_TOPIC` needed);
|
||||
@@ -135,7 +138,7 @@ the app is off; incoming pushes trigger a silent sync.
|
||||
the app connects to `ws://<tailnet-ip>:8790/ws`. No public exposure.
|
||||
- **Reverse proxy / tunnel** (Caddy, Cloudflare Tunnel, ngrok): terminate TLS at
|
||||
the edge, forward the WebSocket to `127.0.0.1:8790`.
|
||||
- **WSS:** set `ANDROID_WS_CERT` / `ANDROID_WS_KEY` (paths, in
|
||||
- **WSS:** set `IRIS_WS_CERT` / `IRIS_WS_KEY` (paths, in
|
||||
`~/.hermes/.env`) and the server serves `wss://` instead of `ws://`.
|
||||
|
||||
> **Honest limitation:** the app has **no certificate pinning** yet, so
|
||||
@@ -145,8 +148,8 @@ the app is off; incoming pushes trigger a silent sync.
|
||||
## 6. Troubleshooting
|
||||
|
||||
| Symptom | Likely cause / fix |
|
||||
|---|---|
|
||||
| `auth failed` / `error {code:"auth"}` on connect | Wrong token. Check `ANDROID_TOKEN` in `~/.hermes/.env` on the gateway host (setup prints it only when it generates it). |
|
||||
| --- | --- |
|
||||
| `auth failed` / `error {code:"auth"}` on connect | Wrong token. Check `IRIS_TOKEN` in `~/.hermes/.env` on the gateway host (setup prints it only when it generates it). |
|
||||
| Connection refused | Gateway not running (`hermes gateway status`); wrong URL (port `8790`, path `/ws`, LAN IP instead of `127.0.0.1` from a phone); firewall blocking the port. |
|
||||
| Push not arriving | Backend not configured (gateway log: `push backend … not configured`); app backgrounded with no working backend; ntfy.sh SSE flakiness — use a self-hosted ntfy. |
|
||||
| Desktop jpackage launcher warning | Non-fatal (JDK-8348560 on Linux JDK 17); the app runs and connects regardless. |
|
||||
@@ -159,7 +162,7 @@ import asyncio, json, websockets
|
||||
async def main():
|
||||
async with websockets.connect("ws://127.0.0.1:8790/ws") as ws:
|
||||
await ws.send(json.dumps({"v":1,"type":"hello","payload":{
|
||||
"token":"<ANDROID_TOKEN>","device_id":"test","device_name":"probe",
|
||||
"token":"<IRIS_TOKEN>","device_id":"test","device_name":"probe",
|
||||
"caps":{"min_protocol":1}}}))
|
||||
print("recv:", await ws.recv())
|
||||
asyncio.run(main())
|
||||
@@ -167,4 +170,4 @@ PY
|
||||
```
|
||||
|
||||
Expect a `hello.ack`. If you get `error {code:"auth"}`, the token/host/port is
|
||||
wrong.
|
||||
wrong.
|
||||
Reference in new issue
Block a user