Add QR pairing (terminal QR, in-app scanner, iris://pair deep link)
This commit is contained in:
1 parent
27dc7917f2
commit
7a6d922d12
63 files changed
+2073
-630
No files matched your search
+56
-9
@@ -1,4 +1,4 @@
|
||||
# 14 — Milestones (M0–M7)
|
||||
# 14 — Milestones (M0–M8)
|
||||
|
||||
Phased delivery. Each milestone ends with a **demo** (on-device where noted) and
|
||||
has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
@@ -6,7 +6,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
---
|
||||
|
||||
## M0 — Toolchain & scaffolding
|
||||
|
||||
**Goal:** everything builds; the plugin is discoverable; the repo is safe.
|
||||
|
||||
- [X] Install JDK 17, Android SDK, set `ANDROID_HOME` (`12-toolchain.md`).
|
||||
- [X] `cd hermes-agent && uv sync` (hermes venv works).
|
||||
- [X] Create monorepo scaffold (`02-monorepo.md`): `gateway-plugin/`, `app/`
|
||||
@@ -16,20 +18,22 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
- [X] CMP project builds empty: `./gradlew :androidApp:assembleDebug`,
|
||||
`./gradlew :desktopApp:run` (blank window).
|
||||
- [X] Plugin skeleton: `plugin.yaml` + `adapter.py` with `register(ctx)` + a
|
||||
no-op `AndroidAdapter` → `hermes gateway status` lists **android**.
|
||||
- **Demo:** `hermes gateway status` shows `android`; `./gradlew
|
||||
no-op `IrisAdapter` → `hermes gateway status` lists **iris**.
|
||||
- **Demo:** `hermes gateway status` shows `iris`; `./gradlew
|
||||
:androidApp:installDebug` installs a blank app on the MIX 2S.
|
||||
- **Accept:** blank app installs + launches on-device; plugin visible in
|
||||
`hermes gateway status`; `hermes-agent/` is git-ignored (verify with
|
||||
`git status --ignored`).
|
||||
|
||||
## M1 — Gateway core loop (text round-trip)
|
||||
|
||||
**Goal:** pair + send a text message + get a (non-streaming) reply.
|
||||
|
||||
- [X] WS server (`ws_server.py`): bind, `hello` auth (constant-time),
|
||||
`hello.ack`, heartbeat, connection registry.
|
||||
- [X] `AndroidAdapter.send()` → `message` frame; inbound `message.send` →
|
||||
- [X] `IrisAdapter.send()` → `message` frame; inbound `message.send` →
|
||||
`MessageEvent` → `handle_message`.
|
||||
- [X] Pairing store + `ANDROID_TOKEN`; QR payload in `interactive_setup`.
|
||||
- [X] Pairing store + `IRIS_TOKEN`; QR payload in `interactive_setup`.
|
||||
- [X] App: Connect screen (URL+token, real `hello` test), `GatewayClient`
|
||||
(connect + reconnect), ChatScreen sends + renders `message`.
|
||||
- [X] `ws_probe.py` harness drives a real turn.
|
||||
@@ -38,9 +42,11 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
reconnect after gateway restart re-pairs.
|
||||
|
||||
## M2 — Streaming + reasoning + tools + commentary
|
||||
|
||||
**Goal:** the "agent transparency" features.
|
||||
|
||||
- [X] Map consumer `send`/`edit_message` → `message.start/update/stop`.
|
||||
- [X] Reasoning: set `show_reasoning` for android; adapter splits prefix →
|
||||
- [X] Reasoning: set `show_reasoning` for iris; adapter splits prefix →
|
||||
`reasoning` field. **Verify format with `ws_probe.py`.** (The model
|
||||
returns a separate `reasoning_content` field. In the *streaming* case the
|
||||
gateway drops it — the stream consumer only forwards `content` and the
|
||||
@@ -64,12 +70,14 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
rendering; reasoning copy button works; frame shapes match `04-wire-protocol`.
|
||||
|
||||
## M3 — Channels/threads + cron + search
|
||||
|
||||
**Goal:** organization + cron delegation + search.
|
||||
|
||||
- [X] Channel directory (SQLite): default channel ensured; `channel.create/
|
||||
rename/set_default/delete` + `channel.*` frames.
|
||||
- [X] Threads: toggle in default chat; `thread_id` lanes; `create_handoff_thread`.
|
||||
- [X] `parse_target_ref_fn` + `cron_deliver_env_var` → cron
|
||||
`deliver=android:<chat>[:<thread>]` works.
|
||||
`deliver=iris:<chat>[:<thread>]` works.
|
||||
- [X] `search.py` FTS5 bridge; `search` frame (all / this-chat) → results.
|
||||
- [X] App: channel list (drawer/rail), thread toggle + topic switcher, "new
|
||||
channel" + "set as cron target", SearchScreen with scope toggle + jump.
|
||||
@@ -79,7 +87,7 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
isolate context; search scoping correct; channel list reconciles on events.
|
||||
- **Status (complete):** channel directory + threads + search + outbox sync
|
||||
verified end-to-end via `ws_probe.py` (create/rename/set_default/delete,
|
||||
thread lanes, FTS5 search, sync); cron `deliver=android:<chat>[:<thread>]`
|
||||
thread lanes, FTS5 search, sync); cron `deliver=iris:<chat>[:<thread>]`
|
||||
target resolution verified via `resolve_send_target`. App on-device: channel
|
||||
drawer, thread toggle + topic switcher, new channel, search overlay with
|
||||
jump. Minor UI gaps deferred to M7 polish: "set as cron target" is set via
|
||||
@@ -89,7 +97,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
not e2e-tested (it shares the verified resolution path).
|
||||
|
||||
## M4 — Media
|
||||
|
||||
**Goal:** attach + receive + play media.
|
||||
|
||||
- [X] Inbound: `media.upload` chunked → `cache_*_from_bytes` → `media_urls`;
|
||||
size limit + sha256 + MIME re-sniff.
|
||||
- [X] Outbound: `send_*` → `media.offer`; `media.pull` chunked; delivery-path
|
||||
@@ -117,12 +127,14 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
`04-wire-protocol.md` + `frames.schema.json`.
|
||||
|
||||
## M5 — Push + offline (FCM + ntfy)
|
||||
|
||||
**Goal:** reach the phone when backgrounded; catch up on reconnect.
|
||||
|
||||
- [x] Outbox (SQLite) + sync cursor; `sync`/`sync.done`; retention prune
|
||||
(row cap 5000 + prune banner, throttled 1/h).
|
||||
- [x] `push.py`: `FcmBackend` (HTTP v1 + service account, httpx; JWT via
|
||||
PyJWT+cryptography) + `NtfyBackend` (X-Data header); selected by
|
||||
`ANDROID_PUSH_BACKEND`. ntfy server exposed in `server_caps.push_ntfy_server`.
|
||||
`IRIS_PUSH_BACKEND`. ntfy server exposed in `server_caps.push_ntfy_server`.
|
||||
- [x] Fire push on no-live-subscriber; data payload for silent sync.
|
||||
High-priority kinds (approval/clarify/cron) push even when live.
|
||||
- [x] App: FCM service (`onNewToken` → `fcm.register`; inert without a
|
||||
@@ -143,7 +155,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
loss/dup (verified); banners show for foreground events (implemented).
|
||||
|
||||
## M6 — Desktop app
|
||||
|
||||
**Goal:** the same app on a big screen.
|
||||
|
||||
- [x] `desktopMain`: tray + OS notifications; `MediaPlayer` actual (mpv/WebView);
|
||||
`MediaPicker` actual (file dialog); `SecureStore` actual; window mgmt.
|
||||
- [x] Two-pane default layout; keyboard shortcuts; optional inspector pane.
|
||||
@@ -178,7 +192,9 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
macOS/Windows packaging are deferred to M7.
|
||||
|
||||
## M7 — Polish + E2E + docs
|
||||
|
||||
**Goal:** ship-quality.
|
||||
|
||||
- [x] Telegram-style layout pass (per reference image): header, bubbles, date
|
||||
separators, ✓✓, model/token footer, banner, bottom bar.
|
||||
- [x] Theming (dark default, accent), onboarding/pairing UX, empty/loading/
|
||||
@@ -222,7 +238,38 @@ has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||
|
||||
---
|
||||
|
||||
## M8 — QR pairing
|
||||
|
||||
**Goal:** QR-based pairing — a scannable QR at `hermes gateway setup` plus an
|
||||
in-app scanner and `iris://pair` deep link (closes gap #12, `docs/20`).
|
||||
|
||||
- [x] Pure-stdlib QR encoder + terminal renderer (`gateway-plugin/qr.py`):
|
||||
byte mode, EC M with L fallback, versions 1–10, ISO penalty masking,
|
||||
**zero new Python deps**.
|
||||
- [x] `interactive_setup` renders the QR after the pairing URL (text lines
|
||||
stay as the primary path).
|
||||
- [x] `PairLink` parser (`iris/util/PairLink.kt`) + jvmTest (valid/missing
|
||||
token/bad port/wrong scheme/wrong host/percent-encoded/secure/default
|
||||
port).
|
||||
- [x] CameraX + ML Kit scanner (`QrScanActivity`, on-device, no Play
|
||||
services) + Connect-screen **Scan QR** button (Android only; hidden on
|
||||
desktop) + `CAMERA` permission.
|
||||
- [x] `iris://pair` deep link (system-scanner / other-phone fallback) reusing
|
||||
the same parser.
|
||||
- **Accept:** `docs/20` §20.6; gap #12 in `09-pairing-security.md` closed.
|
||||
- **Status (2026-08-22):** Encoder cross-checked byte-for-byte against an
|
||||
independent reference and decoded by an independent decoder (zbarimg); fixed
|
||||
v1-M and v7-M matrix vectors lock the algorithm. `hermes gateway setup`
|
||||
prints a scannable QR (v7-M, 45 modules) for the 64-hex-token payload. App:
|
||||
Connect screen shows **Scan QR** (Android), which opens `QrScanActivity`
|
||||
(CameraX camera2 + ML Kit barcode), requests `CAMERA`, and pre-fills URL +
|
||||
token via `PairLink.parse` without auto-connecting; `iris://pair` deep link
|
||||
pre-fills the same way. Docs updated per `docs/20` Part C.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing notes
|
||||
|
||||
- **M1/M2 depend on the `ws_probe.py` harness** to lock frame shapes early —
|
||||
build it in M1.
|
||||
- **M3 (cron) and M5 (push) both touch the outbox** — build the outbox in M3,
|
||||
|
||||
Reference in new issue
Block a user