iris_x_hermes/docs/14-milestones.md

# 14 — Milestones (M0–M7)

Phased delivery. Each milestone ends with a **demo** (on-device where noted) and
has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.

---

## M0 — Toolchain & scaffolding
**Goal:** everything builds; the plugin is discoverable; the repo is safe.
- [X] Install JDK 17, Android SDK, set `ANDROID_HOME` (`12-toolchain.md`).
- [X] `cd hermes-agent && uv sync` (hermes venv works).
- [X] Create monorepo scaffold (`02-monorepo.md`): `gateway-plugin/`, `app/`
      (CMP: `shared`, `androidApp`, `desktopApp`), root `.gitignore`
      (**excludes `hermes-agent/`**), root `README.md`.
- [X] `git init` + a pre-commit/CI guard that fails if `hermes-agent/` is staged.
- [X] CMP project builds empty: `./gradlew :androidApp:assembleDebug`,
      `./gradlew :desktopApp:run` (blank window).
- [X] Plugin skeleton: `plugin.yaml` + `adapter.py` with `register(ctx)` + a
      no-op `AndroidAdapter` → `hermes gateway status` lists **android**.
- **Demo:** `hermes gateway status` shows `android`; `./gradlew
  :androidApp:installDebug` installs a blank app on the MIX 2S.
- **Accept:** blank app installs + launches on-device; plugin visible in
  `hermes gateway status`; `hermes-agent/` is git-ignored (verify with
  `git status --ignored`).

## M1 — Gateway core loop (text round-trip)
**Goal:** pair + send a text message + get a (non-streaming) reply.
- [X] WS server (`ws_server.py`): bind, `hello` auth (constant-time),
      `hello.ack`, heartbeat, connection registry.
- [X] `AndroidAdapter.send()` → `message` frame; inbound `message.send` →
      `MessageEvent` → `handle_message`.
- [X] Pairing store + `ANDROID_TOKEN`; QR payload in `interactive_setup`.
- [X] App: Connect screen (URL+token, real `hello` test), `GatewayClient`
      (connect + reconnect), ChatScreen sends + renders `message`.
- [X] `ws_probe.py` harness drives a real turn.
- **Demo (on-device):** pair the phone, send "hello", see the agent's reply.
- **Accept:** text round-trip works on-device; wrong token is rejected;
  reconnect after gateway restart re-pairs.

## M2 — Streaming + reasoning + tools + commentary
**Goal:** the "agent transparency" features.
- [X] Map consumer `send`/`edit_message` → `message.start/update/stop`.
- [X] Reasoning: set `show_reasoning` for android; adapter splits prefix →
      `reasoning` field. **Verify format with `ws_probe.py`.** (The model
      returns a separate `reasoning_content` field. In the *streaming* case the
      gateway drops it — the stream consumer only forwards `content` and the
      final send is suppressed — so the adapter captures it via the
      `on_stream_delta` plugin hook (`kind="reasoning"`, gated by
      `plugins.stream_reasoning_deltas: true`) and attaches it to
      `message.stop`. Non-streaming already carried it via the prefix split.)
- [X] Tool events: classify tool-progress `send()`s → structured
      `tool.start/progress/end` (turn-state machine). **Verify with probe.**
- [X] Commentary → `commentary` frames. Typing → `typing`.
- [X] App: live bubble (coalesced updates), `ReasoningBlock` (collapse + copy),
      `ToolCard` with **Everything/Truncated/Nothing** setting, dimmed
      `commentary` bubble.
- **Demo (on-device):** a multi-step prompt streams, shows reasoning above the
  answer, tool cards (toggle verbosity), and an intermediate beat.
  (Verified on the MIX 2S: streaming, tool cards + verbosity toggle, and the
  reasoning panel rendering above the answer. The commentary beat is
  model/agent-dependent and did not fire with the current model, but the
  rendering + frame handling are in place.)
- **Accept:** all four render correctly; tool verbosity setting changes
  rendering; reasoning copy button works; frame shapes match `04-wire-protocol`.

## M3 — Channels/threads + cron + search
**Goal:** organization + cron delegation + search.
- [X] Channel directory (SQLite): default channel ensured; `channel.create/
      rename/set_default/delete` + `channel.*` frames.
- [X] Threads: toggle in default chat; `thread_id` lanes; `create_handoff_thread`.
- [X] `parse_target_ref_fn` + `cron_deliver_env_var` → cron
      `deliver=android:<chat>[:<thread>]` works.
- [X] `search.py` FTS5 bridge; `search` frame (all / this-chat) → results.
- [X] App: channel list (drawer/rail), thread toggle + topic switcher, "new
      channel" + "set as cron target", SearchScreen with scope toggle + jump.
- **Demo (on-device):** create "Cron Reports"; create a cron job delivering to
  it; it fires into that channel; search finds a message (both scopes).
- **Accept:** cron output lands in the chosen channel (not default); threads
  isolate context; search scoping correct; channel list reconciles on events.
- **Status (complete):** channel directory + threads + search + outbox sync
  verified end-to-end via `ws_probe.py` (create/rename/set_default/delete,
  thread lanes, FTS5 search, sync); cron `deliver=android:<chat>[:<thread>]`
  target resolution verified via `resolve_send_target`. App on-device: channel
  drawer, thread toggle + topic switcher, new channel, search overlay with
  jump. Minor UI gaps deferred to M7 polish: "set as cron target" is set via
  the cron job's `deliver` field (no dedicated button); the search overlay
  defaults to scope "all" (the all/this-chat toggle is supported in the
  controller but not yet exposed). An actual cron job firing into a channel was
  not e2e-tested (it shares the verified resolution path).

## M4 — Media
**Goal:** attach + receive + play media.
- [X] Inbound: `media.upload` chunked → `cache_*_from_bytes` → `media_urls`;
      size limit + sha256 + MIME re-sniff.
- [X] Outbound: `send_*` → `media.offer`; `media.pull` chunked; delivery-path
      security.
- [X] App: SAF pickers + preview chips + upload; `media.pull` → cache;
      **ExoPlayer** inline (audio mini-player, video fullscreen/PiP); image/doc
      viewers.
- **Demo (on-device):** attach a photo + video (agent sees them); ask agent to
  send an image/video → plays live in-app.
- **Accept:** both directions work; over-limit rejected; playback is live;
  only allowed files are servable.
- **Status (complete):** Both directions verified end-to-end on the MIX 2S.
  Inbound: SAF-picked photo → chunked binary `media.upload` (256 KiB) →
  gateway size+sha256 verify + MIME re-sniff → `cache_image_from_bytes` →
  `media.upload.ack` → `message.send` with `media_refs` → agent vision
  described the image accurately. Outbound: agent `send_image` → `media.offer`
  → app auto-`media.pull` (chunked) → cache → image rendered inline. Live
  playback verified on-device: agent offered a 2s MP4 + 2s MP3 → app pulled
  both → ExoPlayer video player (blue frame, 00:02/00:02, controls) + audio
  mini-player rendered and playable. Over-limit rejection, sha256 mismatch,
  and delivery-path security are covered by the 17-test
  `tests/gateway/test_android.py` suite (all pass). Note: the app serializes
  pulls (single `binarySession` slot) via a `Mutex` so concurrent offers don't
  interleave their byte streams. `media.upload.ack` documented in
  `04-wire-protocol.md` + `frames.schema.json`.

## M5 — Push + offline (FCM + ntfy)
**Goal:** reach the phone when backgrounded; catch up on reconnect.
- [x] Outbox (SQLite) + sync cursor; `sync`/`sync.done`; retention prune
      (row cap 5000 + prune banner, throttled 1/h).
- [x] `push.py`: `FcmBackend` (HTTP v1 + service account, httpx; JWT via
      PyJWT+cryptography) + `NtfyBackend` (X-Data header); selected by
      `ANDROID_PUSH_BACKEND`. ntfy server exposed in `server_caps.push_ntfy_server`.
- [x] Fire push on no-live-subscriber; data payload for silent sync.
      High-priority kinds (approval/clarify/cron) push even when live.
- [x] App: FCM service (`onNewToken` → `fcm.register`; inert without a
      Firebase project), per-chat notification channels, deep-link
      (custom action + `iris://`), ntfy listener foreground-service fallback.
      App generates a device-specific ntfy topic; auto-sync on reconnect.
- [x] In-app `notification` banners (channel events, approval, clarify, cron,
      outbox-pruned); system notification mirror when backgrounded.
- **Demo (on-device, verified 2026-08-19):** device offline → message
  broadcast → frame parked in outbox → `push via ntfy -> <device>` fired →
  app reconnect auto-synced (cursor 0→8) → parked message rendered in-app.
  ntfy publish confirmed (POST 200). ntfy listener service runs (foreground
  notification posted); listener→system-notification leg not exercised live
  because ntfy.sh's public SSE endpoint was serving its web UI (not streams)
  during the test — verify against a self-hosted ntfy.
- **Accept:** push arrives when backgrounded (ntfy verified; FCM code path
  complete, needs a Firebase project to exercise); reconnect syncs with no
  loss/dup (verified); banners show for foreground events (implemented).

## M6 — Desktop app
**Goal:** the same app on a big screen.
- [ ] `desktopMain`: tray + OS notifications; `MediaPlayer` actual (mpv/WebView);
      `MediaPicker` actual (file dialog); `SecureStore` actual; window mgmt.
- [ ] Two-pane default layout; keyboard shortcuts; optional inspector pane.
- [ ] Parity pass vs Android feature checklist (`11-desktop-app.md`).
- [ ] jpackage builds (Linux first; macOS/Windows as available).
- **Demo:** desktop app pairs to the same gateway; full feature parity; tray
  notifications; media plays.
- **Accept:** all Android features work on desktop; tray + shortcuts work;
  native binary launches.

## M7 — Polish + E2E + docs
**Goal:** ship-quality.
- [ ] Telegram-style layout pass (per reference image): header, bubbles, date
      separators, ✓✓, model/token footer, banner, bottom bar.
- [ ] Theming (dark default, accent), onboarding/pairing UX, empty/loading/
      reconnecting/degraded states with honest copy.
- [ ] Full E2E suite (`13-testing.md` scenarios 1–12) automated where possible.
- [ ] Docs: `docs/protocol/frames.schema.json` finalized; `docs/setup.md`
      (user-facing pairing + FCM/ntfy + remote access); root README.
- [ ] Security hardening checklist (`09-pairing-security.md`) verified.
- **Demo:** end-to-end on phone + desktop simultaneously; cron into a channel;
  push; media; search.
- **Accept:** all feature-checklist items pass on-device; E2E green; docs
  complete; `hermes-agent/` still never committed.

---

## Sequencing notes
- **M1/M2 depend on the `ws_probe.py` harness** to lock frame shapes early —
  build it in M1.
- **M3 (cron) and M5 (push) both touch the outbox** — build the outbox in M3,
  extend for push in M5.
- **M6 (desktop) reuses M1–M5 shared code** — do it after the Android features
  are stable so the shared module is settled.
- Parallelizable: plugin (Python) and app (Kotlin) can be worked on
  concurrently once the protocol (`04-wire-protocol.md`) is agreed; the probe
  harness is the integration seam.