iris_x_hermes/docs/14-milestones.md

14 — Milestones (M0–M7)

Phased delivery. Each milestone ends with a demo (on-device where noted) and has explicit acceptance criteria. Work top-to-bottom; don't skip M0/M1.

---

M0 — Toolchain & scaffolding

Goal: everything builds; the plugin is discoverable; the repo is safe.

• Install JDK 17, Android SDK, set ANDROID_HOME (12-toolchain.md).
• cd hermes-agent && uv sync (hermes venv works).
• Create monorepo scaffold (02-monorepo.md): gateway-plugin/, app/ (CMP: shared, androidApp, desktopApp), root .gitignore (excludes hermes-agent/), root README.md.
• git init + a pre-commit/CI guard that fails if hermes-agent/ is staged.
• CMP project builds empty: ./gradlew :androidApp:assembleDebug, ./gradlew :desktopApp:run (blank window).
• Plugin skeleton: plugin.yaml + adapter.py with register(ctx) + a no-op AndroidAdapter → hermes gateway status lists android.
• Demo: hermes gateway status shows android; ./gradlew :androidApp:installDebug installs a blank app on the MIX 2S.
• Accept: blank app installs + launches on-device; plugin visible in hermes gateway status; hermes-agent/ is git-ignored (verify with git status --ignored).

M1 — Gateway core loop (text round-trip)

Goal: pair + send a text message + get a (non-streaming) reply.

• WS server (ws_server.py): bind, hello auth (constant-time), hello.ack, heartbeat, connection registry.
• AndroidAdapter.send() → message frame; inbound message.send → MessageEvent → handle_message.
• Pairing store + ANDROID_TOKEN; QR payload in interactive_setup.
• App: Connect screen (URL+token, real hello test), GatewayClient (connect + reconnect), ChatScreen sends + renders message.
• ws_probe.py harness drives a real turn.
• Demo (on-device): pair the phone, send "hello", see the agent's reply.
• Accept: text round-trip works on-device; wrong token is rejected; reconnect after gateway restart re-pairs.

M2 — Streaming + reasoning + tools + commentary

Goal: the "agent transparency" features.

• Map consumer send/edit_message → message.start/update/stop.
• Reasoning: set show_reasoning for android; adapter splits prefix → reasoning field. Verify format with ws_probe.py. (The model returns a separate reasoning_content field. In the streaming case the gateway drops it — the stream consumer only forwards content and the final send is suppressed — so the adapter captures it via the on_stream_delta plugin hook (kind="reasoning", gated by plugins.stream_reasoning_deltas: true) and attaches it to message.stop. Non-streaming already carried it via the prefix split.)
• Tool events: classify tool-progress send()s → structured tool.start/progress/end (turn-state machine). Verify with probe.
• Commentary → commentary frames. Typing → typing.
• App: live bubble (coalesced updates), ReasoningBlock (collapse + copy), ToolCard with Everything/Truncated/Nothing setting, dimmed commentary bubble.
• Demo (on-device): a multi-step prompt streams, shows reasoning above the answer, tool cards (toggle verbosity), and an intermediate beat. (Verified on the MIX 2S: streaming, tool cards + verbosity toggle, and the reasoning panel rendering above the answer. The commentary beat is model/agent-dependent and did not fire with the current model, but the rendering + frame handling are in place.)
• Accept: all four render correctly; tool verbosity setting changes rendering; reasoning copy button works; frame shapes match 04-wire-protocol.

M3 — Channels/threads + cron + search

Goal: organization + cron delegation + search.

• Channel directory (SQLite): default channel ensured; channel.create/ rename/set_default/delete + channel.* frames.
• Threads: toggle in default chat; thread_id lanes; create_handoff_thread.
• parse_target_ref_fn + cron_deliver_env_var → cron deliver=android:<chat>[:<thread>] works.
• search.py FTS5 bridge; search frame (all / this-chat) → results.
• App: channel list (drawer/rail), thread toggle + topic switcher, "new channel" + "set as cron target", SearchScreen with scope toggle + jump.
• Demo (on-device): create "Cron Reports"; create a cron job delivering to it; it fires into that channel; search finds a message (both scopes).
• Accept: cron output lands in the chosen channel (not default); threads isolate context; search scoping correct; channel list reconciles on events.
• Status (complete): channel directory + threads + search + outbox sync verified end-to-end via ws_probe.py (create/rename/set_default/delete, thread lanes, FTS5 search, sync); cron deliver=android:<chat>[:<thread>] target resolution verified via resolve_send_target. App on-device: channel drawer, thread toggle + topic switcher, new channel, search overlay with jump. Minor UI gaps deferred to M7 polish: "set as cron target" is set via the cron job's deliver field (no dedicated button); the search overlay defaults to scope "all" (the all/this-chat toggle is supported in the controller but not yet exposed). An actual cron job firing into a channel was not e2e-tested (it shares the verified resolution path).

M4 — Media

Goal: attach + receive + play media.

• Inbound: media.upload chunked → cache_*_from_bytes → media_urls; size limit + sha256 + MIME re-sniff.
• Outbound: send_* → media.offer; media.pull chunked; delivery-path security.
• App: SAF pickers + preview chips + upload; media.pull → cache; ExoPlayer inline (audio mini-player, video fullscreen/PiP); image/doc viewers.
• Demo (on-device): attach a photo + video (agent sees them); ask agent to send an image/video → plays live in-app.
• Accept: both directions work; over-limit rejected; playback is live; only allowed files are servable.
• Status (complete): Both directions verified end-to-end on the MIX 2S. Inbound: SAF-picked photo → chunked binary media.upload (256 KiB) → gateway size+sha256 verify + MIME re-sniff → cache_image_from_bytes → media.upload.ack → message.send with media_refs → agent vision described the image accurately. Outbound: agent send_image → media.offer → app auto-media.pull (chunked) → cache → image rendered inline. Live playback verified on-device: agent offered a 2s MP4 + 2s MP3 → app pulled both → ExoPlayer video player (blue frame, 00:02/00:02, controls) + audio mini-player rendered and playable. Over-limit rejection, sha256 mismatch, and delivery-path security are covered by the 17-test tests/gateway/test_android.py suite (all pass). Note: the app serializes pulls (single binarySession slot) via a Mutex so concurrent offers don't interleave their byte streams. media.upload.ack documented in 04-wire-protocol.md + frames.schema.json.

M5 — Push + offline (FCM + ntfy)

Goal: reach the phone when backgrounded; catch up on reconnect.

• Outbox (SQLite) + sync cursor; sync/sync.done; retention prune (row cap 5000 + prune banner, throttled 1/h).
• push.py: FcmBackend (HTTP v1 + service account, httpx; JWT via PyJWT+cryptography) + NtfyBackend (X-Data header); selected by ANDROID_PUSH_BACKEND. ntfy server exposed in server_caps.push_ntfy_server.
• Fire push on no-live-subscriber; data payload for silent sync. High-priority kinds (approval/clarify/cron) push even when live.
• App: FCM service (onNewToken → fcm.register; inert without a Firebase project), per-chat notification channels, deep-link (custom action + iris://), ntfy listener foreground-service fallback. App generates a device-specific ntfy topic; auto-sync on reconnect.
• In-app notification banners (channel events, approval, clarify, cron, outbox-pruned); system notification mirror when backgrounded.
• Demo (on-device, verified 2026-08-19): device offline → message broadcast → frame parked in outbox → push via ntfy -> <device> fired → app reconnect auto-synced (cursor 0→8) → parked message rendered in-app. ntfy publish confirmed (POST 200). ntfy listener service runs (foreground notification posted); listener→system-notification leg not exercised live because ntfy.sh's public SSE endpoint was serving its web UI (not streams) during the test — verify against a self-hosted ntfy.
• Accept: push arrives when backgrounded (ntfy verified; FCM code path complete, needs a Firebase project to exercise); reconnect syncs with no loss/dup (verified); banners show for foreground events (implemented).

M6 — Desktop app

Goal: the same app on a big screen.

• desktopMain: tray + OS notifications; MediaPlayer actual (mpv/WebView); MediaPicker actual (file dialog); SecureStore actual; window mgmt.
• Two-pane default layout; keyboard shortcuts; optional inspector pane.
• Parity pass vs Android feature checklist (11-desktop-app.md).
• jpackage builds (Linux first; macOS/Windows as available).
• Demo: desktop app pairs to the same gateway; full feature parity; tray notifications; media plays.
• Accept: all Android features work on desktop; tray + shortcuts work; native binary launches.

M7 — Polish + E2E + docs

Goal: ship-quality.

• Telegram-style layout pass (per reference image): header, bubbles, date separators, ✓✓, model/token footer, banner, bottom bar.
• Theming (dark default, accent), onboarding/pairing UX, empty/loading/ reconnecting/degraded states with honest copy.
• Full E2E suite (13-testing.md scenarios 1–12) automated where possible.
• Docs: docs/protocol/frames.schema.json finalized; docs/setup.md (user-facing pairing + FCM/ntfy + remote access); root README.
• Security hardening checklist (09-pairing-security.md) verified.
• Demo: end-to-end on phone + desktop simultaneously; cron into a channel; push; media; search.
• Accept: all feature-checklist items pass on-device; E2E green; docs complete; hermes-agent/ still never committed.

---

Sequencing notes

• M1/M2 depend on the ws_probe.py harness to lock frame shapes early — build it in M1.
• M3 (cron) and M5 (push) both touch the outbox — build the outbox in M3, extend for push in M5.
• M6 (desktop) reuses M1–M5 shared code — do it after the Android features are stable so the shared module is settled.
• Parallelizable: plugin (Python) and app (Kotlin) can be worked on concurrently once the protocol (04-wire-protocol.md) is agreed; the probe harness is the integration seam.