159 lines
8.9 KiB
Markdown
159 lines
8.9 KiB
Markdown
# 14 — Milestones (M0–M7)
|
||
|
||
Phased delivery. Each milestone ends with a **demo** (on-device where noted) and
|
||
has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||
|
||
---
|
||
|
||
## M0 — Toolchain & scaffolding
|
||
**Goal:** everything builds; the plugin is discoverable; the repo is safe.
|
||
- [X] Install JDK 17, Android SDK, set `ANDROID_HOME` (`12-toolchain.md`).
|
||
- [X] `cd hermes-agent && uv sync` (hermes venv works).
|
||
- [X] Create monorepo scaffold (`02-monorepo.md`): `gateway-plugin/`, `app/`
|
||
(CMP: `shared`, `androidApp`, `desktopApp`), root `.gitignore`
|
||
(**excludes `hermes-agent/`**), root `README.md`.
|
||
- [X] `git init` + a pre-commit/CI guard that fails if `hermes-agent/` is staged.
|
||
- [X] CMP project builds empty: `./gradlew :androidApp:assembleDebug`,
|
||
`./gradlew :desktopApp:run` (blank window).
|
||
- [X] Plugin skeleton: `plugin.yaml` + `adapter.py` with `register(ctx)` + a
|
||
no-op `AndroidAdapter` → `hermes gateway status` lists **android**.
|
||
- **Demo:** `hermes gateway status` shows `android`; `./gradlew
|
||
:androidApp:installDebug` installs a blank app on the MIX 2S.
|
||
- **Accept:** blank app installs + launches on-device; plugin visible in
|
||
`hermes gateway status`; `hermes-agent/` is git-ignored (verify with
|
||
`git status --ignored`).
|
||
|
||
## M1 — Gateway core loop (text round-trip)
|
||
**Goal:** pair + send a text message + get a (non-streaming) reply.
|
||
- [X] WS server (`ws_server.py`): bind, `hello` auth (constant-time),
|
||
`hello.ack`, heartbeat, connection registry.
|
||
- [X] `AndroidAdapter.send()` → `message` frame; inbound `message.send` →
|
||
`MessageEvent` → `handle_message`.
|
||
- [X] Pairing store + `ANDROID_TOKEN`; QR payload in `interactive_setup`.
|
||
- [X] App: Connect screen (URL+token, real `hello` test), `GatewayClient`
|
||
(connect + reconnect), ChatScreen sends + renders `message`.
|
||
- [X] `ws_probe.py` harness drives a real turn.
|
||
- **Demo (on-device):** pair the phone, send "hello", see the agent's reply.
|
||
- **Accept:** text round-trip works on-device; wrong token is rejected;
|
||
reconnect after gateway restart re-pairs.
|
||
|
||
## M2 — Streaming + reasoning + tools + commentary
|
||
**Goal:** the "agent transparency" features.
|
||
- [X] Map consumer `send`/`edit_message` → `message.start/update/stop`.
|
||
- [X] Reasoning: set `show_reasoning` for android; adapter splits prefix →
|
||
`reasoning` field. **Verify format with `ws_probe.py`.** (The model
|
||
returns a separate `reasoning_content` field. In the *streaming* case the
|
||
gateway drops it — the stream consumer only forwards `content` and the
|
||
final send is suppressed — so the adapter captures it via the
|
||
`on_stream_delta` plugin hook (`kind="reasoning"`, gated by
|
||
`plugins.stream_reasoning_deltas: true`) and attaches it to
|
||
`message.stop`. Non-streaming already carried it via the prefix split.)
|
||
- [X] Tool events: classify tool-progress `send()`s → structured
|
||
`tool.start/progress/end` (turn-state machine). **Verify with probe.**
|
||
- [X] Commentary → `commentary` frames. Typing → `typing`.
|
||
- [X] App: live bubble (coalesced updates), `ReasoningBlock` (collapse + copy),
|
||
`ToolCard` with **Everything/Truncated/Nothing** setting, dimmed
|
||
`commentary` bubble.
|
||
- **Demo (on-device):** a multi-step prompt streams, shows reasoning above the
|
||
answer, tool cards (toggle verbosity), and an intermediate beat.
|
||
(Verified on the MIX 2S: streaming, tool cards + verbosity toggle, and the
|
||
reasoning panel rendering above the answer. The commentary beat is
|
||
model/agent-dependent and did not fire with the current model, but the
|
||
rendering + frame handling are in place.)
|
||
- **Accept:** all four render correctly; tool verbosity setting changes
|
||
rendering; reasoning copy button works; frame shapes match `04-wire-protocol`.
|
||
|
||
## M3 — Channels/threads + cron + search
|
||
**Goal:** organization + cron delegation + search.
|
||
- [X] Channel directory (SQLite): default channel ensured; `channel.create/
|
||
rename/set_default/delete` + `channel.*` frames.
|
||
- [X] Threads: toggle in default chat; `thread_id` lanes; `create_handoff_thread`.
|
||
- [X] `parse_target_ref_fn` + `cron_deliver_env_var` → cron
|
||
`deliver=android:<chat>[:<thread>]` works.
|
||
- [X] `search.py` FTS5 bridge; `search` frame (all / this-chat) → results.
|
||
- [X] App: channel list (drawer/rail), thread toggle + topic switcher, "new
|
||
channel" + "set as cron target", SearchScreen with scope toggle + jump.
|
||
- **Demo (on-device):** create "Cron Reports"; create a cron job delivering to
|
||
it; it fires into that channel; search finds a message (both scopes).
|
||
- **Accept:** cron output lands in the chosen channel (not default); threads
|
||
isolate context; search scoping correct; channel list reconciles on events.
|
||
- **Status (complete):** channel directory + threads + search + outbox sync
|
||
verified end-to-end via `ws_probe.py` (create/rename/set_default/delete,
|
||
thread lanes, FTS5 search, sync); cron `deliver=android:<chat>[:<thread>]`
|
||
target resolution verified via `resolve_send_target`. App on-device: channel
|
||
drawer, thread toggle + topic switcher, new channel, search overlay with
|
||
jump. Minor UI gaps deferred to M7 polish: "set as cron target" is set via
|
||
the cron job's `deliver` field (no dedicated button); the search overlay
|
||
defaults to scope "all" (the all/this-chat toggle is supported in the
|
||
controller but not yet exposed). An actual cron job firing into a channel was
|
||
not e2e-tested (it shares the verified resolution path).
|
||
|
||
## M4 — Media
|
||
**Goal:** attach + receive + play media.
|
||
- [ ] Inbound: `media.upload` chunked → `cache_*_from_bytes` → `media_urls`;
|
||
size limit + sha256 + MIME re-sniff.
|
||
- [ ] Outbound: `send_*` → `media.offer`; `media.pull` chunked; delivery-path
|
||
security.
|
||
- [ ] App: SAF pickers + preview chips + upload; `media.pull` → cache;
|
||
**ExoPlayer** inline (audio mini-player, video fullscreen/PiP); image/doc
|
||
viewers.
|
||
- **Demo (on-device):** attach a photo + video (agent sees them); ask agent to
|
||
send an image/video → plays live in-app.
|
||
- **Accept:** both directions work; over-limit rejected; playback is live;
|
||
only allowed files are servable.
|
||
|
||
## M5 — Push + offline (FCM + ntfy)
|
||
**Goal:** reach the phone when backgrounded; catch up on reconnect.
|
||
- [ ] Outbox (SQLite) + sync cursor; `sync`/`sync.done`; retention prune.
|
||
- [ ] `push.py`: `FcmBackend` (HTTP v1 + service account, httpx) +
|
||
`NtfyBackend`; selected by `ANDROID_PUSH_BACKEND`.
|
||
- [ ] Fire push on no-live-subscriber; data payload for silent sync.
|
||
- [ ] App: FCM service (foreground banner + background foreground-service sync),
|
||
`onNewToken` → `fcm.register`; per-chat notification channels; deep-link.
|
||
ntfy listener fallback.
|
||
- [ ] In-app `notification` banners (channel_renamed, approval, cron, …).
|
||
- **Demo (on-device):** background the app → trigger a message → notification
|
||
appears → tap → syncs + opens the chat. Repeat with ntfy backend.
|
||
- **Accept:** push arrives when backgrounded (both backends); reconnect syncs
|
||
with no loss/dup; banners show for foreground events.
|
||
|
||
## M6 — Desktop app
|
||
**Goal:** the same app on a big screen.
|
||
- [ ] `desktopMain`: tray + OS notifications; `MediaPlayer` actual (mpv/WebView);
|
||
`MediaPicker` actual (file dialog); `SecureStore` actual; window mgmt.
|
||
- [ ] Two-pane default layout; keyboard shortcuts; optional inspector pane.
|
||
- [ ] Parity pass vs Android feature checklist (`11-desktop-app.md`).
|
||
- [ ] jpackage builds (Linux first; macOS/Windows as available).
|
||
- **Demo:** desktop app pairs to the same gateway; full feature parity; tray
|
||
notifications; media plays.
|
||
- **Accept:** all Android features work on desktop; tray + shortcuts work;
|
||
native binary launches.
|
||
|
||
## M7 — Polish + E2E + docs
|
||
**Goal:** ship-quality.
|
||
- [ ] Telegram-style layout pass (per reference image): header, bubbles, date
|
||
separators, ✓✓, model/token footer, banner, bottom bar.
|
||
- [ ] Theming (dark default, accent), onboarding/pairing UX, empty/loading/
|
||
reconnecting/degraded states with honest copy.
|
||
- [ ] Full E2E suite (`13-testing.md` scenarios 1–12) automated where possible.
|
||
- [ ] Docs: `docs/protocol/frames.schema.json` finalized; `docs/setup.md`
|
||
(user-facing pairing + FCM/ntfy + remote access); root README.
|
||
- [ ] Security hardening checklist (`09-pairing-security.md`) verified.
|
||
- **Demo:** end-to-end on phone + desktop simultaneously; cron into a channel;
|
||
push; media; search.
|
||
- **Accept:** all feature-checklist items pass on-device; E2E green; docs
|
||
complete; `hermes-agent/` still never committed.
|
||
|
||
---
|
||
|
||
## Sequencing notes
|
||
- **M1/M2 depend on the `ws_probe.py` harness** to lock frame shapes early —
|
||
build it in M1.
|
||
- **M3 (cron) and M5 (push) both touch the outbox** — build the outbox in M3,
|
||
extend for push in M5.
|
||
- **M6 (desktop) reuses M1–M5 shared code** — do it after the Android features
|
||
are stable so the shared module is settled.
|
||
- Parallelizable: plugin (Python) and app (Kotlin) can be worked on
|
||
concurrently once the protocol (`04-wire-protocol.md`) is agreed; the probe
|
||
harness is the integration seam.
|