M0: toolchain, monorepo scaffold, gateway plugin skeleton, CMP app
- gateway-plugin/: android platform plugin (plugin.yaml + adapter.py register(ctx) + no-op AndroidAdapter) + stub modules for M1-M5 - app/: Compose Multiplatform project (shared KMP + androidApp + desktopApp) with Gradle wrapper; builds :androidApp:assembleDebug and :desktopApp:compileKotlin - scripts/guard_hermes_agent.sh + pre-commit hook: fail if hermes-agent/ is staged (read-only reference, never committed) - .gitignore excludes hermes-agent/; docs/ reference library
This commit is contained in:
ARIA committed
commit
59acf66c89
49 files changed
+3950
No files matched your search
@@ -0,0 +1,131 @@
|
||||
# 13 — Testing & Debugging (incl. ADB on-device)
|
||||
|
||||
Three layers: **Python plugin tests**, **Kotlin unit/UI tests**, and **on-device
|
||||
E2E via ADB**. Plus a **WS test-client harness** to drive the real gateway
|
||||
without the app (critical for verifying frame shapes early).
|
||||
|
||||
## 13.1 Python plugin tests
|
||||
|
||||
- Location: `gateway-plugin/tests/` (and, for hermes-integration tests, mirror
|
||||
into the hermes `tests/gateway/test_android.py` pattern when running under
|
||||
hermes's suite).
|
||||
- **Run with hermes's hermetic runner** (never bare `pytest`):
|
||||
```bash
|
||||
cd hermes-agent
|
||||
scripts/run_tests.sh tests/gateway/test_android.py
|
||||
scripts/run_tests.sh # full suite (CI parity)
|
||||
```
|
||||
- Coverage to write (behavioral, not change-detector — per hermes test policy):
|
||||
- `register(ctx)` produces a valid `PlatformEntry` (name, cron env var,
|
||||
parse_target_ref).
|
||||
- `check_requirements` / `validate_config` / `is_connected` truth table.
|
||||
- `_parse_target_ref`: `android:<chat>`, `android:<chat>:<thread>`, non-android
|
||||
→ None.
|
||||
- **Reasoning split:** given a `show_reasoning`-style final text, `send()`
|
||||
emits `message {reasoning, text}` correctly; no-prefix → no reasoning field.
|
||||
- **Outbox:** frame with no subscriber → written with a cursor; `sync` replays
|
||||
the right range; retention prunes.
|
||||
- **Pairing:** valid token → `hello.ack`; wrong token → `error auth` + close.
|
||||
Constant-time compare used.
|
||||
- **Media:** upload start→binary→end reassembles + sha256 verified; over-limit
|
||||
→ `media_too_large`; pull serves only allowed paths.
|
||||
- **Push backend selection:** fcm vs ntfy chosen by config; `configured()`
|
||||
reflects missing creds.
|
||||
- **Channel directory:** create/rename/set_default; cron target resolution.
|
||||
- **No `~/.hermes` writes in tests** — use the `_isolate_hermes_home` fixture
|
||||
pattern (temp `HERMES_HOME`). Profile tests also mock `Path.home()`.
|
||||
|
||||
## 13.2 WS test-client harness (do this FIRST, in M1/M2)
|
||||
|
||||
A small Python script (`gateway-plugin/tests/ws_probe.py`) that connects to the
|
||||
**real running gateway** and drives a turn, printing every frame. This is how we
|
||||
**empirically confirm** the exact frame shapes (especially tool-progress vs
|
||||
commentary classification and the reasoning prefix) before/while building the
|
||||
Kotlin client.
|
||||
|
||||
```bash
|
||||
hermes gateway & # with the android plugin
|
||||
python gateway-plugin/tests/ws_probe.py --token <ANDROID_TOKEN> \
|
||||
--send "list the files and summarize"
|
||||
# prints: hello.ack, typing, message.start, message.update…, tool.start, tool.end,
|
||||
# commentary, message.stop {reasoning,…}, …
|
||||
```
|
||||
Use it to lock `04-wire-protocol.md` against reality and to debug the adapter
|
||||
without waiting for the app.
|
||||
|
||||
## 13.3 Kotlin unit / UI tests
|
||||
|
||||
- **Protocol codec:** round-trip every frame type (serialize → deserialize →
|
||||
equal); unknown `type`/fields ignored (forward-compat).
|
||||
- **Repositories:** merge-don't-clobber on `channel.*`; optimistic send + rollback
|
||||
on `error`; sync dedupe by `message_id`; cursor monotonic.
|
||||
- **`GatewayClient`:** reconnect/backoff; `message.update` coalescing (latest
|
||||
wins); request/response correlation by `id`.
|
||||
- **ViewModels:** state transitions (connecting→connected→reconnecting); tool
|
||||
verbosity filtering (everything/truncated/nothing); reasoning present/absent.
|
||||
- **Compose UI tests:** ReasoningBlock collapse/expand + copy; ToolCard
|
||||
spinner→done; composer auto-grow cap; channel list active highlight.
|
||||
- Run: `./gradlew :shared:testDebugUnitTest` (and `:shared:testDesktopTest`).
|
||||
|
||||
## 13.4 On-device E2E via ADB (the MIX 2S, API 29)
|
||||
|
||||
Device: `a5ca2a4b` (Xiaomi MIX 2S). Workflow:
|
||||
|
||||
```bash
|
||||
# install + launch
|
||||
cd app
|
||||
./gradlew :androidApp:installDebug
|
||||
adb shell am start -n dev.iris.app/.MainActivity
|
||||
|
||||
# logs (filter our tags + WS + FCM + ExoPlayer)
|
||||
adb logcat -c
|
||||
adb logcat | grep -Ei "iris|GatewayClient|Firebase|ExoPlayer|MediaCodec"
|
||||
|
||||
# screenshots for visual checks
|
||||
adb exec-out screencap -p > /tmp/shot.png
|
||||
|
||||
# clear app data between pairing attempts
|
||||
adb shell pm clear dev.iris.app
|
||||
|
||||
# push a file to the app's cache (for media tests) / pull logs
|
||||
adb shell run-as dev.iris.app ls files
|
||||
adb logcat -d > /tmp/logcat.txt
|
||||
```
|
||||
|
||||
**E2E scenarios (script where possible):**
|
||||
1. **Pair:** connect screen → enter URL+token → `hello.ack` → main. (Verify auth
|
||||
leg, not just TCP.)
|
||||
2. **Text round-trip:** send "hello" → streamed reply appears (message.start →
|
||||
updates → stop).
|
||||
3. **Reasoning:** ask a reasoning-model question → ReasoningBlock shows above the
|
||||
answer; copy button works.
|
||||
4. **Tools:** trigger a tool (e.g. "list files") → ToolCard shows; toggle
|
||||
verbosity in Settings → rendering changes.
|
||||
5. **Intermediate:** a multi-step prompt → commentary bubble appears dimmed.
|
||||
6. **Channels:** create "Cron Reports" → appears in list; set as cron target.
|
||||
7. **Cron delivery:** create a cron job `deliver=android:android:chan_<n>` → it
|
||||
fires → lands in that channel (not default).
|
||||
8. **Search:** "search everywhere" vs "this chat" → correct scoping; tap → jump.
|
||||
9. **Media (in):** attach a photo + a video → agent receives (vision) → reply.
|
||||
10. **Media (out):** ask the agent to send an image/video → `media.offer` →
|
||||
inline player plays it live.
|
||||
11. **Push:** background the app (`adb shell am start` another app / lock) →
|
||||
trigger a message → FCM notification appears → tap → syncs + deep-links.
|
||||
12. **Reconnect/sync:** kill the WS (stop gateway briefly) → restart → app
|
||||
reconnects → `sync` catches up (no lost/dup messages).
|
||||
|
||||
## 13.5 Debugging tips
|
||||
|
||||
- **Gateway side:** `~/.hermes/logs/gateway.log` (and `hermes logs --follow`).
|
||||
Our plugin logs under the `android` adapter name; secrets redacted.
|
||||
- **WS framing bugs:** use the `ws_probe.py` harness — it isolates the protocol
|
||||
from the app.
|
||||
- **Streaming jitter:** the consumer edits at intervals; if updates look chunky,
|
||||
check `display.platforms.android.streaming` and the consumer's edit interval.
|
||||
- **Media pull stalls:** check chunk size + backpressure; confirm the file is
|
||||
within hermes delivery roots (`validate_media_delivery_path`).
|
||||
- **FCM not arriving:** confirm the token registered (`devices.db`), the service
|
||||
account can mint a token, and the app's `onNewToken` re-registered after
|
||||
`pm clear`.
|
||||
- **Profile leaks:** if tokens look wrong under multiple profiles, verify the
|
||||
scope-aware secret read (`_get_scoped_secret`) is used.
|
||||
Reference in new issue
Block a user