ARIA
a61b47a947
The plugin is named 'iris' (IrisAdapter, IRIS_HOME_CHANNEL, label Iris), but several docs still referred to it as the android platform/plugin and to the product as 'the Android app'. Rename name-mentions to iris/IRIS and product-mentions to 'Iris app'; keep legitimate OS references (androidApp, Android SDK, Android 10, androidx, test_android.py, ...). Also includes pi-lens markdown-lint autofixes (table spacing, trailing newlines) in the touched files.
323 lines
16 KiB
Markdown
323 lines
16 KiB
Markdown
# 20 — QR Pairing (terminal QR + in-app scanner)
|
||
|
||
**Status: implemented (M8, 2026-08-22).**
|
||
|
||
Closes gap #12 in `09-pairing-security.md` ("in-app QR scanner") and implements
|
||
the QR branch of the §9.2 pairing flow, which the docs already promise but the
|
||
code never delivered: today `interactive_setup` prints the pairing URL as
|
||
plain text only, and the app has no `iris://pair` parser at all.
|
||
|
||
Two halves, independent and shippable separately:
|
||
|
||
- **A — Gateway:** `hermes gateway setup` renders a scannable QR in the
|
||
terminal encoding `iris://pair?host=…&port=…&token=…`. **Zero new Python
|
||
dependencies** (pure stdlib encoder).
|
||
- **B — App:** a "Scan QR" button on the Android Connect screen (CameraX +
|
||
ML Kit, on-device, no Play services) that pre-fills URL + token. Not added
|
||
to desktop. Plus an `iris://pair` deep link so *any* scanner (system camera
|
||
app, other phones) can route the QR into the app.
|
||
|
||
---
|
||
|
||
## 20.1 Current state (what exists today)
|
||
|
||
| Piece | State | Location |
|
||
| ------- | ------- | ---------- |
|
||
| QR payload format | ✅ implemented | `gateway-plugin/pairing.py` → `qr_payload(host, port, token, secure)` → `iris://pair?host=<lan-ip>&port=8791&secure=0&token=<64-hex>` |
|
||
| Terminal QR rendering | ❌ missing | `gateway-plugin/adapter.py` → `interactive_setup()` prints the URL text only |
|
||
| App `iris://pair` parser | ❌ missing | app has manual URL + token entry only (`ConnectScreen`) |
|
||
| In-app camera scan | ❌ missing | no camera deps anywhere in `app/` |
|
||
| `iris://` deep link | ⚠️ partial | manifest handles `iris://chat/<id>` only (`androidApp/.../AndroidManifest.xml`, `MainActivity.handleDeepLink`) |
|
||
| QR libs in hermes venv | ❌ absent | `qrcode`/`segno` not installed; `Pillow` is a hermes core dep but only renders images — the QR *matrix* algorithm is still needed either way |
|
||
|
||
Payload size: `iris://pair?host=192.168.x.x&port=8791&secure=0&token=<64 hex>`
|
||
≈ **118 bytes** → QR version **7 at EC level M** (capacity 122 bytes) or v6 at
|
||
L (134). The encoder must therefore support at least versions 1–8; we target
|
||
1–10.
|
||
|
||
---
|
||
|
||
## 20.2 Part A — terminal QR in `interactive_setup`
|
||
|
||
### A1. Pure-stdlib QR encoder — `gateway-plugin/qr.py` (new file)
|
||
|
||
A self-contained ISO/IEC 18004 encoder, **stdlib only** (no `qrcode`, no
|
||
`segno`, no Pillow). Scope is deliberately minimal — we only ever encode
|
||
ASCII pairing URLs:
|
||
|
||
- **Mode:** byte mode only (no alphanumeric/numeric/kanji paths).
|
||
- **Error correction:** level **M** (15 %); auto-fallback to **L** if the
|
||
payload doesn't fit at M within the version cap.
|
||
- **Versions:** 1–10, auto-selected (smallest version whose capacity fits).
|
||
Payloads that don't fit v10-L raise `QrTooLongError` (caller falls back to
|
||
text-only output — see A3).
|
||
- **Components** (all well-known, spec-stable algorithms):
|
||
1. Data encoding: mode indicator `0100`, 8-bit char count (8 bits for
|
||
v1–9, 16 bits for v10), payload bytes, terminator, padding
|
||
(`0xEC`/`0x11` alternation).
|
||
2. Reed–Solomon error correction over GF(256), generator polynomial
|
||
`0x11D`, per (version, EC level) block structure from the spec tables.
|
||
3. Matrix placement: finder patterns + separators, timing patterns,
|
||
alignment patterns (v2+), dark module, format info (BCH(15,5)),
|
||
version info (v7+, BCH(18,6)), zig-zag data placement.
|
||
4. Masking: all 8 masks, ISO penalty scoring (N1–N4), pick lowest.
|
||
- **Public API:**
|
||
|
||
```python
|
||
def qr_matrix(data: str) -> list[list[bool]]:
|
||
"""Encode *data* (ASCII) into a module matrix (True = dark).
|
||
Includes the 4-module quiet zone. Raises QrTooLongError."""
|
||
```
|
||
|
||
~250–350 lines including the spec tables. No I/O, no globals, fully
|
||
unit-testable.
|
||
|
||
### A2. Terminal renderer — `qr.py`
|
||
|
||
```python
|
||
def render_qr(data: str) -> str:
|
||
"""Render *data* as a terminal QR using Unicode half-blocks (▀).
|
||
Returns '' (not an exception) when the payload is too long."""
|
||
```
|
||
|
||
- Pair consecutive module rows into one character row: both dark → `█`,
|
||
top dark → `▀`, bottom dark → `▄`, both light → space. (Matrix height
|
||
including quiet zone is always even: `2·(17+4v)+8`.)
|
||
- Output is a single string of `\n`-joined lines; the caller prints it.
|
||
- No ANSI colors, no cursor tricks — must survive `less`, log files, and
|
||
copy-paste.
|
||
|
||
### A3. Integration — `adapter.py:interactive_setup()`
|
||
|
||
After the existing "Pairing URL / Server URL" lines:
|
||
|
||
```python
|
||
qr = render_qr(qr_payload(host, port, token))
|
||
if qr:
|
||
print_info("Scan with the Iris app (Connect → Scan QR) or any camera app:")
|
||
print(qr)
|
||
else:
|
||
print_warning("QR too large to render; use the pairing URL above.")
|
||
```
|
||
|
||
- The **URL text lines stay** — the QR is a convenience, not a replacement
|
||
(terminals without UTF-8 still work, and the text is copy-pasteable).
|
||
- Printed on every setup run (new *and* existing token), consistent with the
|
||
URL lines which already print the token in cleartext.
|
||
- **Security note:** no new exposure — the token is already printed in the
|
||
pairing URL line today; the QR is the same bytes in a different encoding,
|
||
on the same operator-only stdout. (Gap #5 in the §9.7 table already
|
||
documents the stdout token print.)
|
||
|
||
### A4. Tests — `hermes-agent/tests/gateway/test_android.py`
|
||
|
||
The test file is a thin mirror importing the **live `gateway-plugin/`
|
||
package**, so new tests land there:
|
||
|
||
1. **Fixed test vectors** (guard against silent algorithm drift): at least
|
||
two known-good (data → matrix) pairs from public QR test vectors
|
||
(e.g. the ISO 18004 annex examples / the classic `KARAT` v2-L vector).
|
||
Assert the full matrix, not just dimensions.
|
||
2. **Round-trip via payload:** `qr_matrix(qr_payload(h, p, t))` has the
|
||
expected version/size for a 64-hex token (`17 + 4·7 = 45` modules at
|
||
v7-M, +8 quiet zone).
|
||
3. **Renderer shape:** every line equal length, height = half of matrix
|
||
height, quiet zone renders as blank border, only the 4 block chars +
|
||
space appear.
|
||
4. **`QrTooLongError` / `render_qr` → `""`** for a payload beyond v10-L.
|
||
5. **`interactive_setup` smoke:** with sandboxed HERMES_HOME (conftest
|
||
already does this), capture stdout and assert the QR block appears after
|
||
the pairing URL line.
|
||
|
||
Run: `cd hermes-agent && scripts/run_tests.sh tests/gateway/test_android.py`.
|
||
|
||
---
|
||
|
||
## 20.3 Part B — in-app scanner (Android only)
|
||
|
||
### B1. Dependencies — `app/shared/build.gradle.kts`, `androidMain.dependencies` only
|
||
|
||
| Dependency | Why |
|
||
| ------------ | ----- |
|
||
| `androidx.camera:camera-camera2` | Camera access (CameraX) |
|
||
| `androidx.camera:camera-lifecycle` | Lifecycle-aware binding |
|
||
| `androidx.camera:camera-view` | `PreviewView` for the scan surface |
|
||
| `com.google.mlkit:barcode-scanning` | On-device QR decode; **no Google Play services required** (self-contained model) |
|
||
|
||
- Chosen over `zxing-android-embedded` (decided 2026-07-20): ML Kit has
|
||
better accuracy/latency, is maintained by Google, and works fully
|
||
on-device without GMS.
|
||
- These go in **`androidMain`** only — the no-new-dep rule covers
|
||
`gateway-plugin/`, and the app already carries OkHttp/SQLDelight/KCEF/etc.
|
||
Desktop is untouched.
|
||
- `minSdk 29` is fine for all four (ML Kit barcode needs 21+).
|
||
- Versions go in the existing version catalog / `composeVersion`-style
|
||
constants at the top of the build file (follow the current pattern).
|
||
|
||
### B2. Scanner activity — `shared/src/androidMain/kotlin/iris/platform/QrScanActivity.kt` (new)
|
||
|
||
A minimal `ComponentActivity` (not a Fragment, no nav graph):
|
||
|
||
- Layout: full-screen `PreviewView` + overlay hint text ("Point at the QR
|
||
code") + close button.
|
||
- `ImageAnalysis` (STRATEGY_LATEST, YUV_420_888) →
|
||
`BarcodeScannerOptions(FORMAT_QR_CODE)` → first result →
|
||
`setResult(RESULT_OK, Intent().putExtra("iris.qr.text", raw))` → finish.
|
||
- **Runtime permission:** request `CAMERA` on launch; on denial show a
|
||
message + close (the Connect screen still has manual entry).
|
||
- Registered in `shared/src/androidMain/AndroidManifest.xml` (or the
|
||
androidApp manifest — follow where `NtfyListenerService` is declared)
|
||
with `android:exported="false"`, `android:theme` reusing the app theme.
|
||
- Manifest additions (androidApp manifest):
|
||
|
||
```xml
|
||
<uses-permission android:name="android.permission.CAMERA" />
|
||
<uses-feature android:name="android.hardware.camera" android:required="false" />
|
||
```
|
||
|
||
`required="false"` so the app stays installable on camera-less devices
|
||
(the button then just reports "no camera").
|
||
|
||
### B3. Platform hook — `expect`/`actual`
|
||
|
||
`shared/src/commonMain/kotlin/iris/platform/PlatformQr.kt` (new):
|
||
|
||
```kotlin
|
||
/** Launch the QR scanner. [onResult] gets the decoded text, or null when
|
||
* the user cancelled / no camera / permission denied. Desktop: no-op. */
|
||
expect fun scanQrCode(onResult: (String?) -> Unit)
|
||
```
|
||
|
||
- **androidMain actual:** `ActivityResultLauncher` (from the Compose
|
||
`LocalContext`) starting `QrScanActivity`; maps `RESULT_OK` → text,
|
||
everything else → `null`.
|
||
- **desktopMain actual:** `onResult(null)` immediately (the button is
|
||
hidden on desktop anyway — see B4; the no-op keeps the `expect` total).
|
||
|
||
### B4. Connect screen button — `ConnectScreen.kt`
|
||
|
||
- New **"Scan QR"** `Button` below the token field, rendered only when
|
||
`!isDesktop` (`iris.platform.isDesktop` already exists).
|
||
- On tap: `scanQrCode { raw -> … }`; on non-null `raw`:
|
||
- `PairLink.parse(raw)` (B5) → pre-fill `url` and `token` state, clear
|
||
error, and **do not auto-connect** — the user still taps
|
||
"Test & Connect" (pairing stays an explicit act, per §10.8).
|
||
- Parse failure → set `error` to "Not a pairing QR code" (don't echo the
|
||
raw payload — it may contain someone else's token).
|
||
- On `null` (cancel/denied): no-op, no error.
|
||
|
||
### B5. Pair-link parser — `shared/src/commonMain/kotlin/iris/util/PairLink.kt` (new)
|
||
|
||
```kotlin
|
||
data class PairLink(val url: String, val token: String)
|
||
|
||
object PairLink {
|
||
/** Parse `iris://pair?host=…&port=…&secure=…&token=…` → PairLink.
|
||
* Returns null on any malformation. */
|
||
fun parse(raw: String): PairLink?
|
||
}
|
||
```
|
||
|
||
- Accepts exactly scheme `iris`, host `pair` (case-insensitive scheme).
|
||
- Required: `host` (non-empty), `token` (non-empty). `port` defaults to
|
||
`8791` (the HTTP default, `docs/19`); `secure` defaults to `0`.
|
||
- Builds `url` as `http(s)://<host>:<port>`; validates port 1–65535.
|
||
- URL-decodes `host`/`token` (the Python side `quote()`s them).
|
||
- Pure function, no platform imports → **unit-tested in `jvmTest`**
|
||
(`:shared:testAndroidHostTest` / `:shared:desktopTest` both run it):
|
||
valid link, missing token, bad port, wrong scheme, wrong host,
|
||
percent-encoded host, secure=1 → https, default port.
|
||
|
||
### B6. `iris://pair` deep link (system-scanner fallback)
|
||
|
||
So a QR scanned by *any* app (phone's built-in scanner, a friend's phone)
|
||
lands in Iris:
|
||
|
||
- Manifest: extend the existing `VIEW` intent-filter block (or add a
|
||
sibling) with `<data android:scheme="iris" android:host="pair" />`.
|
||
- `MainActivity.handleDeepLink`: on `iris://pair` → `PairLink.parse(uri)` →
|
||
stash into a `mutableStateOf<PairLink?>` passed into `IrisApp` →
|
||
`ConnectScreen` receives it as `prefillUrl`/`prefillToken` (the params
|
||
already exist). If the app is already connected, ignore (or surface in
|
||
Settings later — out of scope).
|
||
- This reuses B5's parser; add one test for the URI shape Android delivers.
|
||
|
||
---
|
||
|
||
## 20.4 Part C — doc updates (with the implementation)
|
||
|
||
| Doc | Change |
|
||
| ----- | -------- |
|
||
| `09-pairing-security.md` | Gap #12 → **implemented** (fix the stale `adapter.py:648-654` reference while at it); §9.2 QR branch no longer aspirational |
|
||
| `10-android-app.md` §10.8 | "or scan QR" becomes real: scanner button + deep link, camera permission |
|
||
| `14-milestones.md` | New **M8 — QR pairing** section (acceptance criteria below) |
|
||
| `16-open-questions.md` | Record decision: ML Kit over zxing; pure-stdlib encoder over vendoring `segno` |
|
||
| `README.md` | Reading-order table: add row 20 |
|
||
|
||
---
|
||
|
||
## 20.5 Work breakdown & sequencing
|
||
|
||
Ordered so each step is independently verifiable; A and B can be
|
||
interleaved (different languages, no shared surface).
|
||
|
||
| # | Task | Verify |
|
||
| --- | ------ | -------- |
|
||
| 1 | `qr.py` encoder + renderer (A1/A2) | new unit tests green (A4.1–4.4) |
|
||
| 2 | `interactive_setup` integration (A3) | A4.5 + manual: `hermes gateway setup` in a real terminal shows a scannable QR (scan with the phone's *system* camera app as the decoder oracle) |
|
||
| 3 | `PairLink` parser + jvmTest (B5) | `./gradlew :shared:testAndroidHostTest` |
|
||
| 4 | Deps + manifest + `QrScanActivity` (B1/B2) | `:androidApp:assembleDebug` |
|
||
| 5 | `PlatformQr` expect/actual + Connect button (B3/B4) | `:androidApp:assembleDebug` + `:desktopApp:run` (button absent, no crash) |
|
||
| 6 | `iris://pair` deep link (B6) | ADB: `adb shell am start -a android.intent.action.VIEW -d "iris://pair?host=…&port=…&token=…"` → Connect screen pre-filled |
|
||
| 7 | Doc updates (Part C) | — |
|
||
|
||
**On-device E2E (final gate, per `13-testing.md` ADB workflow):**
|
||
|
||
1. `hermes gateway setup` on the gateway host → QR in terminal.
|
||
2. Phone: `adb shell am start -n dev.iris.app/.MainActivity` → Connect →
|
||
**Scan QR** → grant camera → point at the terminal (screenshot the QR
|
||
onto a second screen if needed; the reference device is API 29 —
|
||
verify CameraX works on the MIX 2S in step 4 before building the rest).
|
||
3. Fields pre-filled → **Test & Connect** → chat screen.
|
||
4. Repeat via deep link (step 6 command) with a *different* token.
|
||
5. Negative: scan a non-pairing QR (e.g. a website) → "Not a pairing QR
|
||
code", fields untouched.
|
||
|
||
---
|
||
|
||
## 20.6 Acceptance criteria (M8)
|
||
|
||
- [ ] `hermes gateway setup` prints a QR that a stock Android camera app
|
||
decodes to exactly `qr_payload(host, port, token)`.
|
||
- [ ] QR encoder: fixed test vectors + size/round-trip tests green;
|
||
**zero** new entries in the plugin's import surface (stdlib only —
|
||
verifiable by `ruff`/import scan).
|
||
- [ ] Android: Connect screen shows **Scan QR** (hidden on desktop);
|
||
scanning the setup QR pre-fills URL + token; "Test & Connect" pairs.
|
||
- [ ] Camera permission denied → graceful message, manual entry still works.
|
||
- [ ] `iris://pair` deep link pre-fills the Connect screen (ADB-verified).
|
||
- [ ] `PairLink.parse` unit tests cover the matrix in B5.
|
||
- [ ] Full Python suite green: `scripts/run_tests.sh` (no args).
|
||
- [ ] Docs updated per Part C; gap #12 closed.
|
||
|
||
## 20.7 Risks & mitigations
|
||
|
||
| Risk | Mitigation |
|
||
| ------ | ------------ |
|
||
| Hand-rolled QR encoder has a subtle bug | Fixed spec test vectors (A4.1) + the system-camera-app oracle in the E2E gate; scope locked to byte mode / v1–10 so the surface stays small |
|
||
| Terminal without UTF-8 mangles the QR | URL text lines remain the primary path; QR is additive |
|
||
| CameraX quirks on API 29 (MIX 2S) | Build the scanner activity first (task 4) and verify on-device before wiring the UI |
|
||
| ML Kit model size (~4 MB) | Bundled in the APK, on-device, no runtime download — acceptable for this app's footprint |
|
||
| Token in QR scanned by a bystander's phone | Same trust domain as the token already printed in the terminal; LAN pairing is operator-supervised by design (§9.2). Deep link only pre-fills — it never auto-connects |
|
||
| `secure=1` (WSS) URLs | Parser already handles `secure` → `https://`; QR payload unchanged |
|
||
|
||
## 20.8 Explicit non-goals
|
||
|
||
- **QR display in the app** (showing a QR for other devices to scan) —
|
||
single-device pairing today; revisit if multi-device lands.
|
||
- **`hermes iris pair` stretch CLI** (re-issue token + new QR,
|
||
`09-pairing-security.md` §9.2 line 48) — separate backlog item.
|
||
- **WSS cert pinning** (gap #6) — orthogonal; QR carries `secure=1`
|
||
already, pinning is app-side.
|
||
- **iOS scanner** — no iOS target (per `00-overview.md`).
|