iris_x_hermes/docs/10-android-app.md

# 10 — Iris App (Android; Kotlin + Jetpack Compose)

Native client. Lives in the Compose Multiplatform project at `app/`; the bulk of
the code is in `app/shared` (commonMain) so the Desktop app reuses it.

## 10.1 Tech stack

| Concern | Choice |
| --- | --- |
| Language | Kotlin |
| UI | Jetpack Compose (Material 3), Compose Navigation |
| Async | Kotlinx Coroutines + Flow |
| WS client | OkHttp (`WebSocketListener`) |
| JSON | kotlinx-serialization |
| Local DB | **SQLDelight** (KMP; local cache: messages per lane, channel directory, meta — see 10.7/10.9. Sync cursor + settings live in `SecureStore`, media files in `MediaCache`) |
| Media playback | Media3 **ExoPlayer** (audio + video) |
| Push | Firebase Messaging (FCM) [primary] / ntfy listener [fallback] |
| DI | Hilt |
| Images | Coil |
| minSdk / target | **26** / 34 (test device: MIX 2S, API 29) |

## 10.2 Module layout

```
app/shared/src/
├── commonMain/kotlin/iris/
│   ├── protocol/        # frame data classes (mirror of gateway protocol.py)
│   ├── net/             # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
│   ├── data/            # ChannelRepository, MessageRepository, MediaRepository,
│   │                    #   SearchRepository, SettingsRepository (SQLDelight + WS)
│   ├── state/           # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
│   ├── ui/
│   │   ├── theme/       # Material 3 theme (dark default), type, color
│   │   ├── components/  # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
│   │   │                #   ChannelRow, Composer, PickerSheet, SearchBar, Banner
│   │   └── screens/     # ConnectScreen, ChatListScreen, ChatScreen,
│   │                    #   SearchScreen, SettingsScreen, ChannelMenu
│   └── util/            # time formatting, markdown, id gen
├── androidMain/kotlin/iris/
│   ├── platform/        # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
│   │                    #   Notifications, SecureStore (EncryptedSharedPreferences)
│   ├── fcm/             # FirebaseMessagingService, foreground sync service
│   └── IrisApp.kt       # Application (Hilt), notification channels
└── androidApp/          # MainActivity, AndroidManifest, res, google-services
```

## 10.3 `GatewayClient` (net)

- OkHttp `WebSocket` to `ws(s)://host:port/ws`.
- **Reconnect:** exponential backoff + jitter; on reconnect send `hello` then
  `sync {cursor}` (replays undelivered outbox only).
- **Initial channel open:** on first open of a chat/thread, send `history`
  (newest page) to populate the view. `sync` does NOT load history.
- **Heartbeat:** send `ping` every 20s; reap on missed `pong` (3×) → reconnect.
- **Request/response:** map of `id → CompletableDeferred`; events → a
  `SharedFlow<Frame>` consumed by repositories.
- **Backpressure:** collect frames on a bounded channel; coalesce
  `message.update` per `message_id` (keep latest) to avoid flooding the UI.
- **Thread:** all frame emission on a single dispatcher; UI observes via Flow.

## 10.4 State rules (from hermes desktop AGENTS.md — apply here)

- **Server is authoritative** for channels/messages/cursor; the app's Room copy
  is a **cache**. Reconcile (merge, don't clobber) on `channel.*`/`sync`.
- **Optimistic then honest:** send a message → show it immediately (pending),
  roll back visibly on `error`, authoritative `sync` gets the last word.
- **Guard against the past:** generation counters / request tokens so a stale
  response never overwrites newer intent.
- **Isolate the foreground:** only the visible chat publishes into the shared
  view; background chats update their own cache quietly.
- **Coalesce noise, flush signal:** batch cosmetic updates; let terminal
  transitions (turn done, needs input, failed) reach the user immediately.

## 10.5 Feature implementation (your checklist)

### Input box, auto-grow (max height)

- Compose `BasicTextField` inside a `Box` with
  `Modifier.heightIn(min = 1.line, max = 160.dp)`. Grows with lines, caps at
  160dp, then scrolls internally.
- Bottom bar (matches reference): `Menü` button (left), emoji button, the
  auto-grow field, attach (paperclip), mic/send (right). Send on Enter
  (configurable: Enter=send vs Enter=newline).

### Slash commands

- **`/` drawer (implemented):** typing `/` in the composer rolls a drawer up
  over the input listing the command catalog. The catalog is served by the
  gateway via `commands.catalog` request → response with
  `{name, description, args_hint, category, aliases}` per command (derived
  from hermes' `COMMAND_REGISTRY`, gateway-available subset + plugin commands),
  so it always matches hermes (`/new`, `/model`, `/reasoning`, `/status`,
  `/cron`, …). The drawer is shown while the input is a bare `/…` (no space
  yet); typing fuzzy-filters it **client-side** (subsequence match over name +
  aliases — no `commands.complete` round-trip). Tapping a row sends
  `message.send {text:"/cmd"}` and closes the drawer; an unknown command
  empties the match list and closes it (the raw text can still be sent —
  hermes answers with its unknown-command reply). Escape (desktop) clears the
  `/` input.
- **Menu button (planned):** `Menü` opens a **bottom sheet** listing the same
  catalog for discovery without typing.
- **Interactive commands** (`/model`, `/reasoning`, `/fast`, approvals,
  clarifies) render as **native pickers** from `picker.*` frames (a dialog /
  sheet with the options; answer via `picker.select`) — planned.

### Streaming (app-controlled)

- **Settings → "Streaming"** toggle (default on). When off, the app ignores
  `message.start`/`message.update` frames and shows each reply as a single
  final message on `message.stop` (the typing indicator covers the wait).
  Per-device display preference — the gateway keeps streaming for other
  devices (see `05-streaming.md` §5.1).

### Tool output (app-controlled verbosity)

- `ToolCard` renders `tool.start/progress/end` frames.
- The gateway always supplies the **full** tool data: it forces
  `display.platforms.iris.tool_progress: verbose` (so the progress line
  carries the full args JSON → `tool.start.args`) and captures each completed
  call via the `post_tool_call` hook (→ `tool.end` `output_preview` /
  `duration` / `ok`). The app decides how much to show.
- **Settings → "Tool detail": Everything / Truncated / Nothing.**
  - Everything: cards start expanded — name + full args + output.
  - Truncated (default): compact one-liner; tap a card to reveal the full
    args + output (what was actually called).
  - Nothing: suppress tool cards.
- Spinner while running; ✓/✗ + duration on `tool.end`.

### Reasoning before message

- `ReasoningBlock` (collapsible, "💭 Reasoning" header, monospace body, **copy**
  button) rendered **above** the message body from the `reasoning` field.
  Matches the reference screenshot.
- **Settings → "Reasoning"** toggle (default on): auto-collapse — long
  reasoning blocks start collapsed, short ones expanded. Off: all reasoning
  blocks start expanded. Per-device display preference; the per-block
  tap-to-toggle always works.

### Intermediate messages

- `commentary` frames → dimmed/smaller bubble, distinct from final answers.

### Message selection + delete

- **Long-press** a message bubble (touch) or **right-click** it (desktop mouse)
  enters selection mode: the tapped message is selected (a circular check appears
  beside each bubble) and the composer is replaced by a selection toolbar
  (count + **Delete** + cancel ✕).
- While selecting, **tap** a bubble to toggle it; the ✕ (or deselecting the last
  message) exits selection mode. Only finalized messages are selectable — a
  streaming bubble has no final id yet and a pending echo isn't on the server.
- **Delete** → confirm dialog → `message.delete {message_ids:[…]}` for the
  current lane. The server completely deletes the message(s): they are removed
  from the outbox (so `history`/`sync` no longer return them) **and** from the
  hermes session store (so no search trace survives and they are not
  recoverable), and `message.deleted` is broadcast to every device; each device
  drops them from its cache (the requesting device also drops them locally for
  snappy UX). Deleting is idempotent.

### Agent busy / stop / steer

- `agent.busy` → show "thinking…" indicator in chat header (animated dots).
- `agent.idle` → clear indicator.
- **Stop button** (appears in header while busy): sends `agent.stop`.
- **Steer:** while busy, the composer accepts input; sending it calls
  `agent.steer` (injects mid-turn) rather than queuing a new `message.send`.

### Threading + channels

- **Channel list** (drawer on single-pane; left rail on two-pane) = default chat
  - user channels (avatar, name, last-message preview, unread badge, active
  highlight bar) — matches the reference left sidebar.
- **Thread toggle** in the default chat header: "Threads on/off". On → topic
  switcher above the message list (each topic = a `thread_id`).
- **Auto-threading** (Threads on, Telegram topic-mode workflow,
  `06-channels-cron-search.md` §6.3.1): a message sent in a channel's flat
  lane goes out as `message.send {auto_thread:true}`; the gateway mints a
  fresh thread (instant derived name, AI-named a moment later via
  `channel.renamed`) and the app jumps into it, relocating the optimistic
  bubble from the flat lane.
- **New channel** (FAB / channel-list menu) → `channel.create` → appears in list;
  menu offers "Set as cron target".
- **Topic context menu** — long-press a topic chip (touch) or right-click it
  (desktop mouse) → "Rename" / "Delete". Rename → `channel.rename` (prefilled
  dialog); Delete → confirm → `channel.delete` (hard delete; the thread leaves
  the switcher, its history is wiped from the outbox and session store, and if
  it was the open lane the app falls back to the channel's flat lane). The
  right-click handler is a skiko `expect`/`actual`
  (`iris/ui/ContextMenu.kt`); on touch it is a no-op (long-press covers it).

### Search

- Search bar (chat header or top) with a **scope toggle**: "Search everywhere" /
  "Search in this chat/channel". → `search` frame → results list → tap jumps to
  the message (navigate + highlight).

### Attach media

- Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
- Selected files show as **preview chips** in the composer (thumbnail + name +
  remove). On send: `media.upload` (chunked) for each, then `message.send` with
  `media_refs`.

### Voice input (mic button)

- Mic button (right of composer, toggles to send when text is present).
- Tap → request `RECORD_AUDIO` permission → start recording (MediaRecorder,
  OGG/Opus, 44.1 kHz mono).
- While recording: timer + waveform; tap again to stop.
- On stop: file becomes a **preview chip** (audio, kind=`voice`) in the
  composer, same as attached media. Send → `media.upload` (chunked) →
  `message.send` with `media_refs`.
- Hermes receives it as an audio attachment; the agent's STT (if configured)
  transcribes it. No client-side STT.

### Push notifications

- FCM service (see `08-push.md`): foreground banner + background foreground
  service → `sync`. Notification channel per chat. Tap → deep-link to chat.
- ntfy fallback: foreground service maintains the subscription.

### Live playback

- AI-sent audio/video → `media.pull` → cache file → **ExoPlayer** inline player
  (audio: mini-player; video: inline + fullscreen + PiP). Documents/images →
  viewer / open-with.

## 10.6 Layout (Telegram-style, per reference image)

**Two layout modes** (decision: user-toggleable, **single-pane default**):

- **Single-pane (default on phones):** chat full-screen; channel list in a
  swipeable drawer (hamburger / edge swipe).
- **Two-pane (Telegram-style, like the reference):** persistent left channel
  rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.

**Chat screen anatomy (matches reference):**

- **Header:** back (single-pane), avatar, name + "Bot" subtitle, edit + overflow
  (⋮) menu (thread toggle, channel menu, set cron target, clear).
- **Message list:** date separators ("7. August"); user bubbles **right**
  (accent color, ✓✓ read receipts); agent bubbles **left** (surface color) with
  optional ReasoningBlock + ToolCards + media + model/token footer
  ("Qwen3-… · 11% · ~") + timestamp.
- **In-app banner** above the composer (e.g. "renamed topic").
- **Composer:** `Menü` / emoji / auto-grow input / attach / mic-send.

**Theme:** dark by default (reference is dark); Material 3; optional dynamic
color. Accent = user's chosen brand color (default indigo, like the reference).

## 10.7 SQLDelight schema (cache)

Implemented in `app/shared/src/commonMain/sqldelight/iris/db/Cache.sq`
(database `IrisDatabase`, package `iris.db`). Rows are stored as **JSON
payloads** so the schema does not drift with the Kotlin model fields
(`MessageItem` / `ChannelInfo` are `@Serializable`):

- `message(lane PK, id PK, ts, payload)` — one row per persisted
  `MessageItem`; `lane` is the lane key (`chatId` or `chatId::threadId`),
  `ts` for ordering. Local system notices are **not** persisted (ephemeral).
- `tool(lane PK, id PK, seq, payload)` — one row per persisted `ToolItem`
  (tool-activity card). Tool cards are **not** part of the gateway's
  `history` (which carries final messages only), so the app persists them
  itself to restore them across a restart. The payload carries `anchor_id`
  (the id of the message the card follows — the last non-streaming message
  when the tool started); on load the card is inserted after its anchor, so
  the order user message → tool card → answer survives a restart. A card
  whose anchor is gone (deleted message) falls to the end of the lane; an
  open card (process died before `tool.end`) is restored closed as
  interrupted.
- `channel(chat_id PK, payload)` — the whole channel directory (channels +
  threads), so the drawer works offline.
- `meta(key PK, value)` — small UI state (currently: `last_lane`, the
  last-viewed lane, restored on startup).

Not in the DB (already persisted elsewhere): sync cursor + settings live in
`SecureStore`; media files in the `MediaCache` (files, not DB rows).

## 10.9 Local cache (offline reading, instant start)

The server is authoritative; the SQLite cache (10.7) makes the app feel
instant and readable offline (industry-standard chat-app behavior):

- **Startup:** `IrisController` restores `ChatStore` + `ChannelStore` from
  the cache *before* the gateway connection is up — the UI is populated
  immediately, no waiting for `history`/`sync`. Restored state is sanitized:
  a streaming bubble is finalized (the `sync` replay finalizes it for real)
  and a pending send becomes *failed* (tap to retry).
- **Writes:** the controller snapshots the in-memory stores into the cache,
  debounced (750 ms) — one atomic full rewrite per change, so every mutation
  path (frames, media pulls, read receipts, deletes, retries) is covered
  without per-mutation hooks. A final synchronous flush runs on `dispose`.
  The outbox replay on reconnect is the safety net for anything lost in the
  debounce window (the cursor only advances on `sync.done`).
- **Connect:** the last-viewed lane (from `meta`) is kept if it still exists
  on the server, otherwise the app falls back to the home channel. The full
  `history` of the active lane is reloaded either way (the cache may be
  stale; `sync` only covers the outbox delta since the saved cursor). This
  runs on a **fast path** (`GatewayClient.onHelloAck`, fired on the WS thread
  the moment `hello.ack` lands) rather than the state collector, which can be
  starved for seconds during startup on slow devices — getting the request
  out early so the response lands inside a flaky network's window. A lane is
  marked "history loaded" only when the response is actually processed, so a
  request/response lost in a WS drop is retried on the next (re)connect.
- **Offline:** with no connection the cached lanes/channels are fully
  readable (the composer is disabled, a connection banner is shown). New
  frames reconcile the cache on reconnect via `sync` + `history`.
- **Forget pairing:** `chatDb.clearAll()` wipes the cache (a different
  gateway means a different chat universe).

Storage: `AndroidSqliteDriver` (app database dir) on Android,
`JdbcSqliteDriver` (`~/.iris/iris_cache.db`) on desktop — both via the
`createCacheDriver()` platform actual (`iris/platform/PlatformStorage.kt`).

## 10.8 Onboarding / Connect screen

- First launch → **Connect**: server URL + token (or scan QR). "Test connection"
  does a real `hello` (not just a TCP probe — per hermes desktop guidance, the
  auth leg must be exercised). On success → save (secure storage) → main.
- **Scan QR** (Android only, `docs/20`): a button below the token field opens
  `QrScanActivity` (CameraX + ML Kit, on-device, no Play services), requests the
  `CAMERA` permission, and pre-fills URL + token from the decoded
  `iris://pair…` payload via `PairLink.parse`. It never auto-connects — the
  user still taps "Test & Connect". A non-pairing QR sets an error and leaves
  the fields untouched. The same payload also arrives as an `iris://pair` deep
  link (system scanner / other phones) and pre-fills the screen the same way.
  Hidden on desktop (no camera).
- States: connecting / connected / reconnecting / degraded / auth-failed — each
  with honest copy and a way out.