322 lines
17 KiB
Markdown
322 lines
17 KiB
Markdown
# 10 — Android App (Kotlin + Jetpack Compose)
|
||
|
||
Native client. Lives in the Compose Multiplatform project at `app/`; the bulk of
|
||
the code is in `app/shared` (commonMain) so the Desktop app reuses it.
|
||
|
||
## 10.1 Tech stack
|
||
|
||
| Concern | Choice |
|
||
| --- | --- |
|
||
| Language | Kotlin |
|
||
| UI | Jetpack Compose (Material 3), Compose Navigation |
|
||
| Async | Kotlinx Coroutines + Flow |
|
||
| WS client | OkHttp (`WebSocketListener`) |
|
||
| JSON | kotlinx-serialization |
|
||
| Local DB | **SQLDelight** (KMP; local cache: messages per lane, channel directory, meta — see 10.7/10.9. Sync cursor + settings live in `SecureStore`, media files in `MediaCache`) |
|
||
| Media playback | Media3 **ExoPlayer** (audio + video) |
|
||
| Push | Firebase Messaging (FCM) [primary] / ntfy listener [fallback] |
|
||
| DI | Hilt |
|
||
| Images | Coil |
|
||
| minSdk / target | **26** / 34 (test device: MIX 2S, API 29) |
|
||
|
||
## 10.2 Module layout
|
||
|
||
```
|
||
app/shared/src/
|
||
├── commonMain/kotlin/iris/
|
||
│ ├── protocol/ # frame data classes (mirror of gateway protocol.py)
|
||
│ ├── net/ # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
|
||
│ ├── data/ # ChannelRepository, MessageRepository, MediaRepository,
|
||
│ │ # SearchRepository, SettingsRepository (SQLDelight + WS)
|
||
│ ├── state/ # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
|
||
│ ├── ui/
|
||
│ │ ├── theme/ # Material 3 theme (dark default), type, color
|
||
│ │ ├── components/ # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
|
||
│ │ │ # ChannelRow, Composer, PickerSheet, SearchBar, Banner
|
||
│ │ └── screens/ # ConnectScreen, ChatListScreen, ChatScreen,
|
||
│ │ # SearchScreen, SettingsScreen, ChannelMenu
|
||
│ └── util/ # time formatting, markdown, id gen
|
||
├── androidMain/kotlin/iris/
|
||
│ ├── platform/ # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
|
||
│ │ # Notifications, SecureStore (EncryptedSharedPreferences)
|
||
│ ├── fcm/ # FirebaseMessagingService, foreground sync service
|
||
│ └── IrisApp.kt # Application (Hilt), notification channels
|
||
└── androidApp/ # MainActivity, AndroidManifest, res, google-services
|
||
```
|
||
|
||
## 10.3 `GatewayClient` (net)
|
||
|
||
- OkHttp `WebSocket` to `ws(s)://host:port/ws`.
|
||
- **Reconnect:** exponential backoff + jitter; on reconnect send `hello` then
|
||
`sync {cursor}` (replays undelivered outbox only).
|
||
- **Initial channel open:** on first open of a chat/thread, send `history`
|
||
(newest page) to populate the view. `sync` does NOT load history.
|
||
- **Heartbeat:** send `ping` every 20s; reap on missed `pong` (3×) → reconnect.
|
||
- **Request/response:** map of `id → CompletableDeferred`; events → a
|
||
`SharedFlow<Frame>` consumed by repositories.
|
||
- **Backpressure:** collect frames on a bounded channel; coalesce
|
||
`message.update` per `message_id` (keep latest) to avoid flooding the UI.
|
||
- **Thread:** all frame emission on a single dispatcher; UI observes via Flow.
|
||
|
||
## 10.4 State rules (from hermes desktop AGENTS.md — apply here)
|
||
|
||
- **Server is authoritative** for channels/messages/cursor; the app's Room copy
|
||
is a **cache**. Reconcile (merge, don't clobber) on `channel.*`/`sync`.
|
||
- **Optimistic then honest:** send a message → show it immediately (pending),
|
||
roll back visibly on `error`, authoritative `sync` gets the last word.
|
||
- **Guard against the past:** generation counters / request tokens so a stale
|
||
response never overwrites newer intent.
|
||
- **Isolate the foreground:** only the visible chat publishes into the shared
|
||
view; background chats update their own cache quietly.
|
||
- **Coalesce noise, flush signal:** batch cosmetic updates; let terminal
|
||
transitions (turn done, needs input, failed) reach the user immediately.
|
||
|
||
## 10.5 Feature implementation (your checklist)
|
||
|
||
### Input box, auto-grow (max height)
|
||
|
||
- Compose `BasicTextField` inside a `Box` with
|
||
`Modifier.heightIn(min = 1.line, max = 160.dp)`. Grows with lines, caps at
|
||
160dp, then scrolls internally.
|
||
- Bottom bar (matches reference): `Menü` button (left), emoji button, the
|
||
auto-grow field, attach (paperclip), mic/send (right). Send on Enter
|
||
(configurable: Enter=send vs Enter=newline).
|
||
|
||
### Slash commands
|
||
|
||
- **`/` drawer (implemented):** typing `/` in the composer rolls a drawer up
|
||
over the input listing the command catalog. The catalog is served by the
|
||
gateway via `commands.catalog` request → response with
|
||
`{name, description, args_hint, category, aliases}` per command (derived
|
||
from hermes' `COMMAND_REGISTRY`, gateway-available subset + plugin commands),
|
||
so it always matches hermes (`/new`, `/model`, `/reasoning`, `/status`,
|
||
`/cron`, …). The drawer is shown while the input is a bare `/…` (no space
|
||
yet); typing fuzzy-filters it **client-side** (subsequence match over name +
|
||
aliases — no `commands.complete` round-trip). Tapping a row sends
|
||
`message.send {text:"/cmd"}` and closes the drawer; an unknown command
|
||
empties the match list and closes it (the raw text can still be sent —
|
||
hermes answers with its unknown-command reply). Escape (desktop) clears the
|
||
`/` input.
|
||
- **Menu button (planned):** `Menü` opens a **bottom sheet** listing the same
|
||
catalog for discovery without typing.
|
||
- **Interactive commands** (`/model`, `/reasoning`, `/fast`, approvals,
|
||
clarifies) render as **native pickers** from `picker.*` frames (a dialog /
|
||
sheet with the options; answer via `picker.select`) — planned.
|
||
|
||
### Streaming (app-controlled)
|
||
|
||
- **Settings → "Streaming"** toggle (default on). When off, the app ignores
|
||
`message.start`/`message.update` frames and shows each reply as a single
|
||
final message on `message.stop` (the typing indicator covers the wait).
|
||
Per-device display preference — the gateway keeps streaming for other
|
||
devices (see `05-streaming.md` §5.1).
|
||
|
||
### Tool output (app-controlled verbosity)
|
||
|
||
- `ToolCard` renders `tool.start/progress/end` frames.
|
||
- The gateway always supplies the **full** tool data: it forces
|
||
`display.platforms.android.tool_progress: verbose` (so the progress line
|
||
carries the full args JSON → `tool.start.args`) and captures each completed
|
||
call via the `post_tool_call` hook (→ `tool.end` `output_preview` /
|
||
`duration` / `ok`). The app decides how much to show.
|
||
- **Settings → "Tool detail": Everything / Truncated / Nothing.**
|
||
- Everything: cards start expanded — name + full args + output.
|
||
- Truncated (default): compact one-liner; tap a card to reveal the full
|
||
args + output (what was actually called).
|
||
- Nothing: suppress tool cards.
|
||
- Spinner while running; ✓/✗ + duration on `tool.end`.
|
||
|
||
### Reasoning before message
|
||
|
||
- `ReasoningBlock` (collapsible, "💭 Reasoning" header, monospace body, **copy**
|
||
button) rendered **above** the message body from the `reasoning` field.
|
||
Matches the reference screenshot.
|
||
- **Settings → "Reasoning"** toggle (default on): auto-collapse — long
|
||
reasoning blocks start collapsed, short ones expanded. Off: all reasoning
|
||
blocks start expanded. Per-device display preference; the per-block
|
||
tap-to-toggle always works.
|
||
|
||
### Intermediate messages
|
||
|
||
- `commentary` frames → dimmed/smaller bubble, distinct from final answers.
|
||
|
||
### Message selection + delete
|
||
|
||
- **Long-press** a message bubble (touch) or **right-click** it (desktop mouse)
|
||
enters selection mode: the tapped message is selected (a circular check appears
|
||
beside each bubble) and the composer is replaced by a selection toolbar
|
||
(count + **Delete** + cancel ✕).
|
||
- While selecting, **tap** a bubble to toggle it; the ✕ (or deselecting the last
|
||
message) exits selection mode. Only finalized messages are selectable — a
|
||
streaming bubble has no final id yet and a pending echo isn't on the server.
|
||
- **Delete** → confirm dialog → `message.delete {message_ids:[…]}` for the
|
||
current lane. The server completely deletes the message(s): they are removed
|
||
from the outbox (so `history`/`sync` no longer return them) **and** from the
|
||
hermes session store (so no search trace survives and they are not
|
||
recoverable), and `message.deleted` is broadcast to every device; each device
|
||
drops them from its cache (the requesting device also drops them locally for
|
||
snappy UX). Deleting is idempotent.
|
||
|
||
### Agent busy / stop / steer
|
||
|
||
- `agent.busy` → show "thinking…" indicator in chat header (animated dots).
|
||
- `agent.idle` → clear indicator.
|
||
- **Stop button** (appears in header while busy): sends `agent.stop`.
|
||
- **Steer:** while busy, the composer accepts input; sending it calls
|
||
`agent.steer` (injects mid-turn) rather than queuing a new `message.send`.
|
||
|
||
### Threading + channels
|
||
|
||
- **Channel list** (drawer on single-pane; left rail on two-pane) = default chat
|
||
- user channels (avatar, name, last-message preview, unread badge, active
|
||
highlight bar) — matches the reference left sidebar.
|
||
- **Thread toggle** in the default chat header: "Threads on/off". On → topic
|
||
switcher above the message list (each topic = a `thread_id`).
|
||
- **Auto-threading** (Threads on, Telegram topic-mode workflow,
|
||
`06-channels-cron-search.md` §6.3.1): a message sent in a channel's flat
|
||
lane goes out as `message.send {auto_thread:true}`; the gateway mints a
|
||
fresh thread (instant derived name, AI-named a moment later via
|
||
`channel.renamed`) and the app jumps into it, relocating the optimistic
|
||
bubble from the flat lane.
|
||
- **New channel** (FAB / channel-list menu) → `channel.create` → appears in list;
|
||
menu offers "Set as cron target".
|
||
- **Topic context menu** — long-press a topic chip (touch) or right-click it
|
||
(desktop mouse) → "Rename" / "Delete". Rename → `channel.rename` (prefilled
|
||
dialog); Delete → confirm → `channel.delete` (hard delete; the thread leaves
|
||
the switcher, its history is wiped from the outbox and session store, and if
|
||
it was the open lane the app falls back to the channel's flat lane). The
|
||
right-click handler is a skiko `expect`/`actual`
|
||
(`iris/ui/ContextMenu.kt`); on touch it is a no-op (long-press covers it).
|
||
|
||
### Search
|
||
|
||
- Search bar (chat header or top) with a **scope toggle**: "Search everywhere" /
|
||
"Search in this chat/channel". → `search` frame → results list → tap jumps to
|
||
the message (navigate + highlight).
|
||
|
||
### Attach media
|
||
|
||
- Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
|
||
- Selected files show as **preview chips** in the composer (thumbnail + name +
|
||
remove). On send: `media.upload` (chunked) for each, then `message.send` with
|
||
`media_refs`.
|
||
|
||
### Voice input (mic button)
|
||
|
||
- Mic button (right of composer, toggles to send when text is present).
|
||
- Tap → request `RECORD_AUDIO` permission → start recording (MediaRecorder,
|
||
OGG/Opus, 44.1 kHz mono).
|
||
- While recording: timer + waveform; tap again to stop.
|
||
- On stop: file becomes a **preview chip** (audio, kind=`voice`) in the
|
||
composer, same as attached media. Send → `media.upload` (chunked) →
|
||
`message.send` with `media_refs`.
|
||
- Hermes receives it as an audio attachment; the agent's STT (if configured)
|
||
transcribes it. No client-side STT.
|
||
|
||
### Push notifications
|
||
|
||
- FCM service (see `08-push.md`): foreground banner + background foreground
|
||
service → `sync`. Notification channel per chat. Tap → deep-link to chat.
|
||
- ntfy fallback: foreground service maintains the subscription.
|
||
|
||
### Live playback
|
||
|
||
- AI-sent audio/video → `media.pull` → cache file → **ExoPlayer** inline player
|
||
(audio: mini-player; video: inline + fullscreen + PiP). Documents/images →
|
||
viewer / open-with.
|
||
|
||
## 10.6 Layout (Telegram-style, per reference image)
|
||
|
||
**Two layout modes** (decision: user-toggleable, **single-pane default**):
|
||
|
||
- **Single-pane (default on phones):** chat full-screen; channel list in a
|
||
swipeable drawer (hamburger / edge swipe).
|
||
- **Two-pane (Telegram-style, like the reference):** persistent left channel
|
||
rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.
|
||
|
||
**Chat screen anatomy (matches reference):**
|
||
|
||
- **Header:** back (single-pane), avatar, name + "Bot" subtitle, edit + overflow
|
||
(⋮) menu (thread toggle, channel menu, set cron target, clear).
|
||
- **Message list:** date separators ("7. August"); user bubbles **right**
|
||
(accent color, ✓✓ read receipts); agent bubbles **left** (surface color) with
|
||
optional ReasoningBlock + ToolCards + media + model/token footer
|
||
("Qwen3-… · 11% · ~") + timestamp.
|
||
- **In-app banner** above the composer (e.g. "renamed topic").
|
||
- **Composer:** `Menü` / emoji / auto-grow input / attach / mic-send.
|
||
|
||
**Theme:** dark by default (reference is dark); Material 3; optional dynamic
|
||
color. Accent = user's chosen brand color (default indigo, like the reference).
|
||
|
||
## 10.7 SQLDelight schema (cache)
|
||
|
||
Implemented in `app/shared/src/commonMain/sqldelight/iris/db/Cache.sq`
|
||
(database `IrisDatabase`, package `iris.db`). Rows are stored as **JSON
|
||
payloads** so the schema does not drift with the Kotlin model fields
|
||
(`MessageItem` / `ChannelInfo` are `@Serializable`):
|
||
|
||
- `message(lane PK, id PK, ts, payload)` — one row per persisted
|
||
`MessageItem`; `lane` is the lane key (`chatId` or `chatId::threadId`),
|
||
`ts` for ordering. Local system notices are **not** persisted (ephemeral).
|
||
- `tool(lane PK, id PK, seq, payload)` — one row per persisted `ToolItem`
|
||
(tool-activity card). Tool cards are **not** part of the gateway's
|
||
`history` (which carries final messages only), so the app persists them
|
||
itself to restore them across a restart. The payload carries `anchor_id`
|
||
(the id of the message the card follows — the last non-streaming message
|
||
when the tool started); on load the card is inserted after its anchor, so
|
||
the order user message → tool card → answer survives a restart. A card
|
||
whose anchor is gone (deleted message) falls to the end of the lane; an
|
||
open card (process died before `tool.end`) is restored closed as
|
||
interrupted.
|
||
- `channel(chat_id PK, payload)` — the whole channel directory (channels +
|
||
threads), so the drawer works offline.
|
||
- `meta(key PK, value)` — small UI state (currently: `last_lane`, the
|
||
last-viewed lane, restored on startup).
|
||
|
||
Not in the DB (already persisted elsewhere): sync cursor + settings live in
|
||
`SecureStore`; media files in the `MediaCache` (files, not DB rows).
|
||
|
||
## 10.9 Local cache (offline reading, instant start)
|
||
|
||
The server is authoritative; the SQLite cache (10.7) makes the app feel
|
||
instant and readable offline (industry-standard chat-app behavior):
|
||
|
||
- **Startup:** `IrisController` restores `ChatStore` + `ChannelStore` from
|
||
the cache *before* the gateway connection is up — the UI is populated
|
||
immediately, no waiting for `history`/`sync`. Restored state is sanitized:
|
||
a streaming bubble is finalized (the `sync` replay finalizes it for real)
|
||
and a pending send becomes *failed* (tap to retry).
|
||
- **Writes:** the controller snapshots the in-memory stores into the cache,
|
||
debounced (750 ms) — one atomic full rewrite per change, so every mutation
|
||
path (frames, media pulls, read receipts, deletes, retries) is covered
|
||
without per-mutation hooks. A final synchronous flush runs on `dispose`.
|
||
The outbox replay on reconnect is the safety net for anything lost in the
|
||
debounce window (the cursor only advances on `sync.done`).
|
||
- **Connect:** the last-viewed lane (from `meta`) is kept if it still exists
|
||
on the server, otherwise the app falls back to the home channel. The full
|
||
`history` of the active lane is reloaded either way (the cache may be
|
||
stale; `sync` only covers the outbox delta since the saved cursor). This
|
||
runs on a **fast path** (`GatewayClient.onHelloAck`, fired on the WS thread
|
||
the moment `hello.ack` lands) rather than the state collector, which can be
|
||
starved for seconds during startup on slow devices — getting the request
|
||
out early so the response lands inside a flaky network's window. A lane is
|
||
marked "history loaded" only when the response is actually processed, so a
|
||
request/response lost in a WS drop is retried on the next (re)connect.
|
||
- **Offline:** with no connection the cached lanes/channels are fully
|
||
readable (the composer is disabled, a connection banner is shown). New
|
||
frames reconcile the cache on reconnect via `sync` + `history`.
|
||
- **Forget pairing:** `chatDb.clearAll()` wipes the cache (a different
|
||
gateway means a different chat universe).
|
||
|
||
Storage: `AndroidSqliteDriver` (app database dir) on Android,
|
||
`JdbcSqliteDriver` (`~/.iris/iris_cache.db`) on desktop — both via the
|
||
`createCacheDriver()` platform actual (`iris/platform/PlatformStorage.kt`).
|
||
|
||
## 10.8 Onboarding / Connect screen
|
||
|
||
- First launch → **Connect**: server URL + token (or scan QR). "Test connection"
|
||
does a real `hello` (not just a TCP probe — per hermes desktop guidance, the
|
||
auth leg must be exercised). On success → save (secure storage) → main.
|
||
- States: connecting / connected / reconnecting / degraded / auth-failed — each
|
||
with honest copy and a way out.
|