iris_x_hermes/docs/10-android-app.md

10 — Android App (Kotlin + Jetpack Compose)

Native client. Lives in the Compose Multiplatform project at app/; the bulk of the code is in app/shared (commonMain) so the Desktop app reuses it.

10.1 Tech stack

Concern	Choice
Language	Kotlin
UI	Jetpack Compose (Material 3), Compose Navigation
Async	Kotlinx Coroutines + Flow
WS client	OkHttp (WebSocketListener)
JSON	kotlinx-serialization
Local DB	SQLDelight (KMP; local cache: messages per lane, channel directory, meta — see 10.7/10.9. Sync cursor + settings live in SecureStore, media files in MediaCache)
Media playback	Media3 ExoPlayer (audio + video)
Push	Firebase Messaging (FCM) [primary] / ntfy listener [fallback]
DI	Hilt
Images	Coil
minSdk / target	26 / 34 (test device: MIX 2S, API 29)

10.2 Module layout

app/shared/src/
├── commonMain/kotlin/iris/
│   ├── protocol/        # frame data classes (mirror of gateway protocol.py)
│   ├── net/             # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
│   ├── data/            # ChannelRepository, MessageRepository, MediaRepository,
│   │                    #   SearchRepository, SettingsRepository (SQLDelight + WS)
│   ├── state/           # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
│   ├── ui/
│   │   ├── theme/       # Material 3 theme (dark default), type, color
│   │   ├── components/  # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
│   │   │                #   ChannelRow, Composer, PickerSheet, SearchBar, Banner
│   │   └── screens/     # ConnectScreen, ChatListScreen, ChatScreen,
│   │                    #   SearchScreen, SettingsScreen, ChannelMenu
│   └── util/            # time formatting, markdown, id gen
├── androidMain/kotlin/iris/
│   ├── platform/        # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
│   │                    #   Notifications, SecureStore (EncryptedSharedPreferences)
│   ├── fcm/             # FirebaseMessagingService, foreground sync service
│   └── IrisApp.kt       # Application (Hilt), notification channels
└── androidApp/          # MainActivity, AndroidManifest, res, google-services

10.3 GatewayClient (net)

• OkHttp WebSocket to ws(s)://host:port/ws.
• Reconnect: exponential backoff + jitter; on reconnect send hello then sync {cursor} (replays undelivered outbox only).
• Initial channel open: on first open of a chat/thread, send history (newest page) to populate the view. sync does NOT load history.
• Heartbeat: send ping every 20s; reap on missed pong (3×) → reconnect.
• Request/response: map of id → CompletableDeferred; events → a SharedFlow<Frame> consumed by repositories.
• Backpressure: collect frames on a bounded channel; coalesce message.update per message_id (keep latest) to avoid flooding the UI.
• Thread: all frame emission on a single dispatcher; UI observes via Flow.

10.4 State rules (from hermes desktop AGENTS.md — apply here)

• Server is authoritative for channels/messages/cursor; the app's Room copy is a cache. Reconcile (merge, don't clobber) on channel.*/sync.
• Optimistic then honest: send a message → show it immediately (pending), roll back visibly on error, authoritative sync gets the last word.
• Guard against the past: generation counters / request tokens so a stale response never overwrites newer intent.
• Isolate the foreground: only the visible chat publishes into the shared view; background chats update their own cache quietly.
• Coalesce noise, flush signal: batch cosmetic updates; let terminal transitions (turn done, needs input, failed) reach the user immediately.

10.5 Feature implementation (your checklist)

Input box, auto-grow (max height)

• Compose BasicTextField inside a Box with Modifier.heightIn(min = 1.line, max = 160.dp). Grows with lines, caps at 160dp, then scrolls internally.
• Bottom bar (matches reference): Menü button (left), emoji button, the auto-grow field, attach (paperclip), mic/send (right). Send on Enter (configurable: Enter=send vs Enter=newline).

Slash commands

• / drawer (implemented): typing / in the composer rolls a drawer up over the input listing the command catalog. The catalog is served by the gateway via commands.catalog request → response with {name, description, args_hint, category, aliases} per command (derived from hermes' COMMAND_REGISTRY, gateway-available subset + plugin commands), so it always matches hermes (/new, /model, /reasoning, /status, /cron, …). The drawer is shown while the input is a bare /… (no space yet); typing fuzzy-filters it client-side (subsequence match over name + aliases — no commands.complete round-trip). Tapping a row sends message.send {text:"/cmd"} and closes the drawer; an unknown command empties the match list and closes it (the raw text can still be sent — hermes answers with its unknown-command reply). Escape (desktop) clears the / input.
• Menu button (planned): Menü opens a bottom sheet listing the same catalog for discovery without typing.
• Interactive commands (/model, /reasoning, /fast, approvals, clarifies) render as native pickers from picker.* frames (a dialog / sheet with the options; answer via picker.select) — planned.

Streaming (app-controlled)

• Settings → "Streaming" toggle (default on). When off, the app ignores message.start/message.update frames and shows each reply as a single final message on message.stop (the typing indicator covers the wait). Per-device display preference — the gateway keeps streaming for other devices (see 05-streaming.md §5.1).

Tool output (app-controlled verbosity)

• ToolCard renders tool.start/progress/end frames.
• The gateway always supplies the full tool data: it forces display.platforms.iris.tool_progress: verbose (so the progress line carries the full args JSON → tool.start.args) and captures each completed call via the post_tool_call hook (→ tool.end output_preview / duration / ok). The app decides how much to show.
• Settings → "Tool detail": Everything / Truncated / Nothing.
  • Everything: cards start expanded — name + full args + output.
  • Truncated (default): compact one-liner; tap a card to reveal the full args + output (what was actually called).
  • Nothing: suppress tool cards.
• Spinner while running; ✓/✗ + duration on tool.end.

Reasoning before message

• ReasoningBlock (collapsible, "💭 Reasoning" header, monospace body, copy button) rendered above the message body from the reasoning field. Matches the reference screenshot.
• Settings → "Reasoning" toggle (default on): auto-collapse — long reasoning blocks start collapsed, short ones expanded. Off: all reasoning blocks start expanded. Per-device display preference; the per-block tap-to-toggle always works.

Intermediate messages

• commentary frames → dimmed/smaller bubble, distinct from final answers.

Message selection + delete

• Long-press a message bubble (touch) or right-click it (desktop mouse) enters selection mode: the tapped message is selected (a circular check appears beside each bubble) and the composer is replaced by a selection toolbar (count + Delete + cancel ✕).
• While selecting, tap a bubble to toggle it; the ✕ (or deselecting the last message) exits selection mode. Only finalized messages are selectable — a streaming bubble has no final id yet and a pending echo isn't on the server.
• Delete → confirm dialog → message.delete {message_ids:[…]} for the current lane. The server completely deletes the message(s): they are removed from the outbox (so history/sync no longer return them) and from the hermes session store (so no search trace survives and they are not recoverable), and message.deleted is broadcast to every device; each device drops them from its cache (the requesting device also drops them locally for snappy UX). Deleting is idempotent.

Agent busy / stop / steer

• agent.busy → show "thinking…" indicator in chat header (animated dots).
• agent.idle → clear indicator.
• Stop button (appears in header while busy): sends agent.stop.
• Steer: while busy, the composer accepts input; sending it calls agent.steer (injects mid-turn) rather than queuing a new message.send.

Threading + channels

• Channel list (drawer on single-pane; left rail on two-pane) = default chat
  • user channels (avatar, name, last-message preview, unread badge, active highlight bar) — matches the reference left sidebar.
• Thread toggle in the default chat header: "Threads on/off". On → topic switcher above the message list (each topic = a thread_id).
• Auto-threading (Threads on, Telegram topic-mode workflow, 06-channels-cron-search.md §6.3.1): a message sent in a channel's flat lane goes out as message.send {auto_thread:true}; the gateway mints a fresh thread (instant derived name, AI-named a moment later via channel.renamed) and the app jumps into it, relocating the optimistic bubble from the flat lane.
• New channel (FAB / channel-list menu) → channel.create → appears in list; menu offers "Set as cron target".
• Topic context menu — long-press a topic chip (touch) or right-click it (desktop mouse) → "Rename" / "Delete". Rename → channel.rename (prefilled dialog); Delete → confirm → channel.delete (hard delete; the thread leaves the switcher, its history is wiped from the outbox and session store, and if it was the open lane the app falls back to the channel's flat lane). The right-click handler is a skiko expect/actual (iris/ui/ContextMenu.kt); on touch it is a no-op (long-press covers it).

Search

• Search bar (chat header or top) with a scope toggle: "Search everywhere" / "Search in this chat/channel". → search frame → results list → tap jumps to the message (navigate + highlight).

Attach media

• Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
• Selected files show as preview chips in the composer (thumbnail + name + remove). On send: media.upload (chunked) for each, then message.send with media_refs.

Voice input (mic button)

• Mic button (right of composer, toggles to send when text is present).
• Tap → request RECORD_AUDIO permission → start recording (MediaRecorder, OGG/Opus, 44.1 kHz mono).
• While recording: timer + waveform; tap again to stop.
• On stop: file becomes a preview chip (audio, kind=voice) in the composer, same as attached media. Send → media.upload (chunked) → message.send with media_refs.
• Hermes receives it as an audio attachment; the agent's STT (if configured) transcribes it. No client-side STT.

Push notifications

• FCM service (see 08-push.md): foreground banner + background foreground service → sync. Notification channel per chat. Tap → deep-link to chat.
• ntfy fallback: foreground service maintains the subscription.

Live playback

• AI-sent audio/video → media.pull → cache file → ExoPlayer inline player (audio: mini-player; video: inline + fullscreen + PiP). Documents/images → viewer / open-with.

10.6 Layout (Telegram-style, per reference image)

Two layout modes (decision: user-toggleable, single-pane default):

• Single-pane (default on phones): chat full-screen; channel list in a swipeable drawer (hamburger / edge swipe).
• Two-pane (Telegram-style, like the reference): persistent left channel rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.

Chat screen anatomy (matches reference):

• Header: back (single-pane), avatar, name + "Bot" subtitle, edit + overflow (⋮) menu (thread toggle, channel menu, set cron target, clear).
• Message list: date separators ("7. August"); user bubbles right (accent color, ✓✓ read receipts); agent bubbles left (surface color) with optional ReasoningBlock + ToolCards + media + model/token footer ("Qwen3-… · 11% · ~") + timestamp.
• In-app banner above the composer (e.g. "renamed topic").
• Composer: Menü / emoji / auto-grow input / attach / mic-send.

Theme: dark by default (reference is dark); Material 3; optional dynamic color. Accent = user's chosen brand color (default indigo, like the reference).

10.7 SQLDelight schema (cache)

Implemented in app/shared/src/commonMain/sqldelight/iris/db/Cache.sq (database IrisDatabase, package iris.db). Rows are stored as JSON payloads so the schema does not drift with the Kotlin model fields (MessageItem / ChannelInfo are @Serializable):

• message(lane PK, id PK, ts, payload) — one row per persisted MessageItem; lane is the lane key (chatId or chatId::threadId), ts for ordering. Local system notices are not persisted (ephemeral).
• tool(lane PK, id PK, seq, payload) — one row per persisted ToolItem (tool-activity card). Tool cards are not part of the gateway's history (which carries final messages only), so the app persists them itself to restore them across a restart. The payload carries anchor_id (the id of the message the card follows — the last non-streaming message when the tool started); on load the card is inserted after its anchor, so the order user message → tool card → answer survives a restart. A card whose anchor is gone (deleted message) falls to the end of the lane; an open card (process died before tool.end) is restored closed as interrupted.
• channel(chat_id PK, payload) — the whole channel directory (channels + threads), so the drawer works offline.
• meta(key PK, value) — small UI state (currently: last_lane, the last-viewed lane, restored on startup).

Not in the DB (already persisted elsewhere): sync cursor + settings live in SecureStore; media files in the MediaCache (files, not DB rows).

10.9 Local cache (offline reading, instant start)

The server is authoritative; the SQLite cache (10.7) makes the app feel instant and readable offline (industry-standard chat-app behavior):

• Startup: IrisController restores ChatStore + ChannelStore from the cache before the gateway connection is up — the UI is populated immediately, no waiting for history/sync. Restored state is sanitized: a streaming bubble is finalized (the sync replay finalizes it for real) and a pending send becomes failed (tap to retry).
• Writes: the controller snapshots the in-memory stores into the cache, debounced (750 ms) — one atomic full rewrite per change, so every mutation path (frames, media pulls, read receipts, deletes, retries) is covered without per-mutation hooks. A final synchronous flush runs on dispose. The outbox replay on reconnect is the safety net for anything lost in the debounce window (the cursor only advances on sync.done).
• Connect: the last-viewed lane (from meta) is kept if it still exists on the server, otherwise the app falls back to the home channel. The full history of the active lane is reloaded either way (the cache may be stale; sync only covers the outbox delta since the saved cursor). This runs on a fast path (GatewayClient.onHelloAck, fired on the WS thread the moment hello.ack lands) rather than the state collector, which can be starved for seconds during startup on slow devices — getting the request out early so the response lands inside a flaky network's window. A lane is marked "history loaded" only when the response is actually processed, so a request/response lost in a WS drop is retried on the next (re)connect.
• Offline: with no connection the cached lanes/channels are fully readable (the composer is disabled, a connection banner is shown). New frames reconcile the cache on reconnect via sync + history.
• Forget pairing: chatDb.clearAll() wipes the cache (a different gateway means a different chat universe).

Storage: AndroidSqliteDriver (app database dir) on Android, JdbcSqliteDriver (~/.iris/iris_cache.db) on desktop — both via the createCacheDriver() platform actual (iris/platform/PlatformStorage.kt).

10.8 Onboarding / Connect screen

• First launch → Connect: server URL + token (or scan QR). "Test connection" does a real hello (not just a TCP probe — per hermes desktop guidance, the auth leg must be exercised). On success → save (secure storage) → main.
• Scan QR (Android only, docs/20): a button below the token field opens QrScanActivity (CameraX + ML Kit, on-device, no Play services), requests the CAMERA permission, and pre-fills URL + token from the decoded iris://pair… payload via PairLink.parse. It never auto-connects — the user still taps "Test & Connect". A non-pairing QR sets an error and leaves the fields untouched. The same payload also arrives as an iris://pair deep link (system scanner / other phones) and pre-fills the screen the same way. Hidden on desktop (no camera).
• States: connecting / connected / reconnecting / degraded / auth-failed — each with honest copy and a way out.