iris_x_hermes/docs/10-android-app.md

10 — Android App (Kotlin + Jetpack Compose)

Native client. Lives in the Compose Multiplatform project at app/; the bulk of the code is in app/shared (commonMain) so the Desktop app reuses it.

10.1 Tech stack

Concern	Choice
Language	Kotlin
UI	Jetpack Compose (Material 3), Compose Navigation
Async	Kotlinx Coroutines + Flow
WS client	OkHttp (WebSocketListener)
JSON	kotlinx-serialization
Local DB	SQLDelight (KMP; channels, messages cache, media index, sync cursor, settings)
Media playback	Media3 ExoPlayer (audio + video)
Push	Firebase Messaging (FCM) [primary] / ntfy listener [fallback]
DI	Hilt
Images	Coil
minSdk / target	26 / 34 (test device: MIX 2S, API 29)

10.2 Module layout

app/shared/src/
├── commonMain/kotlin/iris/
│   ├── protocol/        # frame data classes (mirror of gateway protocol.py)
│   ├── net/             # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
│   ├── data/            # ChannelRepository, MessageRepository, MediaRepository,
│   │                    #   SearchRepository, SettingsRepository (SQLDelight + WS)
│   ├── state/           # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
│   ├── ui/
│   │   ├── theme/       # Material 3 theme (dark default), type, color
│   │   ├── components/  # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
│   │   │                #   ChannelRow, Composer, PickerSheet, SearchBar, Banner
│   │   └── screens/     # ConnectScreen, ChatListScreen, ChatScreen,
│   │                    #   SearchScreen, SettingsScreen, ChannelMenu
│   └── util/            # time formatting, markdown, id gen
├── androidMain/kotlin/iris/
│   ├── platform/        # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
│   │                    #   Notifications, SecureStore (EncryptedSharedPreferences)
│   ├── fcm/             # FirebaseMessagingService, foreground sync service
│   └── IrisApp.kt       # Application (Hilt), notification channels
└── androidApp/          # MainActivity, AndroidManifest, res, google-services

10.3 GatewayClient (net)

• OkHttp WebSocket to ws(s)://host:port/ws.
• Reconnect: exponential backoff + jitter; on reconnect send hello then sync {cursor} (replays undelivered outbox only).
• Initial channel open: on first open of a chat/thread, send history (newest page) to populate the view. sync does NOT load history.
• Heartbeat: send ping every 20s; reap on missed pong (3×) → reconnect.
• Request/response: map of id → CompletableDeferred; events → a SharedFlow<Frame> consumed by repositories.
• Backpressure: collect frames on a bounded channel; coalesce message.update per message_id (keep latest) to avoid flooding the UI.
• Thread: all frame emission on a single dispatcher; UI observes via Flow.

10.4 State rules (from hermes desktop AGENTS.md — apply here)

• Server is authoritative for channels/messages/cursor; the app's Room copy is a cache. Reconcile (merge, don't clobber) on channel.*/sync.
• Optimistic then honest: send a message → show it immediately (pending), roll back visibly on error, authoritative sync gets the last word.
• Guard against the past: generation counters / request tokens so a stale response never overwrites newer intent.
• Isolate the foreground: only the visible chat publishes into the shared view; background chats update their own cache quietly.
• Coalesce noise, flush signal: batch cosmetic updates; let terminal transitions (turn done, needs input, failed) reach the user immediately.

10.5 Feature implementation (your checklist)

Input box, auto-grow (max height)

• Compose BasicTextField inside a Box with Modifier.heightIn(min = 1.line, max = 160.dp). Grows with lines, caps at 160dp, then scrolls internally.
• Bottom bar (matches reference): Menü button (left), emoji button, the auto-grow field, attach (paperclip), mic/send (right). Send on Enter (configurable: Enter=send vs Enter=newline).

Slash commands

• / drawer (implemented): typing / in the composer rolls a drawer up over the input listing the command catalog. The catalog is served by the gateway via commands.catalog request → response with {name, description, args_hint, category, aliases} per command (derived from hermes' COMMAND_REGISTRY, gateway-available subset + plugin commands), so it always matches hermes (/new, /model, /reasoning, /status, /cron, …). The drawer is shown while the input is a bare /… (no space yet); typing fuzzy-filters it client-side (subsequence match over name + aliases — no commands.complete round-trip). Tapping a row sends message.send {text:"/cmd"} and closes the drawer; an unknown command empties the match list and closes it (the raw text can still be sent — hermes answers with its unknown-command reply). Escape (desktop) clears the / input.
• Menu button (planned): Menü opens a bottom sheet listing the same catalog for discovery without typing.
• Interactive commands (/model, /reasoning, /fast, approvals, clarifies) render as native pickers from picker.* frames (a dialog / sheet with the options; answer via picker.select) — planned.

Streaming (app-controlled)

• Settings → "Streaming" toggle (default on). When off, the app ignores message.start/message.update frames and shows each reply as a single final message on message.stop (the typing indicator covers the wait). Per-device display preference — the gateway keeps streaming for other devices (see 05-streaming.md §5.1).

Tool output (app-controlled verbosity)

• ToolCard renders tool.start/progress/end frames.
• The gateway always supplies the full tool data: it forces display.platforms.android.tool_progress: verbose (so the progress line carries the full args JSON → tool.start.args) and captures each completed call via the post_tool_call hook (→ tool.end output_preview / duration / ok). The app decides how much to show.
• Settings → "Tool detail": Everything / Truncated / Nothing.
  • Everything: cards start expanded — name + full args + output.
  • Truncated (default): compact one-liner; tap a card to reveal the full args + output (what was actually called).
  • Nothing: suppress tool cards.
• Spinner while running; ✓/✗ + duration on tool.end.

Reasoning before message

• ReasoningBlock (collapsible, "💭 Reasoning" header, monospace body, copy button) rendered above the message body from the reasoning field. Matches the reference screenshot.
• Settings → "Reasoning" toggle (default on): auto-collapse — long reasoning blocks start collapsed, short ones expanded. Off: all reasoning blocks start expanded. Per-device display preference; the per-block tap-to-toggle always works.

Intermediate messages

• commentary frames → dimmed/smaller bubble, distinct from final answers.

Message selection + delete

• Long-press a message bubble (touch) or right-click it (desktop mouse) enters selection mode: the tapped message is selected (a circular check appears beside each bubble) and the composer is replaced by a selection toolbar (count + Delete + cancel ✕).
• While selecting, tap a bubble to toggle it; the ✕ (or deselecting the last message) exits selection mode. Only finalized messages are selectable — a streaming bubble has no final id yet and a pending echo isn't on the server.
• Delete → confirm dialog → message.delete {message_ids:[…]} for the current lane. The server completely deletes the message(s): they are removed from the outbox (so history/sync no longer return them) and from the hermes session store (so no search trace survives and they are not recoverable), and message.deleted is broadcast to every device; each device drops them from its cache (the requesting device also drops them locally for snappy UX). Deleting is idempotent.

Agent busy / stop / steer

• agent.busy → show "thinking…" indicator in chat header (animated dots).
• agent.idle → clear indicator.
• Stop button (appears in header while busy): sends agent.stop.
• Steer: while busy, the composer accepts input; sending it calls agent.steer (injects mid-turn) rather than queuing a new message.send.

Threading + channels

• Channel list (drawer on single-pane; left rail on two-pane) = default chat
  • user channels (avatar, name, last-message preview, unread badge, active highlight bar) — matches the reference left sidebar.
• Thread toggle in the default chat header: "Threads on/off". On → topic switcher above the message list (each topic = a thread_id).
• Auto-threading (Threads on, Telegram topic-mode workflow, 06-channels-cron-search.md §6.3.1): a message sent in a channel's flat lane goes out as message.send {auto_thread:true}; the gateway mints a fresh thread (instant derived name, AI-named a moment later via channel.renamed) and the app jumps into it, relocating the optimistic bubble from the flat lane.
• New channel (FAB / channel-list menu) → channel.create → appears in list; menu offers "Set as cron target".
• Topic context menu — long-press a topic chip (touch) or right-click it (desktop mouse) → "Rename" / "Delete". Rename → channel.rename (prefilled dialog); Delete → confirm → channel.delete (hard delete; the thread leaves the switcher, its history is wiped from the outbox and session store, and if it was the open lane the app falls back to the channel's flat lane). The right-click handler is a skiko expect/actual (iris/ui/ContextMenu.kt); on touch it is a no-op (long-press covers it).

Search

• Search bar (chat header or top) with a scope toggle: "Search everywhere" / "Search in this chat/channel". → search frame → results list → tap jumps to the message (navigate + highlight).

Attach media

• Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
• Selected files show as preview chips in the composer (thumbnail + name + remove). On send: media.upload (chunked) for each, then message.send with media_refs.

Voice input (mic button)

• Mic button (right of composer, toggles to send when text is present).
• Tap → request RECORD_AUDIO permission → start recording (MediaRecorder, OGG/Opus, 44.1 kHz mono).
• While recording: timer + waveform; tap again to stop.
• On stop: file becomes a preview chip (audio, kind=voice) in the composer, same as attached media. Send → media.upload (chunked) → message.send with media_refs.
• Hermes receives it as an audio attachment; the agent's STT (if configured) transcribes it. No client-side STT.

Push notifications

• FCM service (see 08-push.md): foreground banner + background foreground service → sync. Notification channel per chat. Tap → deep-link to chat.
• ntfy fallback: foreground service maintains the subscription.

Live playback

• AI-sent audio/video → media.pull → cache file → ExoPlayer inline player (audio: mini-player; video: inline + fullscreen + PiP). Documents/images → viewer / open-with.

10.6 Layout (Telegram-style, per reference image)

Two layout modes (decision: user-toggleable, single-pane default):

• Single-pane (default on phones): chat full-screen; channel list in a swipeable drawer (hamburger / edge swipe).
• Two-pane (Telegram-style, like the reference): persistent left channel rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.

Chat screen anatomy (matches reference):

• Header: back (single-pane), avatar, name + "Bot" subtitle, edit + overflow (⋮) menu (thread toggle, channel menu, set cron target, clear).
• Message list: date separators ("7. August"); user bubbles right (accent color, ✓✓ read receipts); agent bubbles left (surface color) with optional ReasoningBlock + ToolCards + media + model/token footer ("Qwen3-… · 11% · ~") + timestamp.
• In-app banner above the composer (e.g. "renamed topic").
• Composer: Menü / emoji / auto-grow input / attach / mic-send.

Theme: dark by default (reference is dark); Material 3; optional dynamic color. Accent = user's chosen brand color (default indigo, like the reference).

10.7 SQLDelight schema (cache)

• channels(chat_id PK, name, kind, parent_chat_id, is_default, last_preview, last_ts, unread).
• messages(id PK, chat_id, thread_id, role, text, reasoning, model, tokens, ts, status[pending|sent|read], media_json).
• media(media_id PK, local_path, kind, mime, size, ts).
• meta(key PK, value) — sync cursor, settings, device_id, server url, pinned cert fingerprint.

10.8 Onboarding / Connect screen

• First launch → Connect: server URL + token (or scan QR). "Test connection" does a real hello (not just a TCP probe — per hermes desktop guidance, the auth leg must be exercised). On success → save (secure storage) → main.
• States: connecting / connected / reconnecting / degraded / auth-failed — each with honest copy and a way out.