ARIA
60ec2b44a7
- Auto-threading (Telegram topic-mode workflow): message.send {auto_thread}
mints a fresh AI-named thread (instant derived title, LLM upgrade via
channel.renamed); channel.created {auto:true}; the app jumps into the new
thread and relocates the optimistic pending bubble.
- history frame: paged full message history for initial channel open /
scroll-up pagination (reconstructed from the outbox log).
- Streaming on/off: gateway side (display.platforms.android.streaming) plus a
per-device app toggle (Settings → Streaming); reasoning/model/tokens carried
on message frames.
- Context menu: long-press (touch) / right-click (desktop) thread affordances
via a KMP rightClick expect/actual.
- Settings → Reasoning: auto-collapse long reasoning blocks (default on).
- Tool detail: the gateway now always supplies full tool data — it forces
verbose tool progress (full args → tool.start.args) and captures each
completed call via the post_tool_call hook (output/duration/ok → tool.end).
The app reveals the full call + output on expand (Truncated) and
auto-expands cards in Everything mode.
226 lines
12 KiB
Markdown
226 lines
12 KiB
Markdown
# 10 — Android App (Kotlin + Jetpack Compose)
|
||
|
||
Native client. Lives in the Compose Multiplatform project at `app/`; the bulk of
|
||
the code is in `app/shared` (commonMain) so the Desktop app reuses it.
|
||
|
||
## 10.1 Tech stack
|
||
|
||
| Concern | Choice |
|
||
|---|---|
|
||
| Language | Kotlin |
|
||
| UI | Jetpack Compose (Material 3), Compose Navigation |
|
||
| Async | Kotlinx Coroutines + Flow |
|
||
| WS client | OkHttp (`WebSocketListener`) |
|
||
| JSON | kotlinx-serialization |
|
||
| Local DB | **SQLDelight** (KMP; channels, messages cache, media index, sync cursor, settings) |
|
||
| Media playback | Media3 **ExoPlayer** (audio + video) |
|
||
| Push | Firebase Messaging (FCM) [primary] / ntfy listener [fallback] |
|
||
| DI | Hilt |
|
||
| Images | Coil |
|
||
| minSdk / target | **26** / 34 (test device: MIX 2S, API 29) |
|
||
|
||
## 10.2 Module layout
|
||
|
||
```
|
||
app/shared/src/
|
||
├── commonMain/kotlin/iris/
|
||
│ ├── protocol/ # frame data classes (mirror of gateway protocol.py)
|
||
│ ├── net/ # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
|
||
│ ├── data/ # ChannelRepository, MessageRepository, MediaRepository,
|
||
│ │ # SearchRepository, SettingsRepository (SQLDelight + WS)
|
||
│ ├── state/ # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
|
||
│ ├── ui/
|
||
│ │ ├── theme/ # Material 3 theme (dark default), type, color
|
||
│ │ ├── components/ # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
|
||
│ │ │ # ChannelRow, Composer, PickerSheet, SearchBar, Banner
|
||
│ │ └── screens/ # ConnectScreen, ChatListScreen, ChatScreen,
|
||
│ │ # SearchScreen, SettingsScreen, ChannelMenu
|
||
│ └── util/ # time formatting, markdown, id gen
|
||
├── androidMain/kotlin/iris/
|
||
│ ├── platform/ # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
|
||
│ │ # Notifications, SecureStore (EncryptedSharedPreferences)
|
||
│ ├── fcm/ # FirebaseMessagingService, foreground sync service
|
||
│ └── IrisApp.kt # Application (Hilt), notification channels
|
||
└── androidApp/ # MainActivity, AndroidManifest, res, google-services
|
||
```
|
||
|
||
## 10.3 `GatewayClient` (net)
|
||
|
||
- OkHttp `WebSocket` to `ws(s)://host:port/ws`.
|
||
- **Reconnect:** exponential backoff + jitter; on reconnect send `hello` then
|
||
`sync {cursor}` (replays undelivered outbox only).
|
||
- **Initial channel open:** on first open of a chat/thread, send `history`
|
||
(newest page) to populate the view. `sync` does NOT load history.
|
||
- **Heartbeat:** send `ping` every 20s; reap on missed `pong` (3×) → reconnect.
|
||
- **Request/response:** map of `id → CompletableDeferred`; events → a
|
||
`SharedFlow<Frame>` consumed by repositories.
|
||
- **Backpressure:** collect frames on a bounded channel; coalesce
|
||
`message.update` per `message_id` (keep latest) to avoid flooding the UI.
|
||
- **Thread:** all frame emission on a single dispatcher; UI observes via Flow.
|
||
|
||
## 10.4 State rules (from hermes desktop AGENTS.md — apply here)
|
||
|
||
- **Server is authoritative** for channels/messages/cursor; the app's Room copy
|
||
is a **cache**. Reconcile (merge, don't clobber) on `channel.*`/`sync`.
|
||
- **Optimistic then honest:** send a message → show it immediately (pending),
|
||
roll back visibly on `error`, authoritative `sync` gets the last word.
|
||
- **Guard against the past:** generation counters / request tokens so a stale
|
||
response never overwrites newer intent.
|
||
- **Isolate the foreground:** only the visible chat publishes into the shared
|
||
view; background chats update their own cache quietly.
|
||
- **Coalesce noise, flush signal:** batch cosmetic updates; let terminal
|
||
transitions (turn done, needs input, failed) reach the user immediately.
|
||
|
||
## 10.5 Feature implementation (your checklist)
|
||
|
||
### Input box, auto-grow (max height)
|
||
- Compose `BasicTextField` inside a `Box` with
|
||
`Modifier.heightIn(min = 1.line, max = 160.dp)`. Grows with lines, caps at
|
||
160dp, then scrolls internally.
|
||
- Bottom bar (matches reference): `Menü` button (left), emoji button, the
|
||
auto-grow field, attach (paperclip), mic/send (right). Send on Enter
|
||
(configurable: Enter=send vs Enter=newline).
|
||
|
||
### Menu button → all slash commands
|
||
- `Menü` opens a **bottom sheet** listing the command catalog. The catalog is
|
||
served by the gateway via `commands.catalog` request → response with
|
||
`{name, description, args_hint, category}` per command, so it always matches
|
||
hermes (`/new`, `/model`, `/reasoning`, `/status`, `/cron`, `/tools`, …).
|
||
- Typing `/` in the field shows **autocomplete** via `commands.complete`
|
||
request (gateway matches the typed prefix). Selecting inserts `/cmd `.
|
||
- Selecting a command sends `message.send {text:"/cmd args"}`.
|
||
- **Interactive commands** (`/model`, `/reasoning`, `/fast`, approvals,
|
||
clarifies) render as **native pickers** from `picker.*` frames (a dialog /
|
||
sheet with the options; answer via `picker.select`).
|
||
|
||
### Streaming (app-controlled)
|
||
- **Settings → "Streaming"** toggle (default on). When off, the app ignores
|
||
`message.start`/`message.update` frames and shows each reply as a single
|
||
final message on `message.stop` (the typing indicator covers the wait).
|
||
Per-device display preference — the gateway keeps streaming for other
|
||
devices (see `05-streaming.md` §5.1).
|
||
|
||
### Tool output (app-controlled verbosity)
|
||
- `ToolCard` renders `tool.start/progress/end` frames.
|
||
- The gateway always supplies the **full** tool data: it forces
|
||
`display.platforms.android.tool_progress: verbose` (so the progress line
|
||
carries the full args JSON → `tool.start.args`) and captures each completed
|
||
call via the `post_tool_call` hook (→ `tool.end` `output_preview` /
|
||
`duration` / `ok`). The app decides how much to show.
|
||
- **Settings → "Tool detail": Everything / Truncated / Nothing.**
|
||
- Everything: cards start expanded — name + full args + output.
|
||
- Truncated (default): compact one-liner; tap a card to reveal the full
|
||
args + output (what was actually called).
|
||
- Nothing: suppress tool cards.
|
||
- Spinner while running; ✓/✗ + duration on `tool.end`.
|
||
|
||
### Reasoning before message
|
||
- `ReasoningBlock` (collapsible, "💭 Reasoning" header, monospace body, **copy**
|
||
button) rendered **above** the message body from the `reasoning` field.
|
||
Matches the reference screenshot.
|
||
- **Settings → "Reasoning"** toggle (default on): auto-collapse — long
|
||
reasoning blocks start collapsed, short ones expanded. Off: all reasoning
|
||
blocks start expanded. Per-device display preference; the per-block
|
||
tap-to-toggle always works.
|
||
|
||
### Intermediate messages
|
||
- `commentary` frames → dimmed/smaller bubble, distinct from final answers.
|
||
|
||
### Agent busy / stop / steer
|
||
- `agent.busy` → show "thinking…" indicator in chat header (animated dots).
|
||
- `agent.idle` → clear indicator.
|
||
- **Stop button** (appears in header while busy): sends `agent.stop`.
|
||
- **Steer:** while busy, the composer accepts input; sending it calls
|
||
`agent.steer` (injects mid-turn) rather than queuing a new `message.send`.
|
||
|
||
### Threading + channels
|
||
- **Channel list** (drawer on single-pane; left rail on two-pane) = default chat
|
||
+ user channels (avatar, name, last-message preview, unread badge, active
|
||
highlight bar) — matches the reference left sidebar.
|
||
- **Thread toggle** in the default chat header: "Threads on/off". On → topic
|
||
switcher above the message list (each topic = a `thread_id`).
|
||
- **Auto-threading** (Threads on, Telegram topic-mode workflow,
|
||
`06-channels-cron-search.md` §6.3.1): a message sent in a channel's flat
|
||
lane goes out as `message.send {auto_thread:true}`; the gateway mints a
|
||
fresh thread (instant derived name, AI-named a moment later via
|
||
`channel.renamed`) and the app jumps into it, relocating the optimistic
|
||
bubble from the flat lane.
|
||
- **New channel** (FAB / channel-list menu) → `channel.create` → appears in list;
|
||
menu offers "Set as cron target".
|
||
- **Topic context menu** — long-press a topic chip (touch) or right-click it
|
||
(desktop mouse) → "Rename" / "Delete". Rename → `channel.rename` (prefilled
|
||
dialog); Delete → confirm → `channel.delete` (soft-delete; the thread leaves
|
||
the switcher and, if it was the open lane, the app falls back to the channel's
|
||
flat lane). The right-click handler is a skiko `expect`/`actual`
|
||
(`iris/ui/ContextMenu.kt`); on touch it is a no-op (long-press covers it).
|
||
|
||
### Search
|
||
- Search bar (chat header or top) with a **scope toggle**: "Search everywhere" /
|
||
"Search in this chat/channel". → `search` frame → results list → tap jumps to
|
||
the message (navigate + highlight).
|
||
|
||
### Attach media
|
||
- Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
|
||
- Selected files show as **preview chips** in the composer (thumbnail + name +
|
||
remove). On send: `media.upload` (chunked) for each, then `message.send` with
|
||
`media_refs`.
|
||
|
||
### Voice input (mic button)
|
||
- Mic button (right of composer, toggles to send when text is present).
|
||
- Tap → request `RECORD_AUDIO` permission → start recording (MediaRecorder,
|
||
OGG/Opus, 44.1 kHz mono).
|
||
- While recording: timer + waveform; tap again to stop.
|
||
- On stop: file becomes a **preview chip** (audio, kind=`voice`) in the
|
||
composer, same as attached media. Send → `media.upload` (chunked) →
|
||
`message.send` with `media_refs`.
|
||
- Hermes receives it as an audio attachment; the agent's STT (if configured)
|
||
transcribes it. No client-side STT.
|
||
|
||
### Push notifications
|
||
- FCM service (see `08-push.md`): foreground banner + background foreground
|
||
service → `sync`. Notification channel per chat. Tap → deep-link to chat.
|
||
- ntfy fallback: foreground service maintains the subscription.
|
||
|
||
### Live playback
|
||
- AI-sent audio/video → `media.pull` → cache file → **ExoPlayer** inline player
|
||
(audio: mini-player; video: inline + fullscreen + PiP). Documents/images →
|
||
viewer / open-with.
|
||
|
||
## 10.6 Layout (Telegram-style, per reference image)
|
||
|
||
**Two layout modes** (decision: user-toggleable, **single-pane default**):
|
||
- **Single-pane (default on phones):** chat full-screen; channel list in a
|
||
swipeable drawer (hamburger / edge swipe).
|
||
- **Two-pane (Telegram-style, like the reference):** persistent left channel
|
||
rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.
|
||
|
||
**Chat screen anatomy (matches reference):**
|
||
- **Header:** back (single-pane), avatar, name + "Bot" subtitle, edit + overflow
|
||
(⋮) menu (thread toggle, channel menu, set cron target, clear).
|
||
- **Message list:** date separators ("7. August"); user bubbles **right**
|
||
(accent color, ✓✓ read receipts); agent bubbles **left** (surface color) with
|
||
optional ReasoningBlock + ToolCards + media + model/token footer
|
||
("Qwen3-… · 11% · ~") + timestamp.
|
||
- **In-app banner** above the composer (e.g. "renamed topic").
|
||
- **Composer:** `Menü` / emoji / auto-grow input / attach / mic-send.
|
||
|
||
**Theme:** dark by default (reference is dark); Material 3; optional dynamic
|
||
color. Accent = user's chosen brand color (default indigo, like the reference).
|
||
|
||
## 10.7 SQLDelight schema (cache)
|
||
|
||
- `channels(chat_id PK, name, kind, parent_chat_id, is_default, last_preview,
|
||
last_ts, unread)`.
|
||
- `messages(id PK, chat_id, thread_id, role, text, reasoning, model, tokens,
|
||
ts, status[pending|sent|read], media_json)`.
|
||
- `media(media_id PK, local_path, kind, mime, size, ts)`.
|
||
- `meta(key PK, value)` — sync cursor, settings, device_id, server url, pinned
|
||
cert fingerprint.
|
||
|
||
## 10.8 Onboarding / Connect screen
|
||
|
||
- First launch → **Connect**: server URL + token (or scan QR). "Test connection"
|
||
does a real `hello` (not just a TCP probe — per hermes desktop guidance, the
|
||
auth leg must be exercised). On success → save (secure storage) → main.
|
||
- States: connecting / connected / reconnecting / degraded / auth-failed — each
|
||
with honest copy and a way out. |