# 07 — Media (upload, download, playback)

Media travels **over HTTP** (`POST /v1/media` for upload,
`GET /v1/media/{id}` for pull; see `19-http-fallback-transport.md` §19.15).
HTTP is the only transport — the WebSocket leg (chunked binary frames) was
removed entirely. The contracts below (kinds, sha256, re-sniffing, delivery
validation) apply to both directions.

## 7.1 Kinds & MIME

`kind` ∈ `image | audio | video | document | voice`.

- `image` — `image/*` (jpg/png/webp/gif/heic).
- `audio` — `audio/*` (mp3/m4a/ogg/…) — music.
- `video` — `video/*` (mp4/webm/mov).
- `document` — anything else (pdf, docx, zip, txt, …).
- `voice` — short voice note (`audio/ogg; codecs=opus` typical).

The app sniffs `kind` from the picked file's MIME; the plugin re-sniffs on
receipt (don't trust the client) using hermes helpers
(`gateway/platforms/base.py` `_sniff_audio_ext`, `_looks_like_image`).

## 7.2 Inbound (app → agent) — `media.upload`

**Flow:**

1. App picks a file (SAF) → reads size + MIME, computes sha256.
2. App `POST /v1/media` with the raw file body; metadata in
   `X-Iris-Media-*` headers (`media_ref`, `kind`, `mime`, `filename`,
   `sha256`).
3. Plugin verifies size ≤ `max_upload_bytes` and sha256, writes to the media
   cache via hermes `cache_*_from_bytes`:
   - image → `cache_image_from_bytes`
   - audio/voice → `cache_audio_from_bytes`
   - video → `cache_video_from_bytes`
   - document → `cache_document_from_bytes`
   → returns a local path.
4. Plugin replies `media.upload.ack {ok, media_ref}` (failures use `error`).
5. The path is attached to the next `message.send` via `media_refs`, becoming
   `MessageEvent.media_urls` + `media_types`
   (`gateway/platforms/base.py:2337`). The agent's vision/audio tools can then
   read the file.

**Limits:** `get_inbound_media_max_bytes()` / `validate_inbound_media_size`
(`base.py:758/779`) enforce the cap; over-limit → 413 +
`error {code:"media_too_large"}`. (The 1 MiB `MAX_BODY_BYTES` cap applies to
JSON *frame* bodies only, not media uploads.)

**Single-shot:** no chunking/resumability — HTTP carries the body; single-user
scale makes a one-shot upload sufficient.

## 7.3 Outbound (agent → app) — `media.offer` / `media.pull`

**Flow:**

1. Agent produces/references media (e.g. generates an image, or replies with a
   `MEDIA:` tag / image URL). hermes base `extract_media` / `extract_images`
   (`base.py:4439/4884`) pull these out and call the adapter's
   `send_image / send_video / send_document / send_voice / send_image_file /
   send_multiple_images`.
2. Adapter stages the file in the media cache, mints a `media_id`, and emits
   `media.offer {media_id, kind, mime, size, filename}` (inside/with the
   `message` frame's `media[]`).
3. App `GET /v1/media/{id}` — the full file body.
4. App writes to its cache dir and hands the path to the player/viewer.

**Security:** `validate_media_delivery_path` (`base.py:1684`) + the media
delivery root/recency/denied-path checks (`base.py:1312-1480`) ensure the plugin
only serves files hermes is allowed to deliver (no arbitrary file read). The
delivery-path check is re-run **at pull time**, not just at offer time.

## 7.4 Live playback (AI-sent music/video)

**Requirement:** play AI-sent music and video **in the app**.

- **Audio (music/voice):** `ExoPlayer` (Media3). Inline player in the bubble
  (play/pause, seek, duration); a persistent **mini-player** for music that
  survives navigation. Voice notes play inline with a waveform.
- **Video:** `ExoPlayer` inline player (play/pause, seek, fullscreen, PiP on
  Android). Streams from the local cache file after `media.pull`.
- **Documents/images:** image viewer (zoom) / open-with for documents (Android
  `Intent.ACTION_VIEW` with a `FileProvider` URI; Desktop opens with the system
  handler).
- **Desktop:** ExoPlayer is Android-only → the `MediaPlayer` expect/actual uses
  a desktop backend (see `11-desktop-app.md`): a `libmpv`/`mpv`-backed surface
  or a WebView fallback for video, and a desktop audio player for music.

## 7.5 Integrity

- Upload: `sha256` (precomputed by the app, sent in `X-Iris-Media-Sha256`)
  is verified by the plugin; mismatch → `media.upload.ack {ok:false}`.
- Pull: the app checks the received size against the offered `size`.
- A failed transfer → retry the whole upload (single-shot, no resume).

## 7.6 App-side storage

- Cache dir: app-specific external cache (`getExternalCacheDir()/media`).
- LRU eviction by size (configurable, default 500 MB) so old media doesn't fill
  the device.
- A `MediaRepository` tracks `{media_id, local_path, kind, size, ts}` in Room so
  bubbles can re-render players after process death.