M4: media upload/download/playback (both directions)
Gateway plugin: - media.upload (chunked binary) -> size/sha256 verify + MIME re-sniff -> cache_*_from_bytes -> media.upload.ack - media.offer / media.pull (chunked) for agent-sent media, delivery-path security re-checked at pull time - send_* overrides mint media_id and emit media.offer - message.send media_refs resolve to cached inbound media - per-send + per-chunk timeouts so a stalled peer can't starve the rest App (Kotlin CMP): - Protocol: media frame types/payloads/builders - GatewayClient: binary session, uploadMedia (chunked + streaming sha256), pullMedia serialized via Mutex so concurrent offers don't interleave - ChatStore/IrisController: MediaItem, attachments, auto-pull on offer - Platform media: SAF picker, ExoPlayer (audio mini-player + video), image loader, FileProvider document open (Android); AWT-free desktop actuals - ChatScreen: attach button + chips, media rendering, keyboard dismiss on send UI polish: - preserve image aspect ratio (no stretching), cap dominant dimension - adjustResize so only chat content squeezes for the keyboard - clear focus (hide keyboard) on send Docs: media.upload.ack in 04-wire-protocol.md + frames.schema.json + 07-media.md; M4 marked complete in 14-milestones.md. Tests: 17-test tests/gateway/test_android.py suite passes.
This commit is contained in:
1 parent
60296b33fe
commit
913ee91024
28 files changed
+2234
-53
No files matched your search
+4
-3
@@ -29,9 +29,10 @@ receipt (don't trust the client) using hermes helpers
|
||||
- image → `cache_image_from_bytes`
|
||||
- audio/voice → `cache_audio_from_bytes`
|
||||
- video → `cache_video_from_bytes`
|
||||
- document → `cache_document_from_bytes`
|
||||
→ returns a local path.
|
||||
6. The path is attached to the next `message.send` via `media_refs`, becoming
|
||||
- document → `cache_document_from_bytes`
|
||||
→ returns a local path.
|
||||
5b. Plugin replies `media.upload.ack {ok, media_ref}` (failures use `error`).
|
||||
6. The path is attached to the next `message.send` via `media_refs`, becoming
|
||||
`MessageEvent.media_urls` + `media_types`
|
||||
(`gateway/platforms/base.py:2337`). The agent's vision/audio tools can then
|
||||
read the file.
|
||||
|
||||
Reference in new issue
Block a user