M0: toolchain, monorepo scaffold, gateway plugin skeleton, CMP app
- gateway-plugin/: android platform plugin (plugin.yaml + adapter.py register(ctx) + no-op AndroidAdapter) + stub modules for M1-M5 - app/: Compose Multiplatform project (shared KMP + androidApp + desktopApp) with Gradle wrapper; builds :androidApp:assembleDebug and :desktopApp:compileKotlin - scripts/guard_hermes_agent.sh + pre-commit hook: fail if hermes-agent/ is staged (read-only reference, never committed) - .gitignore excludes hermes-agent/; docs/ reference library
This commit is contained in:
ARIA committed
commit
59acf66c89
49 files changed
+3950
No files matched your search
+34
@@ -0,0 +1,34 @@
|
|||||||
|
# ── HARD RULE: hermes-agent is a read-only research reference ─────────────
|
||||||
|
# It must NEVER be committed, pushed, or shipped.
|
||||||
|
/hermes-agent/
|
||||||
|
|
||||||
|
# ── Python (gateway-plugin) ───────────────────────────────────────────────
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*.egg-info/
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
|
||||||
|
# ── Kotlin / Gradle (app) ─────────────────────────────────────────────────
|
||||||
|
.gradle/
|
||||||
|
build/
|
||||||
|
local.properties
|
||||||
|
*.iml
|
||||||
|
.idea/
|
||||||
|
captures/
|
||||||
|
.cxx/
|
||||||
|
|
||||||
|
# ── Android / secrets ─────────────────────────────────────────────────────
|
||||||
|
app/androidApp/src/main/res/values/secrets.xml
|
||||||
|
google-services.json
|
||||||
|
*.jks
|
||||||
|
keystore.jks
|
||||||
|
*.keystore
|
||||||
|
|
||||||
|
# ── OS / misc ─────────────────────────────────────────────────────────────
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
*.log
|
||||||
|
*.tmp
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Iris × Hermes
|
||||||
|
|
||||||
|
A **native Android + Desktop** experience for [hermes-agent](https://github.com/NousResearch/hermes-agent), connected through a **gateway platform plugin**.
|
||||||
|
|
||||||
|
> ## ⚠️ HARD RULES
|
||||||
|
> 1. **`hermes-agent/` is a read-only research reference. It is git-ignored and
|
||||||
|
> must NEVER be committed, pushed, or shipped.** We only *install* our plugin
|
||||||
|
> into a live hermes home (`~/.hermes/plugins/android`); we never modify hermes
|
||||||
|
> core.
|
||||||
|
> 2. **ADB is installed and a device is connected** (`a5ca2a4b`, Xiaomi MIX 2S,
|
||||||
|
> Android 10 / API 29). Use it to install/launch/debug the app on-device.
|
||||||
|
|
||||||
|
## Start here
|
||||||
|
|
||||||
|
📚 **The full implementation reference library is in [`docs/`](docs/README.md).**
|
||||||
|
Read `docs/00-overview.md` first, then follow the numbered docs.
|
||||||
|
|
||||||
|
## The three deliverables (this monorepo)
|
||||||
|
|
||||||
|
| Path | What |
|
||||||
|
|---|---|
|
||||||
|
| `gateway-plugin/` | Python hermes **platform plugin** (`android`). Runs in the `hermes gateway` process; opens a WebSocket server the apps connect to. Zero new Python deps, zero hermes-core changes. |
|
||||||
|
| `app/androidApp` | Native **Kotlin + Jetpack Compose** client. |
|
||||||
|
| `app/desktopApp` | **Kotlin + Compose Multiplatform** client — the Android app, tweaked for a big screen (shares `app/shared`). |
|
||||||
|
|
||||||
|
The Android and Desktop clients live in **one Compose Multiplatform Gradle
|
||||||
|
project** (`app/`) with a shared KMP module (`app/shared`).
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- **Phase:** Planning complete → ready to implement (Milestone M0).
|
||||||
|
- **Milestones:** see [`docs/14-milestones.md`](docs/14-milestones.md).
|
||||||
|
- **Locked decisions:** see [`docs/16-open-questions.md`](docs/16-open-questions.md).
|
||||||
|
|
||||||
|
## Quick orientation
|
||||||
|
|
||||||
|
- Architecture + rationale → [`docs/01-architecture.md`](docs/01-architecture.md)
|
||||||
|
- Wire protocol → [`docs/04-wire-protocol.md`](docs/04-wire-protocol.md) (+ [`docs/protocol/frames.schema.json`](docs/protocol/frames.schema.json))
|
||||||
|
- Gateway plugin design → [`docs/03-gateway-plugin.md`](docs/03-gateway-plugin.md)
|
||||||
|
- hermes source cheat-sheet → [`docs/15-hermes-reference.md`](docs/15-hermes-reference.md)
|
||||||
|
- Toolchain setup → [`docs/12-toolchain.md`](docs/12-toolchain.md)
|
||||||
|
</file>
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 376 KiB |
@@ -0,0 +1,42 @@
|
|||||||
|
plugins {
|
||||||
|
id("com.android.application")
|
||||||
|
kotlin("android")
|
||||||
|
id("org.jetbrains.kotlin.plugin.compose")
|
||||||
|
}
|
||||||
|
|
||||||
|
android {
|
||||||
|
namespace = "dev.iris.app"
|
||||||
|
compileSdk = 34
|
||||||
|
|
||||||
|
defaultConfig {
|
||||||
|
applicationId = "dev.iris.app"
|
||||||
|
minSdk = 29
|
||||||
|
targetSdk = 34
|
||||||
|
versionCode = 1
|
||||||
|
versionName = "0.1.0"
|
||||||
|
}
|
||||||
|
|
||||||
|
buildTypes {
|
||||||
|
release {
|
||||||
|
isMinifyEnabled = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
compileOptions {
|
||||||
|
sourceCompatibility = JavaVersion.VERSION_17
|
||||||
|
targetCompatibility = JavaVersion.VERSION_17
|
||||||
|
}
|
||||||
|
|
||||||
|
kotlinOptions {
|
||||||
|
jvmTarget = "17"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
dependencies {
|
||||||
|
implementation(project(":shared"))
|
||||||
|
implementation(platform("androidx.compose:compose-bom:2024.09.02"))
|
||||||
|
implementation("androidx.compose.material3:material3")
|
||||||
|
implementation("androidx.compose.ui:ui")
|
||||||
|
implementation("androidx.activity:activity-compose:1.9.2")
|
||||||
|
implementation("androidx.core:core-splashscreen:1.0.1")
|
||||||
|
}
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
<?xml version="1.0" encoding="utf-8"?>
|
||||||
|
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
|
||||||
|
|
||||||
|
<uses-permission android:name="android.permission.INTERNET" />
|
||||||
|
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
|
||||||
|
|
||||||
|
<application
|
||||||
|
android:label="Iris"
|
||||||
|
android:allowBackup="true"
|
||||||
|
android:theme="@android:style/Theme.Material.NoActionBar">
|
||||||
|
<activity
|
||||||
|
android:name=".MainActivity"
|
||||||
|
android:exported="true"
|
||||||
|
android:configChanges="orientation|screenSize|screenLayout|keyboardHidden">
|
||||||
|
<intent-filter>
|
||||||
|
<action android:name="android.intent.action.MAIN" />
|
||||||
|
<category android:name="android.intent.category.LAUNCHER" />
|
||||||
|
</intent-filter>
|
||||||
|
</activity>
|
||||||
|
</application>
|
||||||
|
|
||||||
|
</manifest>
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
package dev.iris.app
|
||||||
|
|
||||||
|
import android.os.Bundle
|
||||||
|
import androidx.activity.ComponentActivity
|
||||||
|
import androidx.activity.compose.setContent
|
||||||
|
import iris.IrisApp
|
||||||
|
|
||||||
|
class MainActivity : ComponentActivity() {
|
||||||
|
override fun onCreate(savedInstanceState: Bundle?) {
|
||||||
|
super.onCreate(savedInstanceState)
|
||||||
|
setContent {
|
||||||
|
IrisApp()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
plugins {
|
||||||
|
kotlin("multiplatform") version "2.1.0" apply false
|
||||||
|
kotlin("android") version "2.1.0" apply false
|
||||||
|
kotlin("jvm") version "2.1.0" apply false
|
||||||
|
id("com.android.application") version "8.7.3" apply false
|
||||||
|
id("com.android.library") version "8.7.3" apply false
|
||||||
|
id("org.jetbrains.compose") version "1.7.3" apply false
|
||||||
|
id("org.jetbrains.kotlin.plugin.compose") version "2.1.0" apply false
|
||||||
|
}
|
||||||
|
|
||||||
|
tasks.register("clean") {
|
||||||
|
delete(rootProject.layout.buildDirectory)
|
||||||
|
}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
import org.gradle.internal.os.OperatingSystem
|
||||||
|
|
||||||
|
plugins {
|
||||||
|
kotlin("jvm")
|
||||||
|
id("org.jetbrains.compose")
|
||||||
|
id("org.jetbrains.kotlin.plugin.compose")
|
||||||
|
}
|
||||||
|
|
||||||
|
val composeVersion = "1.7.3"
|
||||||
|
val os = OperatingSystem.current()
|
||||||
|
val arch = System.getProperty("os.arch") ?: "amd64"
|
||||||
|
val desktopTarget = when {
|
||||||
|
os.isMacOsX -> if (arch == "aarch64") "macos-arm64" else "macos-x64"
|
||||||
|
os.isWindows -> "windows-x64"
|
||||||
|
else -> if (arch == "aarch64") "linux-arm64" else "linux-x64"
|
||||||
|
}
|
||||||
|
|
||||||
|
dependencies {
|
||||||
|
implementation(project(":shared"))
|
||||||
|
implementation("org.jetbrains.compose.desktop:desktop-jvm-$desktopTarget:$composeVersion")
|
||||||
|
}
|
||||||
|
|
||||||
|
compose.desktop {
|
||||||
|
application {
|
||||||
|
mainClass = "iris.desktop.MainKt"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
package iris.desktop
|
||||||
|
|
||||||
|
import androidx.compose.ui.window.Window
|
||||||
|
import androidx.compose.ui.window.application
|
||||||
|
import iris.IrisApp
|
||||||
|
|
||||||
|
fun main() = application {
|
||||||
|
Window(onCloseRequest = ::exitApplication, title = "Iris") {
|
||||||
|
IrisApp()
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
org.gradle.jvmargs=-Xmx4g -Dfile.encoding=UTF-8
|
||||||
|
org.gradle.caching=true
|
||||||
|
org.gradle.configuration-cache=true
|
||||||
|
|
||||||
|
android.useAndroidX=true
|
||||||
|
android.nonTransitiveRClass=true
|
||||||
|
|
||||||
|
kotlin.code.style=official
|
||||||
|
|
||||||
|
# Compose Multiplatform
|
||||||
|
org.jetbrains.compose.experimental.jscanvas.enabled=false
|
||||||
BIN
Binary file not shown.
@@ -0,0 +1,7 @@
|
|||||||
|
distributionBase=GRADLE_USER_HOME
|
||||||
|
distributionPath=wrapper/dists
|
||||||
|
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.2-bin.zip
|
||||||
|
networkTimeout=10000
|
||||||
|
validateDistributionUrl=true
|
||||||
|
zipStoreBase=GRADLE_USER_HOME
|
||||||
|
zipStorePath=wrapper/dists
|
||||||
+252
@@ -0,0 +1,252 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
#
|
||||||
|
# Copyright © 2015-2021 the original authors.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
#
|
||||||
|
# SPDX-License-Identifier: Apache-2.0
|
||||||
|
#
|
||||||
|
|
||||||
|
##############################################################################
|
||||||
|
#
|
||||||
|
# Gradle start up script for POSIX generated by Gradle.
|
||||||
|
#
|
||||||
|
# Important for running:
|
||||||
|
#
|
||||||
|
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
|
||||||
|
# noncompliant, but you have some other compliant shell such as ksh or
|
||||||
|
# bash, then to run this script, type that shell name before the whole
|
||||||
|
# command line, like:
|
||||||
|
#
|
||||||
|
# ksh Gradle
|
||||||
|
#
|
||||||
|
# Busybox and similar reduced shells will NOT work, because this script
|
||||||
|
# requires all of these POSIX shell features:
|
||||||
|
# * functions;
|
||||||
|
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
|
||||||
|
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
|
||||||
|
# * compound commands having a testable exit status, especially «case»;
|
||||||
|
# * various built-in commands including «command», «set», and «ulimit».
|
||||||
|
#
|
||||||
|
# Important for patching:
|
||||||
|
#
|
||||||
|
# (2) This script targets any POSIX shell, so it avoids extensions provided
|
||||||
|
# by Bash, Ksh, etc; in particular arrays are avoided.
|
||||||
|
#
|
||||||
|
# The "traditional" practice of packing multiple parameters into a
|
||||||
|
# space-separated string is a well documented source of bugs and security
|
||||||
|
# problems, so this is (mostly) avoided, by progressively accumulating
|
||||||
|
# options in "$@", and eventually passing that to Java.
|
||||||
|
#
|
||||||
|
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
|
||||||
|
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
|
||||||
|
# see the in-line comments for details.
|
||||||
|
#
|
||||||
|
# There are tweaks for specific operating systems such as AIX, CygWin,
|
||||||
|
# Darwin, MinGW, and NonStop.
|
||||||
|
#
|
||||||
|
# (3) This script is generated from the Groovy template
|
||||||
|
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
|
||||||
|
# within the Gradle project.
|
||||||
|
#
|
||||||
|
# You can find Gradle at https://github.com/gradle/gradle/.
|
||||||
|
#
|
||||||
|
##############################################################################
|
||||||
|
|
||||||
|
# Attempt to set APP_HOME
|
||||||
|
|
||||||
|
# Resolve links: $0 may be a link
|
||||||
|
app_path=$0
|
||||||
|
|
||||||
|
# Need this for daisy-chained symlinks.
|
||||||
|
while
|
||||||
|
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
|
||||||
|
[ -h "$app_path" ]
|
||||||
|
do
|
||||||
|
ls=$( ls -ld "$app_path" )
|
||||||
|
link=${ls#*' -> '}
|
||||||
|
case $link in #(
|
||||||
|
/*) app_path=$link ;; #(
|
||||||
|
*) app_path=$APP_HOME$link ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# This is normally unused
|
||||||
|
# shellcheck disable=SC2034
|
||||||
|
APP_BASE_NAME=${0##*/}
|
||||||
|
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
|
||||||
|
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s
|
||||||
|
' "$PWD" ) || exit
|
||||||
|
|
||||||
|
# Use the maximum available, or set MAX_FD != -1 to use that value.
|
||||||
|
MAX_FD=maximum
|
||||||
|
|
||||||
|
warn () {
|
||||||
|
echo "$*"
|
||||||
|
} >&2
|
||||||
|
|
||||||
|
die () {
|
||||||
|
echo
|
||||||
|
echo "$*"
|
||||||
|
echo
|
||||||
|
exit 1
|
||||||
|
} >&2
|
||||||
|
|
||||||
|
# OS specific support (must be 'true' or 'false').
|
||||||
|
cygwin=false
|
||||||
|
msys=false
|
||||||
|
darwin=false
|
||||||
|
nonstop=false
|
||||||
|
case "$( uname )" in #(
|
||||||
|
CYGWIN* ) cygwin=true ;; #(
|
||||||
|
Darwin* ) darwin=true ;; #(
|
||||||
|
MSYS* | MINGW* ) msys=true ;; #(
|
||||||
|
NONSTOP* ) nonstop=true ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
|
||||||
|
|
||||||
|
|
||||||
|
# Determine the Java command to use to start the JVM.
|
||||||
|
if [ -n "$JAVA_HOME" ] ; then
|
||||||
|
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
|
||||||
|
# IBM's JDK on AIX uses strange locations for the executables
|
||||||
|
JAVACMD=$JAVA_HOME/jre/sh/java
|
||||||
|
else
|
||||||
|
JAVACMD=$JAVA_HOME/bin/java
|
||||||
|
fi
|
||||||
|
if [ ! -x "$JAVACMD" ] ; then
|
||||||
|
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
|
||||||
|
|
||||||
|
Please set the JAVA_HOME variable in your environment to match the
|
||||||
|
location of your Java installation."
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
JAVACMD=java
|
||||||
|
if ! command -v java >/dev/null 2>&1
|
||||||
|
then
|
||||||
|
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
|
||||||
|
|
||||||
|
Please set the JAVA_HOME variable in your environment to match the
|
||||||
|
location of your Java installation."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Increase the maximum file descriptors if we can.
|
||||||
|
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
|
||||||
|
case $MAX_FD in #(
|
||||||
|
max*)
|
||||||
|
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
|
||||||
|
# shellcheck disable=SC2039,SC3045
|
||||||
|
MAX_FD=$( ulimit -H -n ) ||
|
||||||
|
warn "Could not query maximum file descriptor limit"
|
||||||
|
esac
|
||||||
|
case $MAX_FD in #(
|
||||||
|
'' | soft) :;; #(
|
||||||
|
*)
|
||||||
|
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
|
||||||
|
# shellcheck disable=SC2039,SC3045
|
||||||
|
ulimit -n "$MAX_FD" ||
|
||||||
|
warn "Could not set maximum file descriptor limit to $MAX_FD"
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Collect all arguments for the java command, stacking in reverse order:
|
||||||
|
# * args from the command line
|
||||||
|
# * the main class name
|
||||||
|
# * -classpath
|
||||||
|
# * -D...appname settings
|
||||||
|
# * --module-path (only if needed)
|
||||||
|
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
|
||||||
|
|
||||||
|
# For Cygwin or MSYS, switch paths to Windows format before running java
|
||||||
|
if "$cygwin" || "$msys" ; then
|
||||||
|
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
|
||||||
|
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
|
||||||
|
|
||||||
|
JAVACMD=$( cygpath --unix "$JAVACMD" )
|
||||||
|
|
||||||
|
# Now convert the arguments - kludge to limit ourselves to /bin/sh
|
||||||
|
for arg do
|
||||||
|
if
|
||||||
|
case $arg in #(
|
||||||
|
-*) false ;; # don't mess with options #(
|
||||||
|
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
|
||||||
|
[ -e "$t" ] ;; #(
|
||||||
|
*) false ;;
|
||||||
|
esac
|
||||||
|
then
|
||||||
|
arg=$( cygpath --path --ignore --mixed "$arg" )
|
||||||
|
fi
|
||||||
|
# Roll the args list around exactly as many times as the number of
|
||||||
|
# args, so each arg winds up back in the position where it started, but
|
||||||
|
# possibly modified.
|
||||||
|
#
|
||||||
|
# NB: a `for` loop captures its iteration list before it begins, so
|
||||||
|
# changing the positional parameters here affects neither the number of
|
||||||
|
# iterations, nor the values presented in `arg`.
|
||||||
|
shift # remove old arg
|
||||||
|
set -- "$@" "$arg" # push replacement arg
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
|
||||||
|
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
||||||
|
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
|
||||||
|
|
||||||
|
# Collect all arguments for the java command:
|
||||||
|
# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
|
||||||
|
# and any embedded shellness will be escaped.
|
||||||
|
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
|
||||||
|
# treated as '${Hostname}' itself on the command line.
|
||||||
|
|
||||||
|
set -- \
|
||||||
|
"-Dorg.gradle.appname=$APP_BASE_NAME" \
|
||||||
|
-classpath "$CLASSPATH" \
|
||||||
|
org.gradle.wrapper.GradleWrapperMain \
|
||||||
|
"$@"
|
||||||
|
|
||||||
|
# Stop when "xargs" is not available.
|
||||||
|
if ! command -v xargs >/dev/null 2>&1
|
||||||
|
then
|
||||||
|
die "xargs is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Use "xargs" to parse quoted args.
|
||||||
|
#
|
||||||
|
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
|
||||||
|
#
|
||||||
|
# In Bash we could simply go:
|
||||||
|
#
|
||||||
|
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
|
||||||
|
# set -- "${ARGS[@]}" "$@"
|
||||||
|
#
|
||||||
|
# but POSIX shell has neither arrays nor command substitution, so instead we
|
||||||
|
# post-process each arg (as a line of input to sed) to backslash-escape any
|
||||||
|
# character that might be a shell metacharacter, then use eval to reverse
|
||||||
|
# that process (while maintaining the separation between arguments), and wrap
|
||||||
|
# the whole thing up as a single "set" statement.
|
||||||
|
#
|
||||||
|
# This will of course break if any of these variables contains a newline or
|
||||||
|
# an unmatched quote.
|
||||||
|
#
|
||||||
|
|
||||||
|
eval "set -- $(
|
||||||
|
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
|
||||||
|
xargs -n1 |
|
||||||
|
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
|
||||||
|
tr '\n' ' '
|
||||||
|
)" '"$@"'
|
||||||
|
|
||||||
|
exec "$JAVACMD" "$@"
|
||||||
Vendored
+94
@@ -0,0 +1,94 @@
|
|||||||
|
@rem
|
||||||
|
@rem Copyright 2015 the original author or authors.
|
||||||
|
@rem
|
||||||
|
@rem Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
@rem you may not use this file except in compliance with the License.
|
||||||
|
@rem You may obtain a copy of the License at
|
||||||
|
@rem
|
||||||
|
@rem https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
@rem
|
||||||
|
@rem Unless required by applicable law or agreed to in writing, software
|
||||||
|
@rem distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
@rem See the License for the specific language governing permissions and
|
||||||
|
@rem limitations under the License.
|
||||||
|
@rem
|
||||||
|
@rem SPDX-License-Identifier: Apache-2.0
|
||||||
|
@rem
|
||||||
|
|
||||||
|
@if "%DEBUG%"=="" @echo off
|
||||||
|
@rem ##########################################################################
|
||||||
|
@rem
|
||||||
|
@rem Gradle startup script for Windows
|
||||||
|
@rem
|
||||||
|
@rem ##########################################################################
|
||||||
|
|
||||||
|
@rem Set local scope for the variables with windows NT shell
|
||||||
|
if "%OS%"=="Windows_NT" setlocal
|
||||||
|
|
||||||
|
set DIRNAME=%~dp0
|
||||||
|
if "%DIRNAME%"=="" set DIRNAME=.
|
||||||
|
@rem This is normally unused
|
||||||
|
set APP_BASE_NAME=%~n0
|
||||||
|
set APP_HOME=%DIRNAME%
|
||||||
|
|
||||||
|
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
|
||||||
|
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
|
||||||
|
|
||||||
|
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
||||||
|
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
|
||||||
|
|
||||||
|
@rem Find java.exe
|
||||||
|
if defined JAVA_HOME goto findJavaFromJavaHome
|
||||||
|
|
||||||
|
set JAVA_EXE=java.exe
|
||||||
|
%JAVA_EXE% -version >NUL 2>&1
|
||||||
|
if %ERRORLEVEL% equ 0 goto execute
|
||||||
|
|
||||||
|
echo. 1>&2
|
||||||
|
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
|
||||||
|
echo. 1>&2
|
||||||
|
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
|
||||||
|
echo location of your Java installation. 1>&2
|
||||||
|
|
||||||
|
goto fail
|
||||||
|
|
||||||
|
:findJavaFromJavaHome
|
||||||
|
set JAVA_HOME=%JAVA_HOME:"=%
|
||||||
|
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
|
||||||
|
|
||||||
|
if exist "%JAVA_EXE%" goto execute
|
||||||
|
|
||||||
|
echo. 1>&2
|
||||||
|
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
|
||||||
|
echo. 1>&2
|
||||||
|
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
|
||||||
|
echo location of your Java installation. 1>&2
|
||||||
|
|
||||||
|
goto fail
|
||||||
|
|
||||||
|
:execute
|
||||||
|
@rem Setup the command line
|
||||||
|
|
||||||
|
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
|
||||||
|
|
||||||
|
|
||||||
|
@rem Execute Gradle
|
||||||
|
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
|
||||||
|
|
||||||
|
:end
|
||||||
|
@rem End local scope for the variables with windows NT shell
|
||||||
|
if %ERRORLEVEL% equ 0 goto mainEnd
|
||||||
|
|
||||||
|
:fail
|
||||||
|
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
|
||||||
|
rem the _cmd.exe /c_ return code!
|
||||||
|
set EXIT_CODE=%ERRORLEVEL%
|
||||||
|
if %EXIT_CODE% equ 0 set EXIT_CODE=1
|
||||||
|
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
|
||||||
|
exit /b %EXIT_CODE%
|
||||||
|
|
||||||
|
:mainEnd
|
||||||
|
if "%OS%"=="Windows_NT" endlocal
|
||||||
|
|
||||||
|
:omega
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
pluginManagement {
|
||||||
|
repositories {
|
||||||
|
mavenCentral()
|
||||||
|
gradlePluginPortal()
|
||||||
|
google()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
dependencyResolutionManagement {
|
||||||
|
repositories {
|
||||||
|
mavenCentral()
|
||||||
|
google()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
rootProject.name = "iris"
|
||||||
|
|
||||||
|
include(":shared")
|
||||||
|
include(":androidApp")
|
||||||
|
include(":desktopApp")
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
plugins {
|
||||||
|
kotlin("multiplatform")
|
||||||
|
id("com.android.library")
|
||||||
|
id("org.jetbrains.compose")
|
||||||
|
id("org.jetbrains.kotlin.plugin.compose")
|
||||||
|
}
|
||||||
|
|
||||||
|
val composeVersion = "1.7.3"
|
||||||
|
|
||||||
|
kotlin {
|
||||||
|
androidTarget()
|
||||||
|
jvm("desktop")
|
||||||
|
|
||||||
|
sourceSets {
|
||||||
|
commonMain.dependencies {
|
||||||
|
implementation("org.jetbrains.compose.runtime:runtime:$composeVersion")
|
||||||
|
implementation("org.jetbrains.compose.foundation:foundation:$composeVersion")
|
||||||
|
implementation("org.jetbrains.compose.material3:material3:$composeVersion")
|
||||||
|
implementation("org.jetbrains.compose.ui:ui:$composeVersion")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
android {
|
||||||
|
namespace = "iris.shared"
|
||||||
|
compileSdk = 34
|
||||||
|
defaultConfig {
|
||||||
|
minSdk = 29
|
||||||
|
}
|
||||||
|
compileOptions {
|
||||||
|
sourceCompatibility = JavaVersion.VERSION_17
|
||||||
|
targetCompatibility = JavaVersion.VERSION_17
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
package iris
|
||||||
|
|
||||||
|
import androidx.compose.foundation.background
|
||||||
|
import androidx.compose.foundation.layout.Box
|
||||||
|
import androidx.compose.foundation.layout.Column
|
||||||
|
import androidx.compose.foundation.layout.Spacer
|
||||||
|
import androidx.compose.foundation.layout.fillMaxSize
|
||||||
|
import androidx.compose.foundation.layout.height
|
||||||
|
import androidx.compose.foundation.layout.size
|
||||||
|
import androidx.compose.foundation.shape.CircleShape
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.material3.Surface
|
||||||
|
import androidx.compose.material3.Text
|
||||||
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.ui.Alignment
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.graphics.Color
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
import androidx.compose.ui.unit.sp
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Root composable shared by the Android and Desktop shells.
|
||||||
|
*
|
||||||
|
* M0: a visible placeholder to confirm the Compose stack renders. M1+
|
||||||
|
* replaces this with the Connect screen, chat UI, and the rest of the app
|
||||||
|
* (see docs/10-android-app.md).
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun IrisApp() {
|
||||||
|
MaterialTheme {
|
||||||
|
Surface(modifier = Modifier.fillMaxSize()) {
|
||||||
|
Box(
|
||||||
|
modifier = Modifier
|
||||||
|
.fillMaxSize()
|
||||||
|
.background(Color(0xFF1B1E28)),
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Column(horizontalAlignment = Alignment.CenterHorizontally) {
|
||||||
|
Box(
|
||||||
|
modifier = Modifier
|
||||||
|
.size(72.dp)
|
||||||
|
.background(Color(0xFF4F7CFF), CircleShape),
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Text(text = "I", color = Color.White, fontSize = 40.sp)
|
||||||
|
}
|
||||||
|
Spacer(modifier = Modifier.height(16.dp))
|
||||||
|
Text(text = "Iris × Hermes", color = Color.White, fontSize = 22.sp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
# 00 — Overview
|
||||||
|
|
||||||
|
## Vision
|
||||||
|
|
||||||
|
A **native, Telegram-quality chat experience** for a personal hermes-agent:
|
||||||
|
install an app on your phone (and a desktop app on your PC), pair it to your
|
||||||
|
running `hermes gateway`, and talk to your agent with streaming replies,
|
||||||
|
visible reasoning, structured tool activity, channels/threads, media, search,
|
||||||
|
and push notifications — with cron jobs able to post into any channel you
|
||||||
|
create.
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- **Native feel.** Real Android app (Kotlin/Compose), not a WebView. Desktop
|
||||||
|
app that is the same app, resized for a big screen.
|
||||||
|
- **First-class gateway citizen.** The app is a hermes *messaging platform*, so
|
||||||
|
everything the gateway already does "just works": slash commands, cron
|
||||||
|
delivery, `send_message` routing, coexistence with Telegram/Discord/etc.
|
||||||
|
- **Full agent transparency.** Streaming text, reasoning shown *before* the
|
||||||
|
answer, structured tool events (the app chooses how much to show), and
|
||||||
|
intermediate assistant beats.
|
||||||
|
- **Organized by default.** A default chat with optional threads, plus
|
||||||
|
user-created channels that cron jobs can target.
|
||||||
|
- **Reachable anywhere.** Live over a WebSocket; background push via FCM
|
||||||
|
(primary) or ntfy (fallback).
|
||||||
|
|
||||||
|
## In scope (v1)
|
||||||
|
|
||||||
|
Everything in the feature checklist below.
|
||||||
|
|
||||||
|
## Out of scope / stretch (v1)
|
||||||
|
|
||||||
|
- **Standalone-cron delivery while the gateway process is fully down** —
|
||||||
|
best-effort FCM/ntfy only (the outbox is served by the running gateway).
|
||||||
|
- **Multi-user / group chat** — this is a *personal* 1-user agent.
|
||||||
|
- **End-to-end encryption** — transport security (WSS) only.
|
||||||
|
- **iOS** — Android + Desktop only (the protocol is transport-agnostic, so an
|
||||||
|
iOS client is a future port, not a v1 goal).
|
||||||
|
|
||||||
|
## Feature checklist → where it's handled
|
||||||
|
|
||||||
|
| Requirement | Gateway plugin | App |
|
||||||
|
|---|---|---|
|
||||||
|
| Input box, auto-grow (max height) | — | Compose `TextField` + bounded `heightIn` |
|
||||||
|
| Menu button → all slash commands | Dispatches `/…`; serves command catalog | Bottom-sheet menu + `/` autocomplete |
|
||||||
|
| Tool output (app decides how much) | Emits **structured** tool events | App setting: everything / truncated / nothing |
|
||||||
|
| Reasoning shown before message | Captures + splits reasoning | Collapsible "Reasoning" block above message |
|
||||||
|
| Intermediate messages | Forwards `Commentary` events | Distinct dimmed bubble |
|
||||||
|
| Threading + channels; default chat; user channels for cron | `chat_id`/`thread_id` model; cron `deliver=android:<chat>[:<thread>]` | Channel list, thread toggle, "new channel" |
|
||||||
|
| Search ("everywhere" / "this chat/channel") | FTS5 session search bridge | Search UI + scope toggle |
|
||||||
|
| Attach media (music/video/images/docs) | Inbound cache; outbound `send_*` | Pickers + chunked upload + preview |
|
||||||
|
| Push notifications | FCM (primary) / ntfy (fallback) | FCM token / ntfy topic + notification service |
|
||||||
|
| Live playback of AI-sent music/video | Serves media bytes over WS | ExoPlayer inline player |
|
||||||
|
|
||||||
|
## Locked decisions (from planning)
|
||||||
|
|
||||||
|
| Decision | Choice |
|
||||||
|
|---|---|
|
||||||
|
| Desktop app tech | **Compose Multiplatform** (shares Android code; "tweaked" for big screen) |
|
||||||
|
| Push backend | **Both** — FCM primary, ntfy fallback (`ANDROID_PUSH_BACKEND`) |
|
||||||
|
| Media transport | **Over the WebSocket** (chunked binary frames; no extra Python deps) |
|
||||||
|
| Phone default layout | **User-toggleable, single-pane default** (auto two-pane on large screens) |
|
||||||
|
|
||||||
|
## Disclaimers (hard rules)
|
||||||
|
|
||||||
|
1. **`hermes-agent/` is a read-only research reference.** It lives next to this
|
||||||
|
folder for study only. It is **git-ignored** and must **never** be committed,
|
||||||
|
pushed, or included in any artifact. Our plugin is *installed* into a live
|
||||||
|
hermes home (`~/.hermes/plugins/android`); we never edit hermes core files.
|
||||||
|
2. **ADB is available and a device is connected** (`a5ca2a4b`, Xiaomi MIX 2S,
|
||||||
|
Android 10 / API 29). Use `adb install` / `adb logcat` / `adb shell am start`
|
||||||
|
to install, launch, and debug the app on-device throughout the build.
|
||||||
|
|
||||||
|
## Verified environment state (2026-08-19)
|
||||||
|
|
||||||
|
| Item | State |
|
||||||
|
|---|---|
|
||||||
|
| OS | CachyOS (Arch-based), `pacman` present |
|
||||||
|
| JDK | **Not installed** → Milestone M0 (`pacman -S jdk17-openjdk`) |
|
||||||
|
| Android SDK | **Not installed** → M0 (cmdline-tools + sdkmanager) |
|
||||||
|
| Gradle | Via project wrapper (`gradlew`), no system install |
|
||||||
|
| ADB | Installed; device `a5ca2a4b` (MIX 2S, API 29) connected |
|
||||||
|
| Python | 3.14.7; `uv` 0.12.3 present |
|
||||||
|
| hermes venv | **Not created** → M0 (`cd hermes-agent && uv sync`) |
|
||||||
|
| hermes core deps | `websockets==15.0.1` and `httpx` are **core** deps → plugin needs **zero new Python deps** |
|
||||||
|
| Disk / RAM | 522 GB free / 62 GB RAM — ample |
|
||||||
|
|
||||||
|
## Naming
|
||||||
|
|
||||||
|
- Product/effort name: **Iris × Hermes** (folder `iris_x_hermes`).
|
||||||
|
- hermes platform name: **`android`** (the plugin registers `Platform("android")`).
|
||||||
|
- WS default port: **8790** (configurable).
|
||||||
|
- Default chat id: **`android:default`** (the home channel).
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
# 01 — Architecture
|
||||||
|
|
||||||
|
## System diagram
|
||||||
|
|
||||||
|
```
|
||||||
|
┌────────────────────────────── User's machine (home server / PC) ─────────────────────────────┐
|
||||||
|
│ │
|
||||||
|
│ hermes gateway (ONE process) │
|
||||||
|
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
|
||||||
|
│ │ Agent core (run_agent.py) ── sessions (SQLite + FTS5) ── cron scheduler │ │
|
||||||
|
│ │ │ │ │
|
||||||
|
│ │ ▼ legacy stream callbacks (delta / tool-progress / commentary) │ │
|
||||||
|
│ │ ┌──────────────────────────────┐ ┌────────────────────────────────┐ │ │
|
||||||
|
│ │ │ android PLATFORM PLUGIN │◄───────►│ WebSocket SERVER (websockets) │ │ │
|
||||||
|
│ │ │ AndroidAdapter │ JSON │ ws://host:8790/ws │ │ │
|
||||||
|
│ │ │ (BasePlatformAdapter) │ frames │ + media + FCM-token + pairing │ │ │
|
||||||
|
│ │ │ • send / edit / stream │ └───────────────┬────────────────┘ │ │
|
||||||
|
│ │ │ • media cache │ │ WSS │ │
|
||||||
|
│ │ │ • outbox (SQLite) │ │ │ │
|
||||||
|
│ │ │ • push (FCM / ntfy) │─────────────────────────┼──────────┐ │ │
|
||||||
|
│ │ └──────────────────────────────┘ │ │ │ │
|
||||||
|
│ └───────────────────────────────────────────────────────────┼──────────┼───────────┘ │
|
||||||
|
└───────────────────────────────────────────────────────────────┼──────────┼───────────────────────┘
|
||||||
|
│ │ push (FCM HTTP v1 / ntfy)
|
||||||
|
┌──────────────────┼──────────▼─────────┐
|
||||||
|
│ │ Google FCM cloud │
|
||||||
|
▼ │ │ │
|
||||||
|
┌────────────────────────┐ │ ▼ │
|
||||||
|
│ ANDROID APP │◄──┴── (wake) ┌──────────┐
|
||||||
|
│ (Kotlin / Compose) │ WSS │ PHONE │
|
||||||
|
│ • WS client (OkHttp) │◄───────────►│ MIX 2S │
|
||||||
|
│ • ExoPlayer │ │ (API 29) │
|
||||||
|
│ • FCM service │ └──────────┘
|
||||||
|
└────────────────────────┘
|
||||||
|
DESKTOP APP (Compose Multiplatform)
|
||||||
|
• same shared code, WSS to same server
|
||||||
|
• tray + OS notifications (no FCM), big-screen two-pane
|
||||||
|
```
|
||||||
|
|
||||||
|
## Process model
|
||||||
|
|
||||||
|
- **One `hermes gateway` process** hosts the agent core, the session store, the
|
||||||
|
cron scheduler, *and* our `android` platform plugin. The plugin's WebSocket
|
||||||
|
server runs on the gateway's asyncio loop (started in `AndroidAdapter.connect()`).
|
||||||
|
- **The app is a client.** It *initiates* the WS connection to the gateway
|
||||||
|
(outbound), so no inbound port is needed on the phone. For LAN/remote access
|
||||||
|
the user points the app at the gateway's LAN IP / Tailscale name / a WSS
|
||||||
|
tunnel (see `09-pairing-security.md`).
|
||||||
|
- **Push is the only inbound path to a sleeping phone**, and it goes through a
|
||||||
|
cloud relay (FCM or ntfy), not a direct connection.
|
||||||
|
|
||||||
|
## Why the *messaging gateway* is the connection point (not `tui_gateway`)
|
||||||
|
|
||||||
|
hermes has two "gateways": the **messaging gateway** (`hermes gateway`, which
|
||||||
|
serves Telegram/Discord/… and cron) and the **`tui_gateway`** JSON-RPC backend
|
||||||
|
(used by the TUI and the existing Electron desktop app). We deliberately use the
|
||||||
|
**messaging gateway** because:
|
||||||
|
|
||||||
|
1. **Cron delivery is native.** Cron jobs resolve `deliver=android:<chat>[:<thread>]`
|
||||||
|
through the platform registry and call our adapter's `send()`. No bridging.
|
||||||
|
2. **`send_message` tool routing** works out of the box (plugin
|
||||||
|
`parse_target_ref_fn`).
|
||||||
|
3. **Slash commands** are dispatched by the gateway's command pipeline — the app
|
||||||
|
just sends `/cmd args` as a message.
|
||||||
|
4. **Coexistence.** The same agent is reachable via Telegram *and* the app at
|
||||||
|
once; sessions/channels are shared.
|
||||||
|
|
||||||
|
The `tui_gateway` WS protocol is *not* reused; we define a clean, purpose-built
|
||||||
|
protocol (`04-wire-protocol.md`) that borrows familiar names (`message.*`,
|
||||||
|
`tool.*`, `reasoning`) but is owned by our plugin.
|
||||||
|
|
||||||
|
## Key architectural decisions + rationale
|
||||||
|
|
||||||
|
| Decision | Rationale |
|
||||||
|
|---|---|
|
||||||
|
| **Community-style platform plugin** (`register(ctx)` → `ctx.register_platform`) | Zero hermes-core changes; survives hermes updates; follows `ADDING_A_PLATFORM.md` "Plugin Path". |
|
||||||
|
| **Single WS transport** for chat, streaming, tools, media, pairing, FCM-token | One connection, one auth, one dependency (`websockets`, already core). Media as chunked binary frames avoids adding an HTTP server. |
|
||||||
|
| **`websockets` + `httpx` only** | Both are hermes *core* deps → the plugin adds **zero** new Python dependencies (respects hermes supply-chain pinning policy). |
|
||||||
|
| **Structured tool events, app-side verbosity** | Per the requirement: the gateway sends full tool data; the *app* decides everything/truncated/nothing. |
|
||||||
|
| **Reasoning split in the adapter** | The gateway prepends reasoning to the final text when `show_reasoning` is on; the adapter splits the stable prefix into a `reasoning` field so the app renders a clean collapsible block. |
|
||||||
|
| **Channels/threads = `chat_id`/`thread_id`** | The gateway's `SessionSource` already models this; cron delivery already targets `platform:chat_id:thread_id`. We map app concepts onto existing primitives. |
|
||||||
|
| **SQLite outbox + sync cursor** | Offline delivery + reconnect catch-up without re-reading full history. |
|
||||||
|
| **Compose Multiplatform** | Desktop is "the Android app, tweaked" → share protocol/state/UI; only platform services + layout differ. |
|
||||||
|
|
||||||
|
## Data flow (one turn)
|
||||||
|
|
||||||
|
1. App sends `message.send {text}` (or `/cmd`).
|
||||||
|
2. Plugin builds a `MessageEvent` (+ `media_urls` if attachments) →
|
||||||
|
`AndroidAdapter.handle_message(event)`.
|
||||||
|
3. Gateway resolves the session (`chat_id`/`thread_id`), runs the agent.
|
||||||
|
4. Agent streams: `stream_delta_callback` → `GatewayStreamConsumer` →
|
||||||
|
`adapter.send()` (first) / `adapter.edit_message()` (updates) →
|
||||||
|
`message.start` / `message.update` frames.
|
||||||
|
5. Tool calls: `tool_progress_callback` → progress queue → `adapter.send()` →
|
||||||
|
`tool.*` frames (structured).
|
||||||
|
6. Intermediate beats: `interim_assistant_callback` → consumer → `commentary` frames.
|
||||||
|
7. Final answer: consumer finalizes → `adapter.send()` → `message` frame
|
||||||
|
(reasoning split into its own field).
|
||||||
|
8. If the app is disconnected at any point: frame is dropped to the **outbox**
|
||||||
|
and a **push** is fired; on reconnect the app `sync`s the delta.
|
||||||
|
|
||||||
|
> Implementation note: the main gateway uses the **legacy callback path**
|
||||||
|
> (not the ACP-only event-native `render_message_event` path). We therefore map
|
||||||
|
> the legacy `send`/`edit_message`/progress calls to our frames. The exact
|
||||||
|
> tool-progress vs commentary classification is verified empirically in M2 by
|
||||||
|
> running the real gateway with a test WS client (see `13-testing.md`).
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
# 02 — Monorepo Structure
|
||||||
|
|
||||||
|
One repository, three artifacts. `hermes-agent/` is **not** part of the repo
|
||||||
|
(git-ignored reference).
|
||||||
|
|
||||||
|
## Top-level tree
|
||||||
|
|
||||||
|
```
|
||||||
|
iris_x_hermes/
|
||||||
|
├── .gitignore # MUST exclude hermes-agent/ (see below)
|
||||||
|
├── README.md # Repo root readme (short; points to docs/)
|
||||||
|
├── docs/ # ← THIS reference library
|
||||||
|
│
|
||||||
|
├── hermes-agent/ # ⚠️ READ-ONLY REFERENCE — NEVER PUSHED (git-ignored)
|
||||||
|
│
|
||||||
|
├── gateway-plugin/ # ① Python plugin → installed to ~/.hermes/plugins/android
|
||||||
|
│ ├── plugin.yaml # manifest (kind: platform, env vars, home channel)
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── adapter.py # AndroidAdapter(BasePlatformAdapter) + register(ctx)
|
||||||
|
│ ├── ws_server.py # websockets server, connection registry, framing
|
||||||
|
│ ├── protocol.py # frame schemas (source of truth, mirrored in Kotlin)
|
||||||
|
│ ├── media.py # inbound cache + outbound chunked streaming
|
||||||
|
│ ├── outbox.py # SQLite offline outbox + sync cursor
|
||||||
|
│ ├── push.py # PushBackend: FcmBackend + NtfyBackend
|
||||||
|
│ ├── pairing.py # token gen/verify, device registry
|
||||||
|
│ ├── search.py # FTS5 session search bridge
|
||||||
|
│ └── tests/ # pytest (run via hermes scripts/run_tests.sh)
|
||||||
|
│
|
||||||
|
├── app/ # ② + ③ Compose Multiplatform project (Kotlin)
|
||||||
|
│ ├── settings.gradle.kts
|
||||||
|
│ ├── build.gradle.kts
|
||||||
|
│ ├── gradle.properties
|
||||||
|
│ ├── gradle/ gradlew gradlew.bat
|
||||||
|
│ ├── shared/ # KMP module — the bulk of the code
|
||||||
|
│ │ ├── build.gradle.kts
|
||||||
|
│ │ └── src/
|
||||||
|
│ │ ├── commonMain/kotlin/iris/… # protocol, WS client, repo, state, Compose UI
|
||||||
|
│ │ ├── androidMain/kotlin/… # FCM, ExoPlayer, SAF picker, notifications
|
||||||
|
│ │ └── desktopMain/kotlin/… # tray, file dialog, player, window
|
||||||
|
│ ├── androidApp/ # thin Android shell (Application, MainActivity)
|
||||||
|
│ │ ├── build.gradle.kts
|
||||||
|
│ │ └── src/main/… # AndroidManifest, res, Firebase options
|
||||||
|
│ └── desktopApp/ # thin Desktop shell (main(), window)
|
||||||
|
│ ├── build.gradle.kts
|
||||||
|
│ └── src/main/kotlin/…
|
||||||
|
│
|
||||||
|
└── (no other top-level code)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Module responsibilities
|
||||||
|
|
||||||
|
### `gateway-plugin/` (Python)
|
||||||
|
- **`plugin.yaml`** — manifest: `name: android-platform`, `kind: platform`,
|
||||||
|
`requires_env` / `optional_env` (surfaced in `hermes config`/setup).
|
||||||
|
- **`adapter.py`** — `AndroidAdapter(BasePlatformAdapter)` + `register(ctx)`.
|
||||||
|
The heart of the plugin. See `03-gateway-plugin.md`.
|
||||||
|
- **`ws_server.py`** — `websockets` server, per-device connection registry,
|
||||||
|
frame encode/decode, heartbeat, broadcast routing to all connected devices.
|
||||||
|
- **`protocol.py`** — dataclasses/constants for every frame (single source of
|
||||||
|
truth; `docs/protocol/frames.schema.json` is generated/mirrored from it).
|
||||||
|
- **`media.py`** — inbound chunked upload → `cache_*_from_bytes`; outbound
|
||||||
|
`media.offer`/`media.pull` chunked streaming.
|
||||||
|
- **`outbox.py`** — SQLite outbox per `chat_id` + monotonic sync cursor.
|
||||||
|
- **`push.py`** — `PushBackend` interface; `FcmBackend` (httpx, FCM HTTP v1) and
|
||||||
|
`NtfyBackend` (reuses hermes ntfy publish). Selected by `ANDROID_PUSH_BACKEND`.
|
||||||
|
- **`pairing.py`** — token generation/verification (constant-time), device
|
||||||
|
registry (SQLite), QR payload.
|
||||||
|
- **`search.py`** — FTS5 query bridge over the hermes session store.
|
||||||
|
|
||||||
|
### `app/shared` (Kotlin KMP)
|
||||||
|
- **`commonMain`** — protocol models (kotlinx-serialization), `GatewayClient`
|
||||||
|
(OkHttp WS), repositories (Room), ViewModels (StateFlow), and the Compose UI
|
||||||
|
(design system, screens). ~80% of app code.
|
||||||
|
- **`androidMain`** — FCM service, ExoPlayer, SAF media picker, system
|
||||||
|
notifications, `MediaPlayer` actual.
|
||||||
|
- **`desktopMain`** — tray + OS notifications, desktop player, file dialog,
|
||||||
|
window management, `MediaPlayer` actual.
|
||||||
|
|
||||||
|
### `app/androidApp` / `app/desktopApp`
|
||||||
|
Thin shells: `Application`/`MainActivity` (Android) and `main()`/window
|
||||||
|
(Desktop). They compose the `shared` UI and inject platform services.
|
||||||
|
|
||||||
|
## Build systems
|
||||||
|
|
||||||
|
- **Python plugin:** no build step (pure Python, stdlib + hermes core deps).
|
||||||
|
Installed by copying/symlinking into `~/.hermes/plugins/android`. Tested with
|
||||||
|
hermes's `scripts/run_tests.sh`.
|
||||||
|
- **Kotlin/CMP:** Gradle (Kotlin DSL) with the Compose Multiplatform plugin.
|
||||||
|
`./gradlew :androidApp:installDebug`, `./gradlew :desktopApp:run`,
|
||||||
|
`./gradlew :shared:testDebugUnitTest`.
|
||||||
|
|
||||||
|
## `.gitignore` (root) — critical
|
||||||
|
|
||||||
|
```gitignore
|
||||||
|
# hermes-agent is a read-only research reference — NEVER commit/push it
|
||||||
|
/hermes-agent/
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
|
||||||
|
# Kotlin / Gradle
|
||||||
|
.gradle/
|
||||||
|
build/
|
||||||
|
local.properties
|
||||||
|
*.iml
|
||||||
|
.idea/
|
||||||
|
|
||||||
|
# Android / Firebase
|
||||||
|
app/androidApp/src/main/res/values/secrets.xml
|
||||||
|
google-services.json
|
||||||
|
*.jks
|
||||||
|
keystore.jks
|
||||||
|
|
||||||
|
# OS / misc
|
||||||
|
.DS_Store
|
||||||
|
*.log
|
||||||
|
```
|
||||||
|
|
||||||
|
> The `/hermes-agent/` line is non-negotiable. Add a pre-commit guard (or CI
|
||||||
|
> check) that fails if any path under `hermes-agent/` is staged.
|
||||||
|
|
||||||
|
## Install layout (runtime)
|
||||||
|
|
||||||
|
- **Plugin:** `~/.hermes/plugins/android/` ← copy of `gateway-plugin/`
|
||||||
|
(or a symlink for dev). Discovered by hermes's `PluginManager`.
|
||||||
|
- **App (dev):** installed on-device via `./gradlew :androidApp:installDebug`.
|
||||||
|
- **App (desktop, dev):** `./gradlew :desktopApp:run`.
|
||||||
@@ -0,0 +1,263 @@
|
|||||||
|
# 03 — Gateway Plugin (Python)
|
||||||
|
|
||||||
|
The plugin is a **community-style hermes platform plugin** named `android`.
|
||||||
|
It follows the "Plugin Path" in `hermes-agent/gateway/platforms/ADDING_A_PLATFORM.md`
|
||||||
|
and the canonical example `hermes-agent/plugins/platforms/irc/adapter.py`.
|
||||||
|
**Zero hermes-core changes. Zero new Python dependencies** (`websockets` and
|
||||||
|
`httpx` are already core deps).
|
||||||
|
|
||||||
|
> Source map for every hermes integration point: see `15-hermes-reference.md`.
|
||||||
|
|
||||||
|
## 3.1 `plugin.yaml` (manifest)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: android-platform
|
||||||
|
label: Android
|
||||||
|
kind: platform
|
||||||
|
version: 0.1.0
|
||||||
|
description: >
|
||||||
|
Native Android / Desktop client gateway adapter for Hermes Agent.
|
||||||
|
Runs a WebSocket server inside the gateway; the app connects with a
|
||||||
|
pairing token. Supports streaming, reasoning, structured tool events,
|
||||||
|
channels/threads, media, FTS5 search, and FCM/ntfy push.
|
||||||
|
author: <you>
|
||||||
|
requires_env:
|
||||||
|
- name: ANDROID_TOKEN
|
||||||
|
description: "Shared pairing token the app presents on connect"
|
||||||
|
prompt: "Android pairing token"
|
||||||
|
password: true
|
||||||
|
optional_env:
|
||||||
|
- name: ANDROID_WS_HOST
|
||||||
|
description: "WS bind host (default 127.0.0.1; use 0.0.0.0 for LAN)"
|
||||||
|
prompt: "WS host"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_PORT
|
||||||
|
description: "WS port (default 8790)"
|
||||||
|
prompt: "WS port"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_HOME_CHANNEL
|
||||||
|
description: "Default chat id for cron/notification delivery (default android:default)"
|
||||||
|
prompt: "Home channel"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_ALLOWED_USERS
|
||||||
|
description: "Comma-separated allowed device_ids (empty = token-only auth)"
|
||||||
|
prompt: "Allowed device ids"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_ALLOW_ALL_USERS
|
||||||
|
description: "Allow any paired device (dev only)"
|
||||||
|
prompt: "Allow all devices? (true/false)"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_PUSH_BACKEND
|
||||||
|
description: "Push backend: fcm (default) or ntfy"
|
||||||
|
prompt: "Push backend"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_FCM_SERVICE_ACCOUNT
|
||||||
|
description: "Path to Firebase service-account JSON (FCM HTTP v1)"
|
||||||
|
prompt: "FCM service account path"
|
||||||
|
password: true
|
||||||
|
- name: ANDROID_FCM_SERVER_KEY
|
||||||
|
description: "Legacy FCM server key (fallback if no service account)"
|
||||||
|
prompt: "FCM server key"
|
||||||
|
password: true
|
||||||
|
- name: NTFY_TOPIC
|
||||||
|
description: "ntfy topic for push (when ANDROID_PUSH_BACKEND=ntfy)"
|
||||||
|
prompt: "ntfy topic"
|
||||||
|
password: false
|
||||||
|
- name: NTFY_SERVER_URL
|
||||||
|
description: "ntfy server URL (default https://ntfy.sh)"
|
||||||
|
prompt: "ntfy server URL"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_CERT
|
||||||
|
description: "TLS cert path for WSS (optional)"
|
||||||
|
prompt: "WSS cert"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_KEY
|
||||||
|
description: "TLS key path for WSS (optional)"
|
||||||
|
prompt: "WSS key"
|
||||||
|
password: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Behavioral (non-secret) settings live in `config.yaml` under
|
||||||
|
`gateway.platforms.android.extra` (host, port, home_channel, outbox retention,
|
||||||
|
max upload bytes, tls). Secrets live in `.env`. (hermes policy: `.env` = secrets
|
||||||
|
only.)
|
||||||
|
|
||||||
|
## 3.2 `register(ctx)` entry point
|
||||||
|
|
||||||
|
```python
|
||||||
|
def register(ctx):
|
||||||
|
ctx.register_platform(
|
||||||
|
name="android",
|
||||||
|
label="Android",
|
||||||
|
adapter_factory=lambda cfg: AndroidAdapter(cfg),
|
||||||
|
check_fn=check_requirements, # passive: websockets importable + token set
|
||||||
|
validate_config=validate_config, # host/port/token present
|
||||||
|
is_connected=is_connected,
|
||||||
|
required_env=["ANDROID_TOKEN"],
|
||||||
|
install_hint="No extra packages needed (websockets + httpx are core deps)",
|
||||||
|
setup_fn=interactive_setup, # hermes gateway setup flow
|
||||||
|
env_enablement_fn=_env_enablement, # seed extra + home_channel from env
|
||||||
|
cron_deliver_env_var="ANDROID_HOME_CHANNEL",
|
||||||
|
standalone_sender_fn=_standalone_send, # best-effort out-of-proc cron (stretch)
|
||||||
|
parse_target_ref_fn=_parse_target_ref, # "android:<chat>[:<thread>]"
|
||||||
|
allowed_users_env="ANDROID_ALLOWED_USERS",
|
||||||
|
allow_all_env="ANDROID_ALLOW_ALL_USERS",
|
||||||
|
max_message_length=0, # 0 = no limit (WS has none)
|
||||||
|
emoji="📱",
|
||||||
|
pii_safe=False,
|
||||||
|
platform_hint=(
|
||||||
|
"You are chatting with the user through their native Iris app "
|
||||||
|
"(Android/Desktop). It renders Markdown, inline code, images, "
|
||||||
|
"audio and video, and shows your reasoning and tool activity. "
|
||||||
|
"Conversations are organized into channels and optional threads. "
|
||||||
|
"Keep formatting rich but readable."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Field reference (all from `PlatformEntry`, `gateway/platform_registry.py:63`):
|
||||||
|
`adapter_factory`, `check_fn`, `validate_config`, `is_connected`, `required_env`,
|
||||||
|
`install_hint`, `setup_fn`, `env_enablement_fn`, `apply_yaml_config_fn`,
|
||||||
|
`cron_deliver_env_var`, `parse_target_ref_fn`, `allowed_users_env`,
|
||||||
|
`allow_all_env`, `max_message_length`, `pii_safe`, `platform_hint`, `emoji`,
|
||||||
|
`ensure_deps_fn`.
|
||||||
|
|
||||||
|
- **`check_requirements()`** — passive probe: `import websockets` succeeds and
|
||||||
|
`ANDROID_TOKEN` is set. Never installs.
|
||||||
|
- **`_env_enablement()`** — returns a dict seeding `PlatformConfig.extra`
|
||||||
|
(host/port/home_channel/push_backend) + a `home_channel` key
|
||||||
|
`{"chat_id": "android:default", "name": "Default"}` so `hermes gateway status`
|
||||||
|
and cron home-channel resolution work without instantiating the adapter.
|
||||||
|
- **`_parse_target_ref(ref)`** — if `ref` starts with `android:`, return
|
||||||
|
`(chat_id, thread_id)` parsed from `android:<chat>[:<thread>]`; else `None`.
|
||||||
|
- **`interactive_setup()`** — prompts for token (or generates one), host/port,
|
||||||
|
push backend + credentials, prints a QR code (pairing) and the app URL.
|
||||||
|
|
||||||
|
## 3.3 `AndroidAdapter(BasePlatformAdapter)`
|
||||||
|
|
||||||
|
Constructor: `super().__init__(config=config, platform=Platform("android"))`.
|
||||||
|
Reads `config.extra` (env overrides win). Initializes: WS server (not started
|
||||||
|
until `connect()`), connection registry, outbox (SQLite under
|
||||||
|
`get_hermes_home()/"android"`), push backend, pairing store, channel directory.
|
||||||
|
|
||||||
|
### Lifecycle
|
||||||
|
- **`connect(*, is_reconnect=False) -> bool`**
|
||||||
|
- Acquire scoped lock (`gateway.status.acquire_scoped_lock("android", key)`)
|
||||||
|
so two profiles can't bind the same port/identity.
|
||||||
|
- Start the `websockets` server on `host:port` (TLS if cert/key set).
|
||||||
|
- `_mark_connected()`; return True.
|
||||||
|
- **`disconnect()`**
|
||||||
|
- Stop server, close all device sockets, release lock, `_mark_disconnected()`.
|
||||||
|
|
||||||
|
### Inbound (app → agent)
|
||||||
|
- WS `message.send {text, reply_to?, media_refs?}` → build `SessionSource` via
|
||||||
|
`self.build_source(chat_id, chat_name, chat_type, user_id, user_name,
|
||||||
|
thread_id)` → build `MessageEvent(text=…, message_type=TEXT, source=…,
|
||||||
|
media_urls=[cached paths], media_types=[…], reply_to_message_id=…)` →
|
||||||
|
`await self.handle_message(event)`.
|
||||||
|
- Slash commands arrive as plain text starting with `/`; the gateway's command
|
||||||
|
pipeline resolves + dispatches them (no special handling needed).
|
||||||
|
- `picker.select {picker_id, value}` → route to the gateway-side resolvers
|
||||||
|
(model picker / choice picker / clarify / approval / slash-confirm) using the
|
||||||
|
shared callback-id conventions (`cl:<id>:<idx>`, `appr:<id>:<choice>`,
|
||||||
|
`sc:<choice>:<id>`).
|
||||||
|
- `channel.create` / `channel.rename` / `channel.set_default` → mutate the
|
||||||
|
channel directory (SQLite) + emit `channel.*` frames to all devices.
|
||||||
|
- `search {query, scope, chat_id?, thread_id?}` → `search.py` → `search.results`.
|
||||||
|
- `media.upload` (chunked) → `media.py` → `cache_*_from_bytes` → `media_ref`.
|
||||||
|
- `media.pull {media_id}` → stream cached bytes as binary frames.
|
||||||
|
- `fcm.register {token}` / `hello` → update device registry.
|
||||||
|
- `read.receipt {message_id}` → mark delivered/read (drives ✓✓), ack.
|
||||||
|
- `sync {cursor}` → `outbox.py` → replay frames since cursor.
|
||||||
|
|
||||||
|
### Outbound (agent → app)
|
||||||
|
- **`send(chat_id, content, reply_to=None, metadata=None) -> SendResult`**
|
||||||
|
- Split reasoning prefix (see `05-streaming.md`) → `reasoning` field.
|
||||||
|
- If **any** device is connected: broadcast `message` frame to all.
|
||||||
|
- Else (no live devices): append to **outbox** + fire **push** (FCM/ntfy).
|
||||||
|
- Return `SendResult(success=True, message_id=<id>)`.
|
||||||
|
- **`edit_message(chat_id, message_id, content)`** → `message.update` frame
|
||||||
|
(drives streaming). If no device, no-op (outbox holds the final `send`).
|
||||||
|
- **`send_typing(chat_id, metadata=None)`** → `typing` frame.
|
||||||
|
- **`get_chat_info(chat_id) -> dict`** → `{"name": <channel name>, "type": "dm"|"channel"}`
|
||||||
|
from the channel directory.
|
||||||
|
- **Media send** — `send_image / send_video / send_document / send_voice /
|
||||||
|
send_image_file / send_multiple_images`: stage the file in the media cache,
|
||||||
|
mint a `media_id`, emit `media.offer {media_id, mime, size, filename, kind}`,
|
||||||
|
serve bytes on `media.pull`. (Base-class `extract_media`/`extract_images`
|
||||||
|
already pull `MEDIA:`/image tags out of agent text and call these.)
|
||||||
|
- **Interactive pickers** — `send_model_picker(...)`, `send_choice_picker(...)`,
|
||||||
|
`send_clarify(...)`, `send_exec_approval(...)`, `send_slash_confirm(...)`:
|
||||||
|
emit `picker.model` / `picker.choice` / `picker.clarify` / `picker.approval` /
|
||||||
|
`picker.confirm` frames with options; store pending state keyed by
|
||||||
|
`picker_id`; resolve on `picker.select`.
|
||||||
|
- **`create_handoff_thread(chat_id, name)`** → create a thread id, register in
|
||||||
|
channel directory, return it (used by cron "continuable" threads).
|
||||||
|
|
||||||
|
### Streaming hooks
|
||||||
|
The main gateway drives delivery through the **legacy callback path**:
|
||||||
|
- `stream_delta_callback` → `GatewayStreamConsumer` → `send()` (first) +
|
||||||
|
`edit_message()` (updates) → `message.start` / `message.update`.
|
||||||
|
- `tool_progress_callback` → progress queue → `send_progress_messages` →
|
||||||
|
`send()` → `tool.*` frames (structured; classified in the adapter).
|
||||||
|
- `interim_assistant_callback` → consumer `on_commentary` → `send()` →
|
||||||
|
`commentary` frames.
|
||||||
|
|
||||||
|
The adapter tracks per-chat **turn state** (in-turn, current streaming
|
||||||
|
`message_id`, last tool index) to classify outbound `send()` calls into
|
||||||
|
`message` vs `tool.*` vs `commentary`. The exact classification markers are
|
||||||
|
verified empirically in M2 (see `13-testing.md`).
|
||||||
|
|
||||||
|
## 3.4 WebSocket server (`ws_server.py`)
|
||||||
|
|
||||||
|
- Library: **`websockets`** (core dep, v15). `websockets.serve(handler, host,
|
||||||
|
port, ssl=ctx)`.
|
||||||
|
- **Handler** per connection:
|
||||||
|
1. Await first frame; must be `hello {token, device_id, device_name, caps,
|
||||||
|
fcm_token?}`. Verify token (constant-time) + allowlist. On failure: send
|
||||||
|
`error {code:"auth"}` and close.
|
||||||
|
2. On success: register in connection registry
|
||||||
|
(`device_id → {ws, caps, fcm_token}`), send
|
||||||
|
`hello.ack {server_caps, sync_cursor, channels[]}`.
|
||||||
|
3. Loop: decode frames, dispatch to adapter inbound handlers.
|
||||||
|
4. On close: deregister; if no devices remain, ensure pending outbox
|
||||||
|
frames have push fired.
|
||||||
|
- **Routing:** `emit(chat_id, frame)` → broadcast to **all** connected
|
||||||
|
devices (no per-chat subscribe; single-user model). Global frames
|
||||||
|
(`channel.*`, `status`) also broadcast to all.
|
||||||
|
- **Heartbeat:** WS ping/pong + app-level `ping`/`pong`; dead peers reaped.
|
||||||
|
- **Backpressure:** per-connection send queue with a bounded buffer; drop
|
||||||
|
`message.update` (coalesce to latest) under pressure, never drop
|
||||||
|
`message`/`tool.end`/`notification`.
|
||||||
|
|
||||||
|
## 3.5 State & storage (all under `get_hermes_home()/"android"`)
|
||||||
|
|
||||||
|
> Use `get_hermes_home()` from `hermes_constants` for **all** paths (profile-safe).
|
||||||
|
> Never hardcode `~/.hermes`.
|
||||||
|
|
||||||
|
- `devices.db` — device registry (device_id, name, caps, fcm_token, ntfy_topic,
|
||||||
|
last_seen, created).
|
||||||
|
- `channels.db` — channel directory (chat_id, name, kind: default|channel|thread,
|
||||||
|
parent_chat_id, created, is_default).
|
||||||
|
- `outbox.db` — undelivered frames per chat_id + monotonic cursor.
|
||||||
|
- `media/` — inbound + outbound media cache (reuse hermes `cache_*_from_bytes`
|
||||||
|
dirs where possible).
|
||||||
|
|
||||||
|
## 3.6 Config resolution
|
||||||
|
|
||||||
|
- **Secrets (`.env`):** `ANDROID_TOKEN`, `ANDROID_FCM_SERVICE_ACCOUNT`,
|
||||||
|
`ANDROID_FCM_SERVER_KEY`, `ANDROID_WS_CERT/KEY`, `NTFY_TOPIC` (if secret).
|
||||||
|
- **Behavioral (`config.yaml` → `gateway.platforms.android.extra`):** `host`,
|
||||||
|
`port`, `home_channel`, `allowed_users`, `push_backend`, `outbox_retention_hours`,
|
||||||
|
`max_upload_bytes`, `tls`.
|
||||||
|
- Env vars override `config.yaml` (hermes convention). Read secrets with the
|
||||||
|
scope-aware `_get_scoped_secret` pattern (see `plugins/platforms/irc/adapter.py:42`)
|
||||||
|
so multiplexed profiles don't leak each other's tokens.
|
||||||
|
|
||||||
|
## 3.7 Failure & lifecycle safety
|
||||||
|
|
||||||
|
- WS server bind failure → `_set_fatal_error("bind_failed", …, retryable=True)`.
|
||||||
|
- All outbound sends are best-effort; a dead socket latches and the frame falls
|
||||||
|
to the outbox.
|
||||||
|
- `disconnect()` cancels the server task and closes sockets cleanly.
|
||||||
|
- Token/PII redaction in all logs (hermes PII policy).
|
||||||
@@ -0,0 +1,327 @@
|
|||||||
|
# 04 — Wire Protocol
|
||||||
|
|
||||||
|
JSON frames over a single WebSocket. One connection per device. Text frames are
|
||||||
|
JSON; media travels as **binary frames** (chunked) referenced by a header frame.
|
||||||
|
|
||||||
|
## Envelope
|
||||||
|
|
||||||
|
Every frame:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"v": 1,
|
||||||
|
"id": 42, // optional; present on requests + their responses
|
||||||
|
"type": "message", // frame type (below)
|
||||||
|
"chat_id": "android:default", // optional; scope for chat-scoped frames
|
||||||
|
"thread_id": "t_123", // optional
|
||||||
|
"payload": { } // type-specific object
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `v` — protocol version (currently `1`). Server rejects unknown major versions.
|
||||||
|
- `id` — request id (client-chosen). Responses/acks echo it. Events have no `id`.
|
||||||
|
- `chat_id` / `thread_id` — top-level for convenience; may also be in `payload`.
|
||||||
|
- Unknown `type`s are ignored (forward-compat); unknown `payload` fields ignored.
|
||||||
|
|
||||||
|
**Binary media frames** are not JSON. A media transfer is: one JSON header frame
|
||||||
|
(`media.upload.start` / `media.pull` ack) followed by raw binary frames, then a
|
||||||
|
JSON `media.upload.end` / final ack. See `07-media.md`.
|
||||||
|
|
||||||
|
## Server → App (events / responses)
|
||||||
|
|
||||||
|
### `hello.ack`
|
||||||
|
Pairing succeeded.
|
||||||
|
```json
|
||||||
|
{"type":"hello.ack","payload":{
|
||||||
|
"server_caps":{"streaming":true,"reasoning":true,"tools":true,"media":true,
|
||||||
|
"search":true,"push":"fcm","pickers":true},
|
||||||
|
"sync_cursor":1042,
|
||||||
|
"channels":[{"chat_id":"android:default","name":"Default","kind":"default","is_default":true}]
|
||||||
|
}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `message`
|
||||||
|
A final / standalone message.
|
||||||
|
```json
|
||||||
|
{"type":"message","chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{
|
||||||
|
"message_id":"m_9001","role":"assistant",
|
||||||
|
"text":"Here is the answer…",
|
||||||
|
"reasoning":"The user asked… so I will…", // optional; render ABOVE text
|
||||||
|
"media":[{"media_id":"md_5","kind":"video","mime":"video/mp4","size":123456,
|
||||||
|
"filename":"clip.mp4"}], // optional
|
||||||
|
"reply_to":"m_8999", // optional
|
||||||
|
"model":"qwen3-27b","tokens":11,"ts":1724000000000
|
||||||
|
}}
|
||||||
|
```
|
||||||
|
`role` ∈ `user | assistant | system | cron`. `reasoning` present only when the
|
||||||
|
agent produced reasoning and `show_reasoning` is on.
|
||||||
|
|
||||||
|
### `message.start` / `message.update` / `message.stop`
|
||||||
|
Streaming a bubble. `update` carries the **full** current text (app replaces).
|
||||||
|
```json
|
||||||
|
{"type":"message.start","chat_id":"…","payload":{"message_id":"m_9002","role":"assistant"}}
|
||||||
|
{"type":"message.update","chat_id":"…","payload":{"message_id":"m_9002","text":"partial…"}}
|
||||||
|
{"type":"message.stop","chat_id":"…","payload":{"message_id":"m_9002","final_text":"full…",
|
||||||
|
"reasoning":"…","model":"…","tokens":11}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `commentary`
|
||||||
|
Intermediate assistant beat (between tool iterations).
|
||||||
|
```json
|
||||||
|
{"type":"commentary","chat_id":"…","payload":{"message_id":"m_9003","text":"Let me inspect the repo first."}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `tool.start` / `tool.progress` / `tool.end`
|
||||||
|
**Structured** tool events. The app decides how much to show (everything /
|
||||||
|
truncated / nothing).
|
||||||
|
```json
|
||||||
|
{"type":"tool.start","chat_id":"…","payload":{
|
||||||
|
"index":3,"name":"terminal","preview":"pytest -q","args":{"command":"pytest -q"}}}
|
||||||
|
{"type":"tool.progress","chat_id":"…","payload":{"index":3,"name":"terminal","note":"running…"}}
|
||||||
|
{"type":"tool.end","chat_id":"…","payload":{"index":3,"name":"terminal","ok":true,"duration":12.4,
|
||||||
|
"output_preview":"12 passed"}}
|
||||||
|
```
|
||||||
|
`args` may be large; the app truncates per its setting. `output_preview` is a
|
||||||
|
short tail (full output is not streamed — it lives in agent history).
|
||||||
|
|
||||||
|
### `typing` / `typing.stop`
|
||||||
|
```json
|
||||||
|
{"type":"typing","chat_id":"…","payload":{"on":true}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `notification`
|
||||||
|
In-app banner (foreground) and/or push mirror (background).
|
||||||
|
```json
|
||||||
|
{"type":"notification","chat_id":"…","payload":{
|
||||||
|
"kind":"channel_renamed","title":"ARIA","body":"Renamed topic to …","ts":1724000000000}}
|
||||||
|
```
|
||||||
|
`kind` ∈ `channel_renamed | channel_created | cron | approval | clarify | generic`.
|
||||||
|
|
||||||
|
### `picker.model` / `picker.choice` / `picker.clarify` / `picker.approval` / `picker.confirm`
|
||||||
|
Interactive prompts. App renders a native picker; answers via `picker.select`.
|
||||||
|
```json
|
||||||
|
{"type":"picker.model","chat_id":"…","payload":{
|
||||||
|
"picker_id":"pm_1","current_model":"qwen3-27b","current_provider":"local",
|
||||||
|
"providers":[{"id":"local","label":"Local","models":[{"id":"qwen3-27b","label":"Qwen3 27B"}]}]}}
|
||||||
|
{"type":"picker.choice","chat_id":"…","payload":{
|
||||||
|
"picker_id":"pc_1","title":"Reasoning effort","choices":[
|
||||||
|
{"value":"low","label":"Low"},{"value":"high","label":"High","is_current":true}]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `channel.list` / `channel.created` / `channel.renamed` / `channel.deleted`
|
||||||
|
Channel directory updates. **Broadcast to all connected devices** (no explicit
|
||||||
|
subscribe; the server pushes to every open WS).
|
||||||
|
```json
|
||||||
|
{"type":"channel.created","payload":{"chat_id":"android:chan_7","name":"Cron Reports",
|
||||||
|
"kind":"channel","parent_chat_id":null}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `history`
|
||||||
|
Response to a `history` request. Returns a page of messages for a chat/thread.
|
||||||
|
```json
|
||||||
|
{"type":"history","id":20,"chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{
|
||||||
|
"messages":[
|
||||||
|
{"message_id":"m_8990","role":"user","text":"Hi","ts":1723990000000},
|
||||||
|
{"message_id":"m_8991","role":"assistant","text":"Hello!","reasoning":"…",
|
||||||
|
"model":"qwen3-27b","tokens":8,"ts":1723990001000}
|
||||||
|
],
|
||||||
|
"has_more":true,
|
||||||
|
"oldest_message_id":"m_8990"
|
||||||
|
}}
|
||||||
|
```
|
||||||
|
`messages` are ordered oldest → newest. Paginate with `before_message_id` in the
|
||||||
|
request. The app uses this to **populate the initial view** when a channel is
|
||||||
|
opened (complements `sync`, which only replays undelivered outbox frames).
|
||||||
|
|
||||||
|
### `commands.catalog`
|
||||||
|
Response to a `commands.catalog` request. Full slash-command list.
|
||||||
|
```json
|
||||||
|
{"type":"commands.catalog","id":21,"payload":{
|
||||||
|
"commands":[
|
||||||
|
{"name":"/new","description":"Start a new session","args_hint":"","category":"session"},
|
||||||
|
{"name":"/model","description":"Switch model","args_hint":"<provider/model>","category":"config"},
|
||||||
|
{"name":"/reasoning","description":"Toggle reasoning effort","args_hint":"[low|medium|high]","category":"config"},
|
||||||
|
{"name":"/status","description":"Show session status","args_hint":"","category":"info"},
|
||||||
|
{"name":"/cron","description":"Manage cron jobs","args_hint":"<list|add|rm>","category":"automation"},
|
||||||
|
{"name":"/tools","description":"List available tools","args_hint":"","category":"info"}
|
||||||
|
]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `commands.complete`
|
||||||
|
Response to a `commands.complete` request. Autocomplete matches for a typed prefix.
|
||||||
|
```json
|
||||||
|
{"type":"commands.complete","id":22,"payload":{
|
||||||
|
"prefix":"/mod",
|
||||||
|
"matches":[
|
||||||
|
{"name":"/model","description":"Switch model","args_hint":"<provider/model>"}
|
||||||
|
]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `agent.busy` / `agent.idle`
|
||||||
|
Agent lifecycle for a chat/thread. App shows a "thinking…" indicator on `busy`.
|
||||||
|
```json
|
||||||
|
{"type":"agent.busy","chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{"reason":"processing"}}
|
||||||
|
{"type":"agent.idle","chat_id":"android:default","thread_id":null,"payload":{}}
|
||||||
|
```
|
||||||
|
`reason` ∈ `processing | tool | waiting_input | cron`.
|
||||||
|
|
||||||
|
### `search.results`
|
||||||
|
```json
|
||||||
|
{"type":"search.results","id":7,"payload":{
|
||||||
|
"query":"deploy","scope":"all","hits":[
|
||||||
|
{"message_id":"m_123","chat_id":"android:chan_7","thread_id":null,
|
||||||
|
"role":"assistant","snippet":"…deploy the service…","ts":1723900000000}]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `media.offer`
|
||||||
|
Agent-sent media is available; app pulls bytes.
|
||||||
|
```json
|
||||||
|
{"type":"media.offer","chat_id":"…","payload":{
|
||||||
|
"media_id":"md_5","kind":"video","mime":"video/mp4","size":123456,"filename":"clip.mp4"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `status`
|
||||||
|
Gateway lifecycle / session info.
|
||||||
|
```json
|
||||||
|
{"type":"status","payload":{"state":"online","session":{"chat_id":"…","model":"…","tokens":11}}}
|
||||||
|
```
|
||||||
|
`state` ∈ `online | restarting | degraded`.
|
||||||
|
|
||||||
|
### `error`
|
||||||
|
```json
|
||||||
|
{"type":"error","id":7,"payload":{"code":"not_found","message":"chat_id unknown"}}
|
||||||
|
```
|
||||||
|
`code` ∈ `auth | not_found | rate_limited | media_too_large | unsupported | internal`.
|
||||||
|
|
||||||
|
### `pong`
|
||||||
|
Keepalive reply to `ping`.
|
||||||
|
|
||||||
|
## App → Server (requests / actions)
|
||||||
|
|
||||||
|
### `hello`
|
||||||
|
First frame; auth + caps.
|
||||||
|
```json
|
||||||
|
{"type":"hello","payload":{
|
||||||
|
"token":"<ANDROID_TOKEN>","device_id":"dev_a1b2","device_name":"MIX 2S",
|
||||||
|
"caps":{"min_protocol":1,"media":true,"push":"fcm"},
|
||||||
|
"fcm_token":"<FCM token>","ntfy_topic":"<topic, if ntfy>"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `message.send`
|
||||||
|
Send text (or a `/slash-command`).
|
||||||
|
```json
|
||||||
|
{"type":"message.send","id":10,"chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{"text":"/model qwen3-27b","reply_to":"m_9001","media_refs":["mu_1"]}}
|
||||||
|
```
|
||||||
|
`media_refs` reference completed `media.upload`s to attach.
|
||||||
|
|
||||||
|
### `media.upload.start` / (binary) / `media.upload.end`
|
||||||
|
See `07-media.md`.
|
||||||
|
```json
|
||||||
|
{"type":"media.upload.start","id":11,"payload":{
|
||||||
|
"media_ref":"mu_1","kind":"image","mime":"image/jpeg","size":204800,"filename":"a.jpg"}}
|
||||||
|
// … binary frames …
|
||||||
|
{"type":"media.upload.end","id":11,"payload":{"media_ref":"mu_1","sha256":"…"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `media.pull`
|
||||||
|
Request agent-sent media bytes.
|
||||||
|
```json
|
||||||
|
{"type":"media.pull","id":12,"payload":{"media_id":"md_5"}}
|
||||||
|
// server replies: binary frames, then {"type":"media.pull.end","id":12,"payload":{"ok":true}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `picker.select`
|
||||||
|
Answer an interactive picker.
|
||||||
|
```json
|
||||||
|
{"type":"picker.select","id":13,"payload":{"picker_id":"pm_1","value":"local/qwen3-27b"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `channel.create` / `channel.rename` / `channel.set_default` / `channel.delete`
|
||||||
|
```json
|
||||||
|
{"type":"channel.create","id":14,"payload":{"name":"Cron Reports","kind":"channel"}}
|
||||||
|
{"type":"channel.rename","id":15,"chat_id":"android:chan_7","payload":{"name":"Reports"}}
|
||||||
|
{"type":"channel.set_default","id":16,"chat_id":"android:chan_7","payload":{}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `search`
|
||||||
|
```json
|
||||||
|
{"type":"search","id":17,"payload":{"query":"deploy","scope":"all"}}
|
||||||
|
{"type":"search","id":18,"payload":{"query":"deploy","scope":"chat","chat_id":"android:chan_7","thread_id":null}}
|
||||||
|
```
|
||||||
|
`scope` ∈ `all | chat`.
|
||||||
|
|
||||||
|
### `read.receipt`
|
||||||
|
App → server: "user has viewed this message." Server stores the read state and
|
||||||
|
broadcasts to other devices (for multi-device ✓✓ sync). The app uses it to
|
||||||
|
mark messages as read locally (✓✓ on user bubbles).
|
||||||
|
```json
|
||||||
|
{"type":"read.receipt","payload":{"chat_id":"android:default","message_id":"m_9001"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `history`
|
||||||
|
Load a page of messages for a chat/thread (initial open, scroll-up pagination).
|
||||||
|
```json
|
||||||
|
{"type":"history","id":20,"chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{"before_message_id":"m_8990","limit":50}}
|
||||||
|
```
|
||||||
|
`before_message_id` — return messages older than this (omit for newest page).
|
||||||
|
`limit` — max messages (default 50, max 200).
|
||||||
|
|
||||||
|
### `commands.catalog`
|
||||||
|
Fetch the full slash-command catalog (for the `Menü` bottom sheet).
|
||||||
|
```json
|
||||||
|
{"type":"commands.catalog","id":21,"payload":{}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `commands.complete`
|
||||||
|
Autocomplete for a typed `/prefix`.
|
||||||
|
```json
|
||||||
|
{"type":"commands.complete","id":22,"payload":{"prefix":"/mod"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `agent.stop`
|
||||||
|
Stop the current agent turn (abort generation / tool execution).
|
||||||
|
```json
|
||||||
|
{"type":"agent.stop","id":23,"chat_id":"android:default","thread_id":null,"payload":{}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `agent.steer`
|
||||||
|
Inject a steering message mid-turn (redirects the agent without a new turn).
|
||||||
|
```json
|
||||||
|
{"type":"agent.steer","id":24,"chat_id":"android:default","thread_id":null,
|
||||||
|
"payload":{"text":"Actually, focus on the error case."}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `sync`
|
||||||
|
Reconnect catch-up. Replays **undelivered outbox frames** (frames sent while
|
||||||
|
this device was offline). Does NOT load full history — use `history` for that.
|
||||||
|
```json
|
||||||
|
{"type":"sync","id":19,"payload":{"cursor":1042}}
|
||||||
|
// server replays outbox frames with cursor > 1042, then {"type":"sync.done","id":19,"payload":{"cursor":1099}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `fcm.register`
|
||||||
|
Update push token.
|
||||||
|
```json
|
||||||
|
{"type":"fcm.register","payload":{"fcm_token":"<new>","ntfy_topic":"<topic>"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `ping`
|
||||||
|
Keepalive. `{"type":"ping","payload":{"ts":1724000000000}}` → `pong`.
|
||||||
|
|
||||||
|
## Ordering & reliability
|
||||||
|
|
||||||
|
- Frames are ordered per connection (TCP/WS). Streaming `message.update` frames
|
||||||
|
for a `message_id` are monotonic; the app may coalesce to the latest.
|
||||||
|
- Terminal frames (`message`, `message.stop`, `tool.end`, `notification`,
|
||||||
|
`picker.*`, `channel.*`, `agent.busy`, `agent.idle`, `history`,
|
||||||
|
`commands.catalog`, `commands.complete`) are **never dropped** under
|
||||||
|
backpressure; only intermediate `message.update`/`tool.progress` are coalesced.
|
||||||
|
- Anything not delivered live goes to the **outbox** and is replayed by `sync`.
|
||||||
|
- **Broadcast:** channel directory events (`channel.*`) and read-receipts are
|
||||||
|
pushed to **all** connected devices for that gateway (no subscribe step).
|
||||||
|
- Requests get exactly one response or `error` (matched by `id`).
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
# 05 — Streaming, Reasoning, Tools, Intermediate Messages
|
||||||
|
|
||||||
|
This doc covers the four "agent transparency" features and exactly how each is
|
||||||
|
produced by the gateway and rendered by the app.
|
||||||
|
|
||||||
|
> The main hermes gateway delivers via the **legacy callback path** (not the
|
||||||
|
> ACP-only event-native `render_message_event` path). We map the legacy
|
||||||
|
> `send`/`edit_message`/progress calls to our WS frames.
|
||||||
|
|
||||||
|
## 5.1 Text streaming
|
||||||
|
|
||||||
|
**Gateway side.** The agent's `stream_delta_callback` feeds a
|
||||||
|
`GatewayStreamConsumer` (`gateway/stream_consumer.py:156`). The consumer
|
||||||
|
accumulates text and, at intervals / thresholds, calls:
|
||||||
|
- `adapter.send(chat_id, text)` — first time a bubble is created.
|
||||||
|
- `adapter.edit_message(chat_id, message_id, text)` — subsequent updates
|
||||||
|
(each carries the **full** accumulated text).
|
||||||
|
|
||||||
|
**Adapter → frames.**
|
||||||
|
- First `send()` of a turn segment → `message.start {message_id, role}`.
|
||||||
|
- Each `edit_message()` → `message.update {message_id, text}` (full text).
|
||||||
|
- Segment/turn finalization → `message.stop {message_id, final_text, reasoning?,
|
||||||
|
model?, tokens?}` (or a final `message` frame when not streaming).
|
||||||
|
|
||||||
|
**App side.** Maintain a live bubble per `message_id`. On `message.update`,
|
||||||
|
replace the bubble text (cheap: it's a full snapshot). On `message.stop`,
|
||||||
|
finalize (attach reasoning/model/tokens footer, stop the cursor). Auto-scroll
|
||||||
|
while the user is at the bottom.
|
||||||
|
|
||||||
|
**Streaming on/off.** Controlled by hermes `display.platforms.android.streaming`
|
||||||
|
(default follows global). When off, the app just gets one final `message` frame.
|
||||||
|
|
||||||
|
## 5.2 Reasoning (shown *before* the message)
|
||||||
|
|
||||||
|
**Requirement:** reasoning appears as a block **above** the answer (like the
|
||||||
|
reference screenshot's "Reasoning:" panel with a copy button).
|
||||||
|
|
||||||
|
**Gateway side.** hermes prepends reasoning to the final response when
|
||||||
|
`show_reasoning` is enabled (`gateway/run.py:20089`). The format is stable and
|
||||||
|
chosen by `reasoning_style` (`gateway/display_config.py:37`):
|
||||||
|
- `code` (default): `💭 **Reasoning:**\n```\n<reasoning>\n```\n\n<response>`
|
||||||
|
- `blockquote`: `> 💭 **Reasoning:**\n> …\n\n<response>`
|
||||||
|
- `subtext`: `-# 💭 Reasoning\n-# …\n\n<response>` (Discord-style)
|
||||||
|
|
||||||
|
**Plugin config.** Set for the `android` platform:
|
||||||
|
```yaml
|
||||||
|
display:
|
||||||
|
platforms:
|
||||||
|
android:
|
||||||
|
show_reasoning: true
|
||||||
|
reasoning_style: code # we split on the code-fence form
|
||||||
|
```
|
||||||
|
|
||||||
|
**Adapter split.** In `send()`, detect the `code`-style prefix and split:
|
||||||
|
```
|
||||||
|
prefix = "💭 **Reasoning:**\n```\n"
|
||||||
|
# find the closing "\n```\n\n" after the prefix
|
||||||
|
reasoning = text[len(prefix):close_idx]
|
||||||
|
body = text[close_idx + len("\n```\n\n"):]
|
||||||
|
```
|
||||||
|
Emit `message {reasoning: <reasoning>, text: <body>, …}`. If no prefix is found
|
||||||
|
(reasoning off / no reasoning), emit `message {text: …}` with no `reasoning`.
|
||||||
|
|
||||||
|
> Robustness: the split is a best-effort parse of a *stable, gateway-owned*
|
||||||
|
> format. If the format ever changes, the fallback is "no reasoning field, full
|
||||||
|
> text" — the app still shows the answer. Verified in M2 against the live
|
||||||
|
> gateway.
|
||||||
|
|
||||||
|
**App side.** `ReasoningBlock` composable: collapsible, header "💭 Reasoning",
|
||||||
|
monospace body, a **copy** button (matches reference). Rendered **above** the
|
||||||
|
message body. Collapsed by default if long; expanded tap.
|
||||||
|
|
||||||
|
## 5.3 Tool output (app controls verbosity)
|
||||||
|
|
||||||
|
**Requirement:** hermes supports several tool-display modes; the **gateway sends
|
||||||
|
full structured data**, and the **app** chooses how much to show
|
||||||
|
(everything / truncated / nothing).
|
||||||
|
|
||||||
|
**Gateway side.** Tool activity flows through `tool_progress_callback` → the
|
||||||
|
gateway progress queue → `send_progress_messages` (`gateway/run.py:4603`) →
|
||||||
|
`adapter.send()`. The adapter receives tool lines during a turn.
|
||||||
|
|
||||||
|
**Adapter → frames.** The adapter classifies tool activity (via turn-state +
|
||||||
|
line format) and emits **structured** frames — not pre-formatted strings:
|
||||||
|
- `tool.start {index, name, preview, args}` — a tool call began.
|
||||||
|
- `tool.progress {index, name, note}` — in-progress update (optional).
|
||||||
|
- `tool.end {index, name, ok, duration, output_preview}` — completed.
|
||||||
|
|
||||||
|
`index` is a monotonic per-turn counter so the app correlates start→end.
|
||||||
|
`args` is the full argument dict (the app truncates). `output_preview` is a
|
||||||
|
short tail; full tool output is **not** streamed (it lives in agent history and
|
||||||
|
is reachable via search).
|
||||||
|
|
||||||
|
**App side — the verbosity setting** (Settings → "Tool detail"):
|
||||||
|
- **Everything** — show tool name, full args (collapsible), and output preview.
|
||||||
|
- **Truncated** (default) — show `emoji name: "short preview"` one-liner,
|
||||||
|
collapsible to expand.
|
||||||
|
- **Nothing** — suppress `tool.*` frames entirely (clean chat).
|
||||||
|
|
||||||
|
Rendering: a `ToolCard` per tool, grouped under the message it belongs to, with
|
||||||
|
a spinner while `tool.end` hasn't arrived, a ✓/✗ on completion, and duration.
|
||||||
|
|
||||||
|
> Classification detail: the exact way to distinguish a tool-progress `send()`
|
||||||
|
> from a regular `send()`/commentary is confirmed empirically in M2 by running
|
||||||
|
> the real gateway with a test WS client and observing the calls. The adapter
|
||||||
|
> keeps a per-chat turn state machine (turn active, current streaming id, last
|
||||||
|
> tool index) to make the classification deterministic.
|
||||||
|
|
||||||
|
## 5.4 Intermediate messages (commentary)
|
||||||
|
|
||||||
|
**Requirement:** show the agent's interim beats (e.g. "I'll inspect the repo
|
||||||
|
first.") as distinct messages.
|
||||||
|
|
||||||
|
**Gateway side.** `interim_assistant_callback` → consumer `on_commentary`
|
||||||
|
(`gateway/stream_consumer.py:518`) → delivered as its own message.
|
||||||
|
|
||||||
|
**Adapter → frames.** `commentary {message_id, text}`.
|
||||||
|
|
||||||
|
**App side.** Render as a **dimmed / smaller** bubble, visually distinct from
|
||||||
|
final answers (e.g. reduced opacity, no model footer). It reads as a "beat" in
|
||||||
|
the conversation, not a full reply.
|
||||||
|
|
||||||
|
## 5.5 Typing indicator
|
||||||
|
|
||||||
|
`send_typing` → `typing {on:true}`; the app shows an animated indicator in the
|
||||||
|
chat header / above the composer until `typing.stop` or the first
|
||||||
|
`message.start`.
|
||||||
|
|
||||||
|
## 5.6 Frame sequence for a typical turn
|
||||||
|
|
||||||
|
```
|
||||||
|
app → message.send {text:"summarize the repo"}
|
||||||
|
srv → typing {on:true}
|
||||||
|
srv → message.start {message_id:m1}
|
||||||
|
srv → message.update {m1, "Let me look…"} (streaming)
|
||||||
|
srv → tool.start {index:1, name:terminal, args:{command:"ls -la"}}
|
||||||
|
srv → tool.end {index:1, name:terminal, ok:true, duration:0.4}
|
||||||
|
srv → commentary {m2, "Found 12 files."}
|
||||||
|
srv → message.start {message_id:m3}
|
||||||
|
srv → message.update {m3, "The repo has…"}
|
||||||
|
srv → message.stop {m3, final_text:"…", reasoning:"…", model:"…", tokens:42}
|
||||||
|
srv → typing {on:false}
|
||||||
|
```
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
# 06 — Channels, Threads, Cron Delivery, Search
|
||||||
|
|
||||||
|
## 6.1 Concept model
|
||||||
|
|
||||||
|
The hermes gateway already models conversations as `SessionSource` with
|
||||||
|
`chat_id` + `thread_id` + `chat_topic` (`gateway/platforms/base.py:7047`
|
||||||
|
`build_source`). We map app concepts onto these existing primitives — **no new
|
||||||
|
gateway identity concepts**.
|
||||||
|
|
||||||
|
| App concept | hermes primitive | Example |
|
||||||
|
|---|---|---|
|
||||||
|
| Default chat | home channel `chat_id` | `android:default` |
|
||||||
|
| A thread (inside default chat) | `thread_id` under the default `chat_id` | `chat_id=android:default, thread_id=t_12` |
|
||||||
|
| A user-created channel | a new `chat_id` | `android:chan_7` |
|
||||||
|
| A thread inside a channel | `thread_id` under that `chat_id` | `chat_id=android:chan_7, thread_id=t_31` |
|
||||||
|
|
||||||
|
- **`chat_id`** = the conversation lane (a channel or the default chat).
|
||||||
|
- **`thread_id`** = an optional sub-lane within a `chat_id` (topic-like).
|
||||||
|
- The **channel directory** (plugin SQLite, `channels.db`) stores
|
||||||
|
`{chat_id, name, kind: default|channel|thread, parent_chat_id, is_default,
|
||||||
|
created}` and is the source of truth for the app's channel list.
|
||||||
|
|
||||||
|
## 6.2 Default chat
|
||||||
|
|
||||||
|
- On first connect, the plugin ensures a **default channel** exists:
|
||||||
|
`chat_id = ANDROID_HOME_CHANNEL` (default `android:default`), `kind=default`,
|
||||||
|
`is_default=true`, name "Default".
|
||||||
|
- It is also the **cron home channel** (`cron_deliver_env_var=
|
||||||
|
ANDROID_HOME_CHANNEL`), so `deliver=android` (bare) routes here.
|
||||||
|
- The app opens the default chat on launch.
|
||||||
|
|
||||||
|
## 6.3 Threads (toggle for overview)
|
||||||
|
|
||||||
|
- **Requirement:** a default chat where the user can **activate threads** for a
|
||||||
|
better overview, or not.
|
||||||
|
- **Thread toggle** (per default chat, in the chat header menu):
|
||||||
|
- **Threads OFF** — flat conversation; all messages use `thread_id=null`.
|
||||||
|
- **Threads ON** — the app groups the conversation into topic-like lanes.
|
||||||
|
Each new "topic" mints a `thread_id` (via `channel.create {kind:thread,
|
||||||
|
parent_chat_id:android:default}` or an implicit thread). The UI shows a
|
||||||
|
topic switcher (like Telegram topics) above the message list.
|
||||||
|
- Threads are **app-organized** but **gateway-real**: each `thread_id` is a
|
||||||
|
distinct hermes session lane, so context is isolated per thread and cron can
|
||||||
|
target a specific thread.
|
||||||
|
- The gateway's `create_handoff_thread` is used where hermes wants to open a
|
||||||
|
named thread (e.g. continuable cron).
|
||||||
|
|
||||||
|
## 6.4 User-created channels (for cron delegation)
|
||||||
|
|
||||||
|
- **Requirement:** the user creates new channels so **cron job outputs can be
|
||||||
|
delegated to them** instead of the default chat.
|
||||||
|
- **`channel.create {name, kind:"channel"}`** → plugin mints
|
||||||
|
`chat_id = android:chan_<n>`, stores in directory, broadcasts
|
||||||
|
`channel.created` to all devices. The new channel appears in the channel list.
|
||||||
|
- **`channel.rename` / `channel.set_default` / `channel.delete`** manage the
|
||||||
|
directory (rename broadcasts `channel.renamed`; delete is soft — marks
|
||||||
|
archived, keeps history for search).
|
||||||
|
- **Cron targeting** (the key payoff): because the plugin registers
|
||||||
|
`parse_target_ref_fn` and `cron_deliver_env_var`, cron jobs and the
|
||||||
|
`send_message` tool can target any channel/thread:
|
||||||
|
- `deliver="android"` → home (default) channel.
|
||||||
|
- `deliver="android:android:chan_7"` → that channel.
|
||||||
|
- `deliver="android:android:chan_7:t_31"` → that channel's thread.
|
||||||
|
- In-chat: the agent's `cronjob` tool can be told "deliver to the *Cron
|
||||||
|
Reports* channel"; the gateway resolves the name via the channel directory.
|
||||||
|
- **In-app affordance:** each channel's menu has "Set as cron target" / shows a
|
||||||
|
badge when a cron job points at it, and the channel name is offered in the
|
||||||
|
`/cron` creation flow.
|
||||||
|
|
||||||
|
## 6.5 Cron delivery mechanics (how it works under the hood)
|
||||||
|
|
||||||
|
- Cron resolves delivery targets in `cron/scheduler.py:2148`
|
||||||
|
(`_resolve_single_delivery_target`). For `platform:chat_id[:thread_id]` it
|
||||||
|
calls `tools.send_message_tool.resolve_send_target`, which uses our
|
||||||
|
`parse_target_ref_fn` to parse `android:<chat>[:<thread>]`.
|
||||||
|
- Delivery then calls the **live adapter's `send(chat_id, text, …)`** (gateway
|
||||||
|
running) → our WS `message` frame (or outbox+push if the app is offline).
|
||||||
|
- Cron deliveries are framed with a `[Cron delivery: <name>]` header by hermes;
|
||||||
|
the app can style cron messages distinctly (e.g. a small "⏰ <job name>"
|
||||||
|
chip) using the `role:"cron"` / header.
|
||||||
|
- **Mirror option:** hermes `cron.mirror_delivery` (default off) can also mirror
|
||||||
|
a cron delivery into the origin session; we leave it off to keep channels
|
||||||
|
clean.
|
||||||
|
|
||||||
|
## 6.6 Search
|
||||||
|
|
||||||
|
- **Requirement:** search with settings "search everywhere" / "search in this
|
||||||
|
chat/channel".
|
||||||
|
- **Backend:** hermes's session store is **SQLite + FTS5**
|
||||||
|
(`hermes_state.py`, `hermes_state_search.py`). The plugin's `search.py` opens
|
||||||
|
the session DB read-only and runs FTS5 queries.
|
||||||
|
- **`search` frame** → `search.results`:
|
||||||
|
- `scope:"all"` — search **everywhere** (all channels/threads/sessions).
|
||||||
|
- `scope:"chat"` — restrict to the given `chat_id` (and optional `thread_id`).
|
||||||
|
- **Result hit:** `{message_id, chat_id, thread_id, role, snippet, ts}`. The app
|
||||||
|
renders a results list; tapping a hit navigates to that channel/thread and
|
||||||
|
scrolls to + highlights the message.
|
||||||
|
- **Query syntax:** plain text (FTS5). Optional `role:` / `channel:` qualifiers
|
||||||
|
are a nice-to-have; v1 is plain-text + scope.
|
||||||
|
- **Privacy:** search is local to the user's own hermes home; no data leaves the
|
||||||
|
machine.
|
||||||
|
|
||||||
|
## 6.7 Channel list frame
|
||||||
|
|
||||||
|
`hello.ack` and `channel.*` frames carry the directory. App keeps a local copy
|
||||||
|
(Room) and reconciles on `channel.*` events (merge, don't clobber — see
|
||||||
|
`10-android-app.md` state rules).
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
# 07 — Media (upload, download, playback)
|
||||||
|
|
||||||
|
Media travels **over the WebSocket** as chunked binary frames (decision: no
|
||||||
|
separate HTTP server; keeps the plugin to `websockets` only). Both directions
|
||||||
|
use the same chunking.
|
||||||
|
|
||||||
|
## 7.1 Kinds & MIME
|
||||||
|
|
||||||
|
`kind` ∈ `image | audio | video | document | voice`.
|
||||||
|
- `image` — `image/*` (jpg/png/webp/gif/heic).
|
||||||
|
- `audio` — `audio/*` (mp3/m4a/ogg/…) — music.
|
||||||
|
- `video` — `video/*` (mp4/webm/mov).
|
||||||
|
- `document` — anything else (pdf, docx, zip, txt, …).
|
||||||
|
- `voice` — short voice note (`audio/ogg; codecs=opus` typical).
|
||||||
|
|
||||||
|
The app sniffs `kind` from the picked file's MIME; the plugin re-sniffs on
|
||||||
|
receipt (don't trust the client) using hermes helpers
|
||||||
|
(`gateway/platforms/base.py` `_sniff_audio_ext`, `_looks_like_image`).
|
||||||
|
|
||||||
|
## 7.2 Inbound (app → agent) — `media.upload`
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. App picks a file (SAF) → reads size + MIME.
|
||||||
|
2. App sends `media.upload.start {media_ref, kind, mime, size, filename}`.
|
||||||
|
3. App streams the file as **binary WS frames** (e.g. 256 KiB chunks).
|
||||||
|
4. App sends `media.upload.end {media_ref, sha256}`.
|
||||||
|
5. Plugin verifies size ≤ `max_upload_bytes` and sha256, writes to the media
|
||||||
|
cache via hermes `cache_*_from_bytes`:
|
||||||
|
- image → `cache_image_from_bytes`
|
||||||
|
- audio/voice → `cache_audio_from_bytes`
|
||||||
|
- video → `cache_video_from_bytes`
|
||||||
|
- document → `cache_document_from_bytes`
|
||||||
|
→ returns a local path.
|
||||||
|
6. The path is attached to the next `message.send` via `media_refs`, becoming
|
||||||
|
`MessageEvent.media_urls` + `media_types`
|
||||||
|
(`gateway/platforms/base.py:2337`). The agent's vision/audio tools can then
|
||||||
|
read the file.
|
||||||
|
|
||||||
|
**Limits:** `get_inbound_media_max_bytes()` / `validate_inbound_media_size`
|
||||||
|
(`base.py:758/779`) enforce the cap; over-limit → `error {code:"media_too_large"}`.
|
||||||
|
|
||||||
|
**Backpressure:** large uploads use the WS flow control; the plugin reads
|
||||||
|
binary frames into a temp file (not memory) to bound RAM.
|
||||||
|
|
||||||
|
## 7.3 Outbound (agent → app) — `media.offer` / `media.pull`
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Agent produces/references media (e.g. generates an image, or replies with a
|
||||||
|
`MEDIA:` tag / image URL). hermes base `extract_media` / `extract_images`
|
||||||
|
(`base.py:4439/4884`) pull these out and call the adapter's
|
||||||
|
`send_image / send_video / send_document / send_voice / send_image_file /
|
||||||
|
send_multiple_images`.
|
||||||
|
2. Adapter stages the file in the media cache, mints a `media_id`, and emits
|
||||||
|
`media.offer {media_id, kind, mime, size, filename}` (inside/with the
|
||||||
|
`message` frame's `media[]`).
|
||||||
|
3. App sends `media.pull {media_id}`.
|
||||||
|
4. Plugin streams the file as **binary WS frames**; ends with
|
||||||
|
`media.pull.end {ok:true}`.
|
||||||
|
5. App writes to its cache dir and hands the path to the player/viewer.
|
||||||
|
|
||||||
|
**Security:** `validate_media_delivery_path` (`base.py:1684`) + the media
|
||||||
|
delivery root/recency/denied-path checks (`base.py:1312-1480`) ensure the plugin
|
||||||
|
only serves files hermes is allowed to deliver (no arbitrary file read).
|
||||||
|
|
||||||
|
## 7.4 Live playback (AI-sent music/video)
|
||||||
|
|
||||||
|
**Requirement:** play AI-sent music and video **in the app**.
|
||||||
|
|
||||||
|
- **Audio (music/voice):** `ExoPlayer` (Media3). Inline player in the bubble
|
||||||
|
(play/pause, seek, duration); a persistent **mini-player** for music that
|
||||||
|
survives navigation. Voice notes play inline with a waveform.
|
||||||
|
- **Video:** `ExoPlayer` inline player (play/pause, seek, fullscreen, PiP on
|
||||||
|
Android). Streams from the local cache file after `media.pull`.
|
||||||
|
- **Documents/images:** image viewer (zoom) / open-with for documents (Android
|
||||||
|
`Intent.ACTION_VIEW` with a `FileProvider` URI; Desktop opens with the system
|
||||||
|
handler).
|
||||||
|
- **Desktop:** ExoPlayer is Android-only → the `MediaPlayer` expect/actual uses
|
||||||
|
a desktop backend (see `11-desktop-app.md`): a `libmpv`/`mpv`-backed surface
|
||||||
|
or a WebView fallback for video, and a desktop audio player for music.
|
||||||
|
|
||||||
|
## 7.5 Chunking parameters
|
||||||
|
|
||||||
|
- Chunk size: **256 KiB** (tunable).
|
||||||
|
- Binary frames carry raw bytes only; framing/metadata is in the JSON header +
|
||||||
|
end frames.
|
||||||
|
- Reassembly is ordered (WS preserves order); a gap/corruption → abort +
|
||||||
|
`error {code:"internal"}` + retry the whole transfer.
|
||||||
|
- `sha256` in `media.upload.end` / a size check on pull verify integrity.
|
||||||
|
|
||||||
|
## 7.6 App-side storage
|
||||||
|
|
||||||
|
- Cache dir: app-specific external cache (`getExternalCacheDir()/media`).
|
||||||
|
- LRU eviction by size (configurable, default 500 MB) so old media doesn't fill
|
||||||
|
the device.
|
||||||
|
- A `MediaRepository` tracks `{media_id, local_path, kind, size, ts}` in Room so
|
||||||
|
bubbles can re-render players after process death.
|
||||||
+105
@@ -0,0 +1,105 @@
|
|||||||
|
# 08 — Push Notifications, Outbox & Sync
|
||||||
|
|
||||||
|
The gateway can't reach a sleeping phone directly. Push goes through a cloud
|
||||||
|
relay. **Decision: FCM primary, ntfy fallback** (`ANDROID_PUSH_BACKEND`).
|
||||||
|
|
||||||
|
## 8.1 When push fires
|
||||||
|
|
||||||
|
- A frame targets a `chat_id` whose device is **disconnected** (WS closed) →
|
||||||
|
drop to **outbox** + fire **push**.
|
||||||
|
- Also fire push for high-priority foreground events the user should see even if
|
||||||
|
the app is backgrounded (approvals, clarifies, cron completions) — the app
|
||||||
|
decides whether to also show an in-app banner.
|
||||||
|
- If the device is **connected**, no push (the live frame is enough).
|
||||||
|
|
||||||
|
## 8.2 `PushBackend` interface (`push.py`)
|
||||||
|
|
||||||
|
```python
|
||||||
|
class PushBackend(Protocol):
|
||||||
|
name: str
|
||||||
|
async def send(self, *, device_id: str, chat_id: str,
|
||||||
|
title: str, body: str,
|
||||||
|
data: dict) -> bool: ...
|
||||||
|
def configured(self) -> bool: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Selected at adapter init by `ANDROID_PUSH_BACKEND` (`fcm` default, `ntfy`).
|
||||||
|
|
||||||
|
### 8.2.1 `FcmBackend` (primary)
|
||||||
|
- **FCM HTTP v1 API** via `httpx` (core dep). Auth = Firebase **service
|
||||||
|
account** (`ANDROID_FCM_SERVICE_ACCOUNT` JSON path) → mint a short-lived
|
||||||
|
OAuth2 access token (cached, refreshed before expiry).
|
||||||
|
- Fallback: legacy **server key** (`ANDROID_FCM_SERVER_KEY`) if no service
|
||||||
|
account (simpler, but legacy).
|
||||||
|
- Target = the device's **FCM token** (registered via `hello` /
|
||||||
|
`fcm.register`, stored in `devices.db`).
|
||||||
|
- Payload:
|
||||||
|
- `notification {title, body}` → system notification (tap → open app).
|
||||||
|
- `data {chat_id, thread_id, kind, message_id, cursor}` → app uses to `sync`.
|
||||||
|
- **Data-only option:** for background catch-up, send a data message (silent) so
|
||||||
|
the app's `FirebaseMessagingService` wakes a foreground service and syncs
|
||||||
|
without a visible notification (used for non-urgent updates).
|
||||||
|
- Batch: FCM allows up to 500 tokens/message; we send per-device (1 user, few
|
||||||
|
devices).
|
||||||
|
|
||||||
|
### 8.2.2 `NtfyBackend` (fallback, self-host friendly)
|
||||||
|
- Reuses hermes's existing ntfy publish path (hermes ships an ntfy adapter).
|
||||||
|
- Publish to `NTFY_TOPIC` on `NTFY_SERVER_URL` (default `https://ntfy.sh`) via
|
||||||
|
`httpx` POST, with an `X-Title` / `X-Message` / `X-Tag` / `X-Priority` and a
|
||||||
|
JSON `data` attachment (`{chat_id, thread_id, kind, cursor}`).
|
||||||
|
- The app subscribes to the topic (via an ntfy client lib or a lightweight
|
||||||
|
listener) to receive pushes without Firebase.
|
||||||
|
- Best for users who self-host ntfy and want zero Firebase.
|
||||||
|
|
||||||
|
## 8.3 Outbox (`outbox.py`)
|
||||||
|
|
||||||
|
- SQLite `outbox.db`: rows `{cursor (monotonic), chat_id, thread_id, frame_json,
|
||||||
|
ts, delivered}`.
|
||||||
|
- **Write** on every outbound frame that has no live subscriber (or always, then
|
||||||
|
mark delivered on live send — simpler + crash-safe).
|
||||||
|
- **Retention:** `outbox_retention_hours` (default 72h); prune on write.
|
||||||
|
- **Cursor:** global monotonic int; `hello.ack` returns the current cursor;
|
||||||
|
`sync {cursor}` replays rows with `cursor > given`.
|
||||||
|
- Bounded: if the outbox grows past a cap (e.g. 5k rows), oldest are pruned and
|
||||||
|
a `notification {kind:"generic", body:"Older messages pruned"}` is sent.
|
||||||
|
|
||||||
|
## 8.4 Sync (reconnect catch-up)
|
||||||
|
|
||||||
|
1. App reconnects → `hello` → `hello.ack {sync_cursor}`.
|
||||||
|
2. App sends `sync {cursor: <last seen>}`.
|
||||||
|
3. Server replays outbox frames (in cursor order) → app applies them (messages,
|
||||||
|
tools, channels, media offers).
|
||||||
|
4. Server sends `sync.done {cursor: <new>}`.
|
||||||
|
5. App updates its local cursor (Room) — idempotent (dedupe by `message_id`).
|
||||||
|
|
||||||
|
This makes the app **eventually consistent** across disconnects, restarts, and
|
||||||
|
gateway restarts.
|
||||||
|
|
||||||
|
## 8.5 App-side push handling (Android)
|
||||||
|
|
||||||
|
- **`FirebaseMessagingService.onMessageReceived`** (foreground): show in-app
|
||||||
|
banner + optionally sync.
|
||||||
|
- **`onNewToken`** → `fcm.register` the new token.
|
||||||
|
- **Background data message** → start a **foreground service** (low-priority,
|
||||||
|
notification channel "Sync") → open WS → `sync` → stop.
|
||||||
|
- **Notification channels** (Android 8+): one per chat so cron channels can have
|
||||||
|
their own style/priority (e.g. "Cron Reports" = high, "Default" = default).
|
||||||
|
Tapping a notification deep-links to the chat/thread.
|
||||||
|
- **ntfy mode:** a foreground service maintains the ntfy subscription; incoming
|
||||||
|
events trigger the same sync path.
|
||||||
|
|
||||||
|
## 8.6 In-app banners (foreground)
|
||||||
|
|
||||||
|
`notification` frames (e.g. `channel_renamed`, `approval`, `clarify`, `cron`)
|
||||||
|
render as a transient banner above the composer (like the reference screenshot's
|
||||||
|
"ARIA hat Thema … umbenannt"). Dismissible; high-priority ones (approval/clarify)
|
||||||
|
persist until acted on.
|
||||||
|
|
||||||
|
## 8.7 Security
|
||||||
|
|
||||||
|
- FCM tokens are per-device, revocable; stored only in `devices.db`.
|
||||||
|
- Push payloads carry **no secrets** and no full message bodies larger than a
|
||||||
|
short preview (privacy on lock screen). Full content is fetched via `sync`
|
||||||
|
over the authenticated WS.
|
||||||
|
- ntfy: use a **private topic + auth token** for any real trust boundary (hermes
|
||||||
|
ntfy adapter guidance).
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
# 09 — Pairing, Auth & Security
|
||||||
|
|
||||||
|
## 9.1 Threat model
|
||||||
|
|
||||||
|
- **Trusted domain:** a personal agent on the user's own machine; one owner, a
|
||||||
|
few of their own devices (phone + desktop).
|
||||||
|
- **Primary risks:** (a) an unauthorized device connecting to the WS and
|
||||||
|
reading/driving the agent; (b) eavesdropping on the WS in transit; (c) token
|
||||||
|
leakage in logs; (d) arbitrary file read via media pull.
|
||||||
|
- **Not addressing (v1):** multi-tenant isolation, adversarial multi-user abuse,
|
||||||
|
E2E encryption.
|
||||||
|
|
||||||
|
## 9.2 Pairing flow
|
||||||
|
|
||||||
|
1. **Generate a token.** `hermes gateway setup` (our `interactive_setup`) either
|
||||||
|
uses an existing `ANDROID_TOKEN` or generates a fresh high-entropy token
|
||||||
|
(e.g. 32 bytes → 64 hex chars) and stores it in `.env`.
|
||||||
|
2. **Present to the app.** Two options:
|
||||||
|
- **QR code:** the setup prints a QR encoding
|
||||||
|
`iris://pair?host=<lan-ip>&port=8790&token=<token>` (or a WSS URL). The
|
||||||
|
phone scans it with the app's camera (or a system scanner) → pre-fills
|
||||||
|
settings.
|
||||||
|
- **Manual:** user types the server URL + token in the app's Connect screen.
|
||||||
|
3. **App connects.** First WS frame is `hello {token, device_id, device_name,
|
||||||
|
caps, fcm_token?}`.
|
||||||
|
4. **Server verifies.** Constant-time compare of `token` vs `ANDROID_TOKEN`
|
||||||
|
(`hmac.compare_digest`). Optionally check `device_id` against
|
||||||
|
`ANDROID_ALLOWED_USERS` (if set) or `ANDROID_ALLOW_ALL_USERS`.
|
||||||
|
5. **On success:** register the device in `devices.db`, send `hello.ack`.
|
||||||
|
**On failure:** send `error {code:"auth"}` and close.
|
||||||
|
|
||||||
|
`device_id` is a stable, app-generated UUID (persisted in the app's
|
||||||
|
secure storage). It identifies the device for routing + push, **not** as a
|
||||||
|
security principal (the token is).
|
||||||
|
|
||||||
|
## 9.3 Auth model
|
||||||
|
|
||||||
|
- **Token = the security principal.** Any connection presenting the valid
|
||||||
|
`ANDROID_TOKEN` is authorized (it's the user's own token).
|
||||||
|
- **Allowlist (optional):** `ANDROID_ALLOWED_USERS` (comma-separated
|
||||||
|
`device_id`s) restricts which *devices* may connect even with the token —
|
||||||
|
useful if the token is shared. `ANDROID_ALLOW_ALL_USERS=true` disables the
|
||||||
|
allowlist (dev only).
|
||||||
|
- **Per-device tokens (stretch):** mint a unique token per device at pairing
|
||||||
|
(revocable) instead of one shared token. v1 uses the shared token + optional
|
||||||
|
device allowlist.
|
||||||
|
- **Re-pairing:** rotating `ANDROID_TOKEN` invalidates all devices; they must
|
||||||
|
re-pair. `hermes android pair` (stretch CLI) re-issues + prints a new QR.
|
||||||
|
|
||||||
|
## 9.4 Transport security
|
||||||
|
|
||||||
|
- **Default (LAN/dev):** plain `ws://` on the trusted LAN. Fine for a home
|
||||||
|
network.
|
||||||
|
- **WSS (recommended for remote):** set `ANDROID_WS_CERT` / `ANDROID_WS_KEY`
|
||||||
|
(self-signed or CA-signed). The app pins/accepts the cert (self-signed → user
|
||||||
|
confirms fingerprint on first pair, like a SSH host key).
|
||||||
|
- **Remote reachability options** (documented, user's choice):
|
||||||
|
- **Tailscale / WireGuard** (recommended): gateway gets a stable tailnet IP;
|
||||||
|
app connects over the private mesh. No public exposure.
|
||||||
|
- **Reverse proxy / tunnel** (Caddy, Cloudflare Tunnel, ngrok): terminate TLS
|
||||||
|
at the edge, forward WS to `127.0.0.1:8790`.
|
||||||
|
- **Public bind** (`0.0.0.0`) + WSS + strong token — last resort.
|
||||||
|
- The app stores the server URL + (for self-signed) the pinned cert fingerprint
|
||||||
|
in secure storage.
|
||||||
|
|
||||||
|
## 9.5 Secret & PII handling
|
||||||
|
|
||||||
|
- **Tokens/keys never logged.** Redact `ANDROID_TOKEN`, FCM tokens/keys, ntfy
|
||||||
|
tokens in all log output (hermes PII policy; `agent/redact.py` patterns).
|
||||||
|
- **`device_id`** is a random UUID (not PII). `device_name` is user-chosen.
|
||||||
|
- **Media pull** is gated by hermes `validate_media_delivery_path` + delivery
|
||||||
|
root/recency/denied-path checks (`gateway/platforms/base.py:1684`) — the
|
||||||
|
plugin can only serve files hermes is allowed to deliver (no arbitrary file
|
||||||
|
read).
|
||||||
|
- **Search** is local-only (user's own hermes home); no data leaves the machine.
|
||||||
|
|
||||||
|
## 9.6 Profile safety
|
||||||
|
|
||||||
|
- All plugin state lives under `get_hermes_home()/"android"` (profile-aware).
|
||||||
|
- Secrets are read with the scope-aware `_get_scoped_secret` pattern (see
|
||||||
|
`plugins/platforms/irc/adapter.py:42`) so multiplexed profiles don't leak
|
||||||
|
each other's tokens (fail-closed under `gateway.multiplex_profiles`).
|
||||||
|
- The WS bind uses a scoped lock (`gateway.status.acquire_scoped_lock`) so two
|
||||||
|
profiles can't bind the same port/identity.
|
||||||
|
|
||||||
|
## 9.7 Hardening checklist
|
||||||
|
|
||||||
|
- [ ] Constant-time token compare.
|
||||||
|
- [ ] Bounded per-connection send buffer + rate limit on inbound frames.
|
||||||
|
- [ ] Reject oversized frames / uploads (`max_upload_bytes`).
|
||||||
|
- [ ] Verify media sha256 + re-sniff MIME (don't trust client).
|
||||||
|
- [ ] Redact all secrets in logs.
|
||||||
|
- [ ] WSS + cert pinning for remote.
|
||||||
|
- [ ] Outbox retention cap + prune.
|
||||||
|
- [ ] Fail-closed secret reads under multiplexing.
|
||||||
@@ -0,0 +1,197 @@
|
|||||||
|
# 10 — Android App (Kotlin + Jetpack Compose)
|
||||||
|
|
||||||
|
Native client. Lives in the Compose Multiplatform project at `app/`; the bulk of
|
||||||
|
the code is in `app/shared` (commonMain) so the Desktop app reuses it.
|
||||||
|
|
||||||
|
## 10.1 Tech stack
|
||||||
|
|
||||||
|
| Concern | Choice |
|
||||||
|
|---|---|
|
||||||
|
| Language | Kotlin |
|
||||||
|
| UI | Jetpack Compose (Material 3), Compose Navigation |
|
||||||
|
| Async | Kotlinx Coroutines + Flow |
|
||||||
|
| WS client | OkHttp (`WebSocketListener`) |
|
||||||
|
| JSON | kotlinx-serialization |
|
||||||
|
| Local DB | **SQLDelight** (KMP; channels, messages cache, media index, sync cursor, settings) |
|
||||||
|
| Media playback | Media3 **ExoPlayer** (audio + video) |
|
||||||
|
| Push | Firebase Messaging (FCM) [primary] / ntfy listener [fallback] |
|
||||||
|
| DI | Hilt |
|
||||||
|
| Images | Coil |
|
||||||
|
| minSdk / target | **26** / 34 (test device: MIX 2S, API 29) |
|
||||||
|
|
||||||
|
## 10.2 Module layout
|
||||||
|
|
||||||
|
```
|
||||||
|
app/shared/src/
|
||||||
|
├── commonMain/kotlin/iris/
|
||||||
|
│ ├── protocol/ # frame data classes (mirror of gateway protocol.py)
|
||||||
|
│ ├── net/ # GatewayClient (OkHttp WS), reconnect, heartbeat, dispatch
|
||||||
|
│ ├── data/ # ChannelRepository, MessageRepository, MediaRepository,
|
||||||
|
│ │ # SearchRepository, SettingsRepository (SQLDelight + WS)
|
||||||
|
│ ├── state/ # ViewModels: ChatListVM, ChatVM, ComposerVM, SettingsVM
|
||||||
|
│ ├── ui/
|
||||||
|
│ │ ├── theme/ # Material 3 theme (dark default), type, color
|
||||||
|
│ │ ├── components/ # MessageBubble, ReasoningBlock, ToolCard, MediaPlayer,
|
||||||
|
│ │ │ # ChannelRow, Composer, PickerSheet, SearchBar, Banner
|
||||||
|
│ │ └── screens/ # ConnectScreen, ChatListScreen, ChatScreen,
|
||||||
|
│ │ # SearchScreen, SettingsScreen, ChannelMenu
|
||||||
|
│ └── util/ # time formatting, markdown, id gen
|
||||||
|
├── androidMain/kotlin/iris/
|
||||||
|
│ ├── platform/ # MediaPlayer actual (ExoPlayer), MediaPicker (SAF),
|
||||||
|
│ │ # Notifications, SecureStore (EncryptedSharedPreferences)
|
||||||
|
│ ├── fcm/ # FirebaseMessagingService, foreground sync service
|
||||||
|
│ └── IrisApp.kt # Application (Hilt), notification channels
|
||||||
|
└── androidApp/ # MainActivity, AndroidManifest, res, google-services
|
||||||
|
```
|
||||||
|
|
||||||
|
## 10.3 `GatewayClient` (net)
|
||||||
|
|
||||||
|
- OkHttp `WebSocket` to `ws(s)://host:port/ws`.
|
||||||
|
- **Reconnect:** exponential backoff + jitter; on reconnect send `hello` then
|
||||||
|
`sync {cursor}` (replays undelivered outbox only).
|
||||||
|
- **Initial channel open:** on first open of a chat/thread, send `history`
|
||||||
|
(newest page) to populate the view. `sync` does NOT load history.
|
||||||
|
- **Heartbeat:** send `ping` every 20s; reap on missed `pong` (3×) → reconnect.
|
||||||
|
- **Request/response:** map of `id → CompletableDeferred`; events → a
|
||||||
|
`SharedFlow<Frame>` consumed by repositories.
|
||||||
|
- **Backpressure:** collect frames on a bounded channel; coalesce
|
||||||
|
`message.update` per `message_id` (keep latest) to avoid flooding the UI.
|
||||||
|
- **Thread:** all frame emission on a single dispatcher; UI observes via Flow.
|
||||||
|
|
||||||
|
## 10.4 State rules (from hermes desktop AGENTS.md — apply here)
|
||||||
|
|
||||||
|
- **Server is authoritative** for channels/messages/cursor; the app's Room copy
|
||||||
|
is a **cache**. Reconcile (merge, don't clobber) on `channel.*`/`sync`.
|
||||||
|
- **Optimistic then honest:** send a message → show it immediately (pending),
|
||||||
|
roll back visibly on `error`, authoritative `sync` gets the last word.
|
||||||
|
- **Guard against the past:** generation counters / request tokens so a stale
|
||||||
|
response never overwrites newer intent.
|
||||||
|
- **Isolate the foreground:** only the visible chat publishes into the shared
|
||||||
|
view; background chats update their own cache quietly.
|
||||||
|
- **Coalesce noise, flush signal:** batch cosmetic updates; let terminal
|
||||||
|
transitions (turn done, needs input, failed) reach the user immediately.
|
||||||
|
|
||||||
|
## 10.5 Feature implementation (your checklist)
|
||||||
|
|
||||||
|
### Input box, auto-grow (max height)
|
||||||
|
- Compose `BasicTextField` inside a `Box` with
|
||||||
|
`Modifier.heightIn(min = 1.line, max = 160.dp)`. Grows with lines, caps at
|
||||||
|
160dp, then scrolls internally.
|
||||||
|
- Bottom bar (matches reference): `Menü` button (left), emoji button, the
|
||||||
|
auto-grow field, attach (paperclip), mic/send (right). Send on Enter
|
||||||
|
(configurable: Enter=send vs Enter=newline).
|
||||||
|
|
||||||
|
### Menu button → all slash commands
|
||||||
|
- `Menü` opens a **bottom sheet** listing the command catalog. The catalog is
|
||||||
|
served by the gateway via `commands.catalog` request → response with
|
||||||
|
`{name, description, args_hint, category}` per command, so it always matches
|
||||||
|
hermes (`/new`, `/model`, `/reasoning`, `/status`, `/cron`, `/tools`, …).
|
||||||
|
- Typing `/` in the field shows **autocomplete** via `commands.complete`
|
||||||
|
request (gateway matches the typed prefix). Selecting inserts `/cmd `.
|
||||||
|
- Selecting a command sends `message.send {text:"/cmd args"}`.
|
||||||
|
- **Interactive commands** (`/model`, `/reasoning`, `/fast`, approvals,
|
||||||
|
clarifies) render as **native pickers** from `picker.*` frames (a dialog /
|
||||||
|
sheet with the options; answer via `picker.select`).
|
||||||
|
|
||||||
|
### Tool output (app-controlled verbosity)
|
||||||
|
- `ToolCard` renders `tool.start/progress/end` frames.
|
||||||
|
- **Settings → "Tool detail": Everything / Truncated / Nothing.**
|
||||||
|
- Everything: name + full args (collapsible) + output preview.
|
||||||
|
- Truncated (default): `emoji name: "short preview"` one-liner, expandable.
|
||||||
|
- Nothing: suppress tool frames.
|
||||||
|
- Spinner while running; ✓/✗ + duration on `tool.end`.
|
||||||
|
|
||||||
|
### Reasoning before message
|
||||||
|
- `ReasoningBlock` (collapsible, "💭 Reasoning" header, monospace body, **copy**
|
||||||
|
button) rendered **above** the message body from the `reasoning` field.
|
||||||
|
Matches the reference screenshot.
|
||||||
|
|
||||||
|
### Intermediate messages
|
||||||
|
- `commentary` frames → dimmed/smaller bubble, distinct from final answers.
|
||||||
|
|
||||||
|
### Agent busy / stop / steer
|
||||||
|
- `agent.busy` → show "thinking…" indicator in chat header (animated dots).
|
||||||
|
- `agent.idle` → clear indicator.
|
||||||
|
- **Stop button** (appears in header while busy): sends `agent.stop`.
|
||||||
|
- **Steer:** while busy, the composer accepts input; sending it calls
|
||||||
|
`agent.steer` (injects mid-turn) rather than queuing a new `message.send`.
|
||||||
|
|
||||||
|
### Threading + channels
|
||||||
|
- **Channel list** (drawer on single-pane; left rail on two-pane) = default chat
|
||||||
|
+ user channels (avatar, name, last-message preview, unread badge, active
|
||||||
|
highlight bar) — matches the reference left sidebar.
|
||||||
|
- **Thread toggle** in the default chat header: "Threads on/off". On → topic
|
||||||
|
switcher above the message list (each topic = a `thread_id`).
|
||||||
|
- **New channel** (FAB / channel-list menu) → `channel.create` → appears in list;
|
||||||
|
menu offers "Set as cron target".
|
||||||
|
|
||||||
|
### Search
|
||||||
|
- Search bar (chat header or top) with a **scope toggle**: "Search everywhere" /
|
||||||
|
"Search in this chat/channel". → `search` frame → results list → tap jumps to
|
||||||
|
the message (navigate + highlight).
|
||||||
|
|
||||||
|
### Attach media
|
||||||
|
- Paperclip → system pickers (Photos / Files / Audio / Video / Docs) via SAF.
|
||||||
|
- Selected files show as **preview chips** in the composer (thumbnail + name +
|
||||||
|
remove). On send: `media.upload` (chunked) for each, then `message.send` with
|
||||||
|
`media_refs`.
|
||||||
|
|
||||||
|
### Voice input (mic button)
|
||||||
|
- Mic button (right of composer, toggles to send when text is present).
|
||||||
|
- Tap → request `RECORD_AUDIO` permission → start recording (MediaRecorder,
|
||||||
|
OGG/Opus, 44.1 kHz mono).
|
||||||
|
- While recording: timer + waveform; tap again to stop.
|
||||||
|
- On stop: file becomes a **preview chip** (audio, kind=`voice`) in the
|
||||||
|
composer, same as attached media. Send → `media.upload` (chunked) →
|
||||||
|
`message.send` with `media_refs`.
|
||||||
|
- Hermes receives it as an audio attachment; the agent's STT (if configured)
|
||||||
|
transcribes it. No client-side STT.
|
||||||
|
|
||||||
|
### Push notifications
|
||||||
|
- FCM service (see `08-push.md`): foreground banner + background foreground
|
||||||
|
service → `sync`. Notification channel per chat. Tap → deep-link to chat.
|
||||||
|
- ntfy fallback: foreground service maintains the subscription.
|
||||||
|
|
||||||
|
### Live playback
|
||||||
|
- AI-sent audio/video → `media.pull` → cache file → **ExoPlayer** inline player
|
||||||
|
(audio: mini-player; video: inline + fullscreen + PiP). Documents/images →
|
||||||
|
viewer / open-with.
|
||||||
|
|
||||||
|
## 10.6 Layout (Telegram-style, per reference image)
|
||||||
|
|
||||||
|
**Two layout modes** (decision: user-toggleable, **single-pane default**):
|
||||||
|
- **Single-pane (default on phones):** chat full-screen; channel list in a
|
||||||
|
swipeable drawer (hamburger / edge swipe).
|
||||||
|
- **Two-pane (Telegram-style, like the reference):** persistent left channel
|
||||||
|
rail + chat. Auto-enabled on tablets / large screens; toggleable in Settings.
|
||||||
|
|
||||||
|
**Chat screen anatomy (matches reference):**
|
||||||
|
- **Header:** back (single-pane), avatar, name + "Bot" subtitle, edit + overflow
|
||||||
|
(⋮) menu (thread toggle, channel menu, set cron target, clear).
|
||||||
|
- **Message list:** date separators ("7. August"); user bubbles **right**
|
||||||
|
(accent color, ✓✓ read receipts); agent bubbles **left** (surface color) with
|
||||||
|
optional ReasoningBlock + ToolCards + media + model/token footer
|
||||||
|
("Qwen3-… · 11% · ~") + timestamp.
|
||||||
|
- **In-app banner** above the composer (e.g. "renamed topic").
|
||||||
|
- **Composer:** `Menü` / emoji / auto-grow input / attach / mic-send.
|
||||||
|
|
||||||
|
**Theme:** dark by default (reference is dark); Material 3; optional dynamic
|
||||||
|
color. Accent = user's chosen brand color (default indigo, like the reference).
|
||||||
|
|
||||||
|
## 10.7 SQLDelight schema (cache)
|
||||||
|
|
||||||
|
- `channels(chat_id PK, name, kind, parent_chat_id, is_default, last_preview,
|
||||||
|
last_ts, unread)`.
|
||||||
|
- `messages(id PK, chat_id, thread_id, role, text, reasoning, model, tokens,
|
||||||
|
ts, status[pending|sent|read], media_json)`.
|
||||||
|
- `media(media_id PK, local_path, kind, mime, size, ts)`.
|
||||||
|
- `meta(key PK, value)` — sync cursor, settings, device_id, server url, pinned
|
||||||
|
cert fingerprint.
|
||||||
|
|
||||||
|
## 10.8 Onboarding / Connect screen
|
||||||
|
|
||||||
|
- First launch → **Connect**: server URL + token (or scan QR). "Test connection"
|
||||||
|
does a real `hello` (not just a TCP probe — per hermes desktop guidance, the
|
||||||
|
auth leg must be exercised). On success → save (secure storage) → main.
|
||||||
|
- States: connecting / connected / reconnecting / degraded / auth-failed — each
|
||||||
|
with honest copy and a way out.
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
# 11 — Desktop App (Kotlin + Compose Multiplatform)
|
||||||
|
|
||||||
|
The desktop app is **the Android app, tweaked for a big screen**. It reuses the
|
||||||
|
entire `app/shared` module (protocol, network, repositories, state, most UI) and
|
||||||
|
only adds desktop platform services + a wider default layout.
|
||||||
|
|
||||||
|
## 11.1 What's shared vs desktop-specific
|
||||||
|
|
||||||
|
| Layer | Shared (commonMain) | Desktop-specific (desktopMain) |
|
||||||
|
|---|---|---|
|
||||||
|
| Protocol / WS client | ✅ | — |
|
||||||
|
| Repositories / state | ✅ | — |
|
||||||
|
| Most Compose UI | ✅ | layout tweaks, keyboard shortcuts |
|
||||||
|
| Push | — | **tray icon + OS notifications** (no FCM) |
|
||||||
|
| Media playback | `MediaPlayer` interface | **desktop player** actual |
|
||||||
|
| File picker | `MediaPicker` interface | **desktop file dialog** actual |
|
||||||
|
| Window | — | resizable window, always-on-top, global hotkey |
|
||||||
|
| Secure store | `SecureStore` interface | keyring / encrypted file actual |
|
||||||
|
|
||||||
|
## 11.2 `desktopMain` platform services
|
||||||
|
|
||||||
|
- **Push → tray + OS notifications.** Desktop is assumed reachable (persistent
|
||||||
|
WS), so no FCM. A **system tray icon** shows connection state + unread count;
|
||||||
|
OS notifications (Java Desktop / `SystemTray` + a cross-platform notifier)
|
||||||
|
fire for background chats / approvals / cron. Clicking focuses the window and
|
||||||
|
deep-links to the chat.
|
||||||
|
- **Media playback → `MediaPlayer` actual.** ExoPlayer is Android-only. Desktop
|
||||||
|
uses a `libmpv`/`mpv`-backed Compose surface for video (and audio), with a
|
||||||
|
WebView-based fallback if `mpv` isn't available. Same `MediaPlayer` interface
|
||||||
|
the Android `ExoPlayer` actual implements, so UI code is identical.
|
||||||
|
- **File picker → `MediaPicker` actual.** A native file dialog (Compose
|
||||||
|
Desktop `FileChooser` / AWT `JFileChooser`) returning local file paths, then
|
||||||
|
the same `media.upload` chunked path.
|
||||||
|
- **Window.** Resizable, remembers size/position, optional always-on-top, a
|
||||||
|
global hotkey to focus/show. Minimize-to-tray option.
|
||||||
|
- **Secure store → `SecureStore` actual.** OS keychain (macOS Keychain, Linux
|
||||||
|
Secret Service / encrypted file, Windows Credential Manager) or an encrypted
|
||||||
|
file for the token + pinned cert.
|
||||||
|
|
||||||
|
## 11.3 Layout (big-screen tweaks)
|
||||||
|
|
||||||
|
- **Two-pane is the default** (persistent left channel rail + chat), since there
|
||||||
|
is width. Single-pane still available via the same toggle.
|
||||||
|
- **Wider chat column** with comfortable max line length; optional **third
|
||||||
|
pane** (inspector: session info, model, tokens, tool detail, channel settings).
|
||||||
|
- **Larger type scale** and spacing for desktop; hover states; mouse + keyboard
|
||||||
|
first.
|
||||||
|
- **Keyboard shortcuts:**
|
||||||
|
- `/` focus the composer and open the command menu.
|
||||||
|
- `Ctrl/Cmd+K` command palette (all slash commands + actions).
|
||||||
|
- `Ctrl/Cmd+N` new channel; `Ctrl/Cmd+T` new thread.
|
||||||
|
- `Ctrl/Cmd+F` search (with the all/this-chat scope toggle).
|
||||||
|
- `↑/↓` navigate channel list; `Enter` open.
|
||||||
|
- `Esc` close sheet/dialog (one cancel gesture does exactly one thing).
|
||||||
|
- **Multi-window (stretch):** open a channel in a separate window / pop-out.
|
||||||
|
|
||||||
|
## 11.4 Distribution
|
||||||
|
|
||||||
|
- **Compose Desktop** → native binaries via **jpackage** (or a plain app image):
|
||||||
|
Linux (.deb/.AppImage), macOS (.dmg), Windows (.msi/.exe).
|
||||||
|
- Bundles the JRE. Auto-update is a stretch goal (v1: manual download).
|
||||||
|
- The desktop app connects to the **same** gateway WS server as the phone (the
|
||||||
|
user's home server / Tailscale). It does **not** spawn its own backend (unlike
|
||||||
|
hermes's existing Electron desktop, which spawns `hermes serve`) — our
|
||||||
|
desktop is a pure client of the messaging gateway, matching the Android app.
|
||||||
|
|
||||||
|
## 11.5 Parity checklist (same functionality as Android)
|
||||||
|
|
||||||
|
- [ ] Streaming, reasoning block, tool cards (3-level verbosity), commentary.
|
||||||
|
- [ ] Channels + threads + new channel + cron target.
|
||||||
|
- [ ] Search (all / this-chat).
|
||||||
|
- [ ] Media attach + live playback (via desktop player).
|
||||||
|
- [ ] Slash command menu + autocomplete + interactive pickers.
|
||||||
|
- [ ] Push (tray + OS notifications) + outbox/sync.
|
||||||
|
- [ ] Pairing/Connect screen (URL + token, WSS cert pin).
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# 12 — Toolchain Setup
|
||||||
|
|
||||||
|
First-time setup on a machine (verified baseline: CachyOS/Arch, `pacman`,
|
||||||
|
`uv` present, ADB present, no JDK/SDK/Gradle).
|
||||||
|
|
||||||
|
## 12.1 JDK 17
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pacman -S jdk17-openjdk
|
||||||
|
java -version # expect 17.x
|
||||||
|
```
|
||||||
|
(Compose Multiplatform + current AGP are happy on JDK 17. Use 17 to match the
|
||||||
|
Android toolchain; 21 also works but 17 is the safe floor.)
|
||||||
|
|
||||||
|
## 12.2 Android SDK
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# cmdline-tools
|
||||||
|
mkdir -p ~/android-sdk/cmdline-tools
|
||||||
|
cd ~/android-sdk/cmdline-tools
|
||||||
|
curl -O https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip
|
||||||
|
unzip commandlinetools-linux-*.zip && mv cmdline-tools latest
|
||||||
|
rm commandlinetools-linux-*.zip
|
||||||
|
|
||||||
|
export ANDROID_HOME=$HOME/android-sdk
|
||||||
|
export PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools
|
||||||
|
|
||||||
|
sdkmanager --licenses
|
||||||
|
sdkmanager "platform-tools" "platforms;android-34" "build-tools;34.0.0"
|
||||||
|
```
|
||||||
|
Persist `ANDROID_HOME`/`PATH` in `~/.bashrc`. ADB is already installed system-wide;
|
||||||
|
`platform-tools` from the SDK is fine too (whichever is first on `PATH`).
|
||||||
|
|
||||||
|
Create `app/local.properties`:
|
||||||
|
```
|
||||||
|
sdk.dir=/home/<you>/android-sdk
|
||||||
|
```
|
||||||
|
|
||||||
|
## 12.3 Gradle
|
||||||
|
|
||||||
|
No system install — use the project wrapper:
|
||||||
|
```bash
|
||||||
|
cd app
|
||||||
|
./gradlew tasks # first run downloads the wrapper distribution
|
||||||
|
```
|
||||||
|
(The wrapper version is pinned in `app/gradle/wrapper/gradle-wrapper.properties`.)
|
||||||
|
|
||||||
|
## 12.4 hermes environment (for the plugin + running the gateway)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd hermes-agent
|
||||||
|
uv sync # creates .venv with all core deps (websockets, httpx, …)
|
||||||
|
source .venv/bin/activate
|
||||||
|
hermes --version # sanity
|
||||||
|
```
|
||||||
|
- Run the gateway with the plugin:
|
||||||
|
```bash
|
||||||
|
# install the plugin (dev: symlink)
|
||||||
|
mkdir -p ~/.hermes/plugins
|
||||||
|
ln -s "$PWD/../gateway-plugin" ~/.hermes/plugins/android
|
||||||
|
hermes gateway status # should list "android"
|
||||||
|
hermes gateway # run
|
||||||
|
```
|
||||||
|
- Tests use hermes's hermetic runner (never bare `pytest`):
|
||||||
|
```bash
|
||||||
|
scripts/run_tests.sh tests/gateway/test_android.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## 12.5 Firebase (FCM) — primary push
|
||||||
|
|
||||||
|
1. Create a Firebase project (console.firebase.google.com).
|
||||||
|
2. Add an **Android app** (package = `androidApp` applicationId, e.g.
|
||||||
|
`dev.iris.app`). Download `google-services.json` → `app/androidApp/`.
|
||||||
|
3. Create a **service account** (Project settings → Service accounts → Generate
|
||||||
|
new private key) → download the JSON. Store its path in
|
||||||
|
`ANDROID_FCM_SERVICE_ACCOUNT` (in `~/.hermes/.env`).
|
||||||
|
4. The app's `FirebaseMessagingService` obtains the FCM token at runtime and
|
||||||
|
registers it via `hello` / `fcm.register`.
|
||||||
|
|
||||||
|
> Skip Firebase → set `ANDROID_PUSH_BACKEND=ntfy` and configure `NTFY_TOPIC` /
|
||||||
|
> `NTFY_SERVER_URL` (self-host ntfy or use ntfy.sh). See `08-push.md`.
|
||||||
|
|
||||||
|
## 12.6 Environment variables (summary)
|
||||||
|
|
||||||
|
**Secrets (`~/.hermes/.env`):**
|
||||||
|
```
|
||||||
|
ANDROID_TOKEN=<64-hex>
|
||||||
|
ANDROID_PUSH_BACKEND=fcm # or ntfy
|
||||||
|
ANDROID_FCM_SERVICE_ACCOUNT=/path/to/service-account.json
|
||||||
|
# ANDROID_FCM_SERVER_KEY=<legacy key> # fallback if no service account
|
||||||
|
# NTFY_TOPIC=iris-push # when ntfy
|
||||||
|
# NTFY_SERVER_URL=https://ntfy.sh
|
||||||
|
# ANDROID_WS_CERT=/path/cert.pem # WSS
|
||||||
|
# ANDROID_WS_KEY=/path/key.pem
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavioral (`~/.hermes/config.yaml`):**
|
||||||
|
```yaml
|
||||||
|
gateway:
|
||||||
|
platforms:
|
||||||
|
android:
|
||||||
|
enabled: true
|
||||||
|
extra:
|
||||||
|
host: 127.0.0.1 # 0.0.0.0 for LAN
|
||||||
|
port: 8790
|
||||||
|
home_channel: android:default
|
||||||
|
push_backend: fcm
|
||||||
|
outbox_retention_hours: 72
|
||||||
|
max_upload_bytes: 104857600 # 100 MB
|
||||||
|
display:
|
||||||
|
platforms:
|
||||||
|
android:
|
||||||
|
show_reasoning: true
|
||||||
|
reasoning_style: code
|
||||||
|
streaming: true
|
||||||
|
tool_progress: all # gateway sends full data; app controls display
|
||||||
|
```
|
||||||
|
|
||||||
|
## 12.7 Verify the stack (smoke test)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. gateway up with plugin
|
||||||
|
hermes gateway status | grep -i android
|
||||||
|
|
||||||
|
# 2. a raw WS client can pair + echo
|
||||||
|
python - <<'PY'
|
||||||
|
import asyncio, json, websockets
|
||||||
|
async def main():
|
||||||
|
async with websockets.connect("ws://127.0.0.1:8790/ws") as ws:
|
||||||
|
await ws.send(json.dumps({"v":1,"type":"hello","payload":{
|
||||||
|
"token":"<ANDROID_TOKEN>","device_id":"test","device_name":"probe",
|
||||||
|
"caps":{"min_protocol":1}}}))
|
||||||
|
print("recv:", await ws.recv())
|
||||||
|
asyncio.run(main())
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expect a `hello.ack`. If you get `error {code:"auth"}`, the token/host/port is
|
||||||
|
wrong.
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
# 13 — Testing & Debugging (incl. ADB on-device)
|
||||||
|
|
||||||
|
Three layers: **Python plugin tests**, **Kotlin unit/UI tests**, and **on-device
|
||||||
|
E2E via ADB**. Plus a **WS test-client harness** to drive the real gateway
|
||||||
|
without the app (critical for verifying frame shapes early).
|
||||||
|
|
||||||
|
## 13.1 Python plugin tests
|
||||||
|
|
||||||
|
- Location: `gateway-plugin/tests/` (and, for hermes-integration tests, mirror
|
||||||
|
into the hermes `tests/gateway/test_android.py` pattern when running under
|
||||||
|
hermes's suite).
|
||||||
|
- **Run with hermes's hermetic runner** (never bare `pytest`):
|
||||||
|
```bash
|
||||||
|
cd hermes-agent
|
||||||
|
scripts/run_tests.sh tests/gateway/test_android.py
|
||||||
|
scripts/run_tests.sh # full suite (CI parity)
|
||||||
|
```
|
||||||
|
- Coverage to write (behavioral, not change-detector — per hermes test policy):
|
||||||
|
- `register(ctx)` produces a valid `PlatformEntry` (name, cron env var,
|
||||||
|
parse_target_ref).
|
||||||
|
- `check_requirements` / `validate_config` / `is_connected` truth table.
|
||||||
|
- `_parse_target_ref`: `android:<chat>`, `android:<chat>:<thread>`, non-android
|
||||||
|
→ None.
|
||||||
|
- **Reasoning split:** given a `show_reasoning`-style final text, `send()`
|
||||||
|
emits `message {reasoning, text}` correctly; no-prefix → no reasoning field.
|
||||||
|
- **Outbox:** frame with no subscriber → written with a cursor; `sync` replays
|
||||||
|
the right range; retention prunes.
|
||||||
|
- **Pairing:** valid token → `hello.ack`; wrong token → `error auth` + close.
|
||||||
|
Constant-time compare used.
|
||||||
|
- **Media:** upload start→binary→end reassembles + sha256 verified; over-limit
|
||||||
|
→ `media_too_large`; pull serves only allowed paths.
|
||||||
|
- **Push backend selection:** fcm vs ntfy chosen by config; `configured()`
|
||||||
|
reflects missing creds.
|
||||||
|
- **Channel directory:** create/rename/set_default; cron target resolution.
|
||||||
|
- **No `~/.hermes` writes in tests** — use the `_isolate_hermes_home` fixture
|
||||||
|
pattern (temp `HERMES_HOME`). Profile tests also mock `Path.home()`.
|
||||||
|
|
||||||
|
## 13.2 WS test-client harness (do this FIRST, in M1/M2)
|
||||||
|
|
||||||
|
A small Python script (`gateway-plugin/tests/ws_probe.py`) that connects to the
|
||||||
|
**real running gateway** and drives a turn, printing every frame. This is how we
|
||||||
|
**empirically confirm** the exact frame shapes (especially tool-progress vs
|
||||||
|
commentary classification and the reasoning prefix) before/while building the
|
||||||
|
Kotlin client.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes gateway & # with the android plugin
|
||||||
|
python gateway-plugin/tests/ws_probe.py --token <ANDROID_TOKEN> \
|
||||||
|
--send "list the files and summarize"
|
||||||
|
# prints: hello.ack, typing, message.start, message.update…, tool.start, tool.end,
|
||||||
|
# commentary, message.stop {reasoning,…}, …
|
||||||
|
```
|
||||||
|
Use it to lock `04-wire-protocol.md` against reality and to debug the adapter
|
||||||
|
without waiting for the app.
|
||||||
|
|
||||||
|
## 13.3 Kotlin unit / UI tests
|
||||||
|
|
||||||
|
- **Protocol codec:** round-trip every frame type (serialize → deserialize →
|
||||||
|
equal); unknown `type`/fields ignored (forward-compat).
|
||||||
|
- **Repositories:** merge-don't-clobber on `channel.*`; optimistic send + rollback
|
||||||
|
on `error`; sync dedupe by `message_id`; cursor monotonic.
|
||||||
|
- **`GatewayClient`:** reconnect/backoff; `message.update` coalescing (latest
|
||||||
|
wins); request/response correlation by `id`.
|
||||||
|
- **ViewModels:** state transitions (connecting→connected→reconnecting); tool
|
||||||
|
verbosity filtering (everything/truncated/nothing); reasoning present/absent.
|
||||||
|
- **Compose UI tests:** ReasoningBlock collapse/expand + copy; ToolCard
|
||||||
|
spinner→done; composer auto-grow cap; channel list active highlight.
|
||||||
|
- Run: `./gradlew :shared:testDebugUnitTest` (and `:shared:testDesktopTest`).
|
||||||
|
|
||||||
|
## 13.4 On-device E2E via ADB (the MIX 2S, API 29)
|
||||||
|
|
||||||
|
Device: `a5ca2a4b` (Xiaomi MIX 2S). Workflow:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# install + launch
|
||||||
|
cd app
|
||||||
|
./gradlew :androidApp:installDebug
|
||||||
|
adb shell am start -n dev.iris.app/.MainActivity
|
||||||
|
|
||||||
|
# logs (filter our tags + WS + FCM + ExoPlayer)
|
||||||
|
adb logcat -c
|
||||||
|
adb logcat | grep -Ei "iris|GatewayClient|Firebase|ExoPlayer|MediaCodec"
|
||||||
|
|
||||||
|
# screenshots for visual checks
|
||||||
|
adb exec-out screencap -p > /tmp/shot.png
|
||||||
|
|
||||||
|
# clear app data between pairing attempts
|
||||||
|
adb shell pm clear dev.iris.app
|
||||||
|
|
||||||
|
# push a file to the app's cache (for media tests) / pull logs
|
||||||
|
adb shell run-as dev.iris.app ls files
|
||||||
|
adb logcat -d > /tmp/logcat.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
**E2E scenarios (script where possible):**
|
||||||
|
1. **Pair:** connect screen → enter URL+token → `hello.ack` → main. (Verify auth
|
||||||
|
leg, not just TCP.)
|
||||||
|
2. **Text round-trip:** send "hello" → streamed reply appears (message.start →
|
||||||
|
updates → stop).
|
||||||
|
3. **Reasoning:** ask a reasoning-model question → ReasoningBlock shows above the
|
||||||
|
answer; copy button works.
|
||||||
|
4. **Tools:** trigger a tool (e.g. "list files") → ToolCard shows; toggle
|
||||||
|
verbosity in Settings → rendering changes.
|
||||||
|
5. **Intermediate:** a multi-step prompt → commentary bubble appears dimmed.
|
||||||
|
6. **Channels:** create "Cron Reports" → appears in list; set as cron target.
|
||||||
|
7. **Cron delivery:** create a cron job `deliver=android:android:chan_<n>` → it
|
||||||
|
fires → lands in that channel (not default).
|
||||||
|
8. **Search:** "search everywhere" vs "this chat" → correct scoping; tap → jump.
|
||||||
|
9. **Media (in):** attach a photo + a video → agent receives (vision) → reply.
|
||||||
|
10. **Media (out):** ask the agent to send an image/video → `media.offer` →
|
||||||
|
inline player plays it live.
|
||||||
|
11. **Push:** background the app (`adb shell am start` another app / lock) →
|
||||||
|
trigger a message → FCM notification appears → tap → syncs + deep-links.
|
||||||
|
12. **Reconnect/sync:** kill the WS (stop gateway briefly) → restart → app
|
||||||
|
reconnects → `sync` catches up (no lost/dup messages).
|
||||||
|
|
||||||
|
## 13.5 Debugging tips
|
||||||
|
|
||||||
|
- **Gateway side:** `~/.hermes/logs/gateway.log` (and `hermes logs --follow`).
|
||||||
|
Our plugin logs under the `android` adapter name; secrets redacted.
|
||||||
|
- **WS framing bugs:** use the `ws_probe.py` harness — it isolates the protocol
|
||||||
|
from the app.
|
||||||
|
- **Streaming jitter:** the consumer edits at intervals; if updates look chunky,
|
||||||
|
check `display.platforms.android.streaming` and the consumer's edit interval.
|
||||||
|
- **Media pull stalls:** check chunk size + backpressure; confirm the file is
|
||||||
|
within hermes delivery roots (`validate_media_delivery_path`).
|
||||||
|
- **FCM not arriving:** confirm the token registered (`devices.db`), the service
|
||||||
|
account can mint a token, and the app's `onNewToken` re-registered after
|
||||||
|
`pm clear`.
|
||||||
|
- **Profile leaks:** if tokens look wrong under multiple profiles, verify the
|
||||||
|
scope-aware secret read (`_get_scoped_secret`) is used.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# 14 — Milestones (M0–M7)
|
||||||
|
|
||||||
|
Phased delivery. Each milestone ends with a **demo** (on-device where noted) and
|
||||||
|
has explicit **acceptance criteria**. Work top-to-bottom; don't skip M0/M1.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M0 — Toolchain & scaffolding
|
||||||
|
**Goal:** everything builds; the plugin is discoverable; the repo is safe.
|
||||||
|
- [ ] Install JDK 17, Android SDK, set `ANDROID_HOME` (`12-toolchain.md`).
|
||||||
|
- [ ] `cd hermes-agent && uv sync` (hermes venv works).
|
||||||
|
- [ ] Create monorepo scaffold (`02-monorepo.md`): `gateway-plugin/`, `app/`
|
||||||
|
(CMP: `shared`, `androidApp`, `desktopApp`), root `.gitignore`
|
||||||
|
(**excludes `hermes-agent/`**), root `README.md`.
|
||||||
|
- [ ] `git init` + a pre-commit/CI guard that fails if `hermes-agent/` is staged.
|
||||||
|
- [ ] CMP project builds empty: `./gradlew :androidApp:assembleDebug`,
|
||||||
|
`./gradlew :desktopApp:run` (blank window).
|
||||||
|
- [ ] Plugin skeleton: `plugin.yaml` + `adapter.py` with `register(ctx)` + a
|
||||||
|
no-op `AndroidAdapter` → `hermes gateway status` lists **android**.
|
||||||
|
- **Demo:** `hermes gateway status` shows `android`; `./gradlew
|
||||||
|
:androidApp:installDebug` installs a blank app on the MIX 2S.
|
||||||
|
- **Accept:** blank app installs + launches on-device; plugin visible in
|
||||||
|
`hermes gateway status`; `hermes-agent/` is git-ignored (verify with
|
||||||
|
`git status --ignored`).
|
||||||
|
|
||||||
|
## M1 — Gateway core loop (text round-trip)
|
||||||
|
**Goal:** pair + send a text message + get a (non-streaming) reply.
|
||||||
|
- [ ] WS server (`ws_server.py`): bind, `hello` auth (constant-time),
|
||||||
|
`hello.ack`, heartbeat, connection registry.
|
||||||
|
- [ ] `AndroidAdapter.send()` → `message` frame; inbound `message.send` →
|
||||||
|
`MessageEvent` → `handle_message`.
|
||||||
|
- [ ] Pairing store + `ANDROID_TOKEN`; QR payload in `interactive_setup`.
|
||||||
|
- [ ] App: Connect screen (URL+token, real `hello` test), `GatewayClient`
|
||||||
|
(connect + reconnect), ChatScreen sends + renders `message`.
|
||||||
|
- [ ] `ws_probe.py` harness drives a real turn.
|
||||||
|
- **Demo (on-device):** pair the phone, send "hello", see the agent's reply.
|
||||||
|
- **Accept:** text round-trip works on-device; wrong token is rejected;
|
||||||
|
reconnect after gateway restart re-pairs.
|
||||||
|
|
||||||
|
## M2 — Streaming + reasoning + tools + commentary
|
||||||
|
**Goal:** the "agent transparency" features.
|
||||||
|
- [ ] Map consumer `send`/`edit_message` → `message.start/update/stop`.
|
||||||
|
- [ ] Reasoning: set `show_reasoning` for android; adapter splits prefix →
|
||||||
|
`reasoning` field. **Verify format with `ws_probe.py`.**
|
||||||
|
- [ ] Tool events: classify tool-progress `send()`s → structured
|
||||||
|
`tool.start/progress/end` (turn-state machine). **Verify with probe.**
|
||||||
|
- [ ] Commentary → `commentary` frames. Typing → `typing`.
|
||||||
|
- [ ] App: live bubble (coalesced updates), `ReasoningBlock` (collapse + copy),
|
||||||
|
`ToolCard` with **Everything/Truncated/Nothing** setting, dimmed
|
||||||
|
`commentary` bubble.
|
||||||
|
- **Demo (on-device):** a multi-step prompt streams, shows reasoning above the
|
||||||
|
answer, tool cards (toggle verbosity), and an intermediate beat.
|
||||||
|
- **Accept:** all four render correctly; tool verbosity setting changes
|
||||||
|
rendering; reasoning copy button works; frame shapes match `04-wire-protocol`.
|
||||||
|
|
||||||
|
## M3 — Channels/threads + cron + search
|
||||||
|
**Goal:** organization + cron delegation + search.
|
||||||
|
- [ ] Channel directory (SQLite): default channel ensured; `channel.create/
|
||||||
|
rename/set_default/delete` + `channel.*` frames.
|
||||||
|
- [ ] Threads: toggle in default chat; `thread_id` lanes; `create_handoff_thread`.
|
||||||
|
- [ ] `parse_target_ref_fn` + `cron_deliver_env_var` → cron
|
||||||
|
`deliver=android:<chat>[:<thread>]` works.
|
||||||
|
- [ ] `search.py` FTS5 bridge; `search` frame (all / this-chat) → results.
|
||||||
|
- [ ] App: channel list (drawer/rail), thread toggle + topic switcher, "new
|
||||||
|
channel" + "set as cron target", SearchScreen with scope toggle + jump.
|
||||||
|
- **Demo (on-device):** create "Cron Reports"; create a cron job delivering to
|
||||||
|
it; it fires into that channel; search finds a message (both scopes).
|
||||||
|
- **Accept:** cron output lands in the chosen channel (not default); threads
|
||||||
|
isolate context; search scoping correct; channel list reconciles on events.
|
||||||
|
|
||||||
|
## M4 — Media
|
||||||
|
**Goal:** attach + receive + play media.
|
||||||
|
- [ ] Inbound: `media.upload` chunked → `cache_*_from_bytes` → `media_urls`;
|
||||||
|
size limit + sha256 + MIME re-sniff.
|
||||||
|
- [ ] Outbound: `send_*` → `media.offer`; `media.pull` chunked; delivery-path
|
||||||
|
security.
|
||||||
|
- [ ] App: SAF pickers + preview chips + upload; `media.pull` → cache;
|
||||||
|
**ExoPlayer** inline (audio mini-player, video fullscreen/PiP); image/doc
|
||||||
|
viewers.
|
||||||
|
- **Demo (on-device):** attach a photo + video (agent sees them); ask agent to
|
||||||
|
send an image/video → plays live in-app.
|
||||||
|
- **Accept:** both directions work; over-limit rejected; playback is live;
|
||||||
|
only allowed files are servable.
|
||||||
|
|
||||||
|
## M5 — Push + offline (FCM + ntfy)
|
||||||
|
**Goal:** reach the phone when backgrounded; catch up on reconnect.
|
||||||
|
- [ ] Outbox (SQLite) + sync cursor; `sync`/`sync.done`; retention prune.
|
||||||
|
- [ ] `push.py`: `FcmBackend` (HTTP v1 + service account, httpx) +
|
||||||
|
`NtfyBackend`; selected by `ANDROID_PUSH_BACKEND`.
|
||||||
|
- [ ] Fire push on no-live-subscriber; data payload for silent sync.
|
||||||
|
- [ ] App: FCM service (foreground banner + background foreground-service sync),
|
||||||
|
`onNewToken` → `fcm.register`; per-chat notification channels; deep-link.
|
||||||
|
ntfy listener fallback.
|
||||||
|
- [ ] In-app `notification` banners (channel_renamed, approval, cron, …).
|
||||||
|
- **Demo (on-device):** background the app → trigger a message → notification
|
||||||
|
appears → tap → syncs + opens the chat. Repeat with ntfy backend.
|
||||||
|
- **Accept:** push arrives when backgrounded (both backends); reconnect syncs
|
||||||
|
with no loss/dup; banners show for foreground events.
|
||||||
|
|
||||||
|
## M6 — Desktop app
|
||||||
|
**Goal:** the same app on a big screen.
|
||||||
|
- [ ] `desktopMain`: tray + OS notifications; `MediaPlayer` actual (mpv/WebView);
|
||||||
|
`MediaPicker` actual (file dialog); `SecureStore` actual; window mgmt.
|
||||||
|
- [ ] Two-pane default layout; keyboard shortcuts; optional inspector pane.
|
||||||
|
- [ ] Parity pass vs Android feature checklist (`11-desktop-app.md`).
|
||||||
|
- [ ] jpackage builds (Linux first; macOS/Windows as available).
|
||||||
|
- **Demo:** desktop app pairs to the same gateway; full feature parity; tray
|
||||||
|
notifications; media plays.
|
||||||
|
- **Accept:** all Android features work on desktop; tray + shortcuts work;
|
||||||
|
native binary launches.
|
||||||
|
|
||||||
|
## M7 — Polish + E2E + docs
|
||||||
|
**Goal:** ship-quality.
|
||||||
|
- [ ] Telegram-style layout pass (per reference image): header, bubbles, date
|
||||||
|
separators, ✓✓, model/token footer, banner, bottom bar.
|
||||||
|
- [ ] Theming (dark default, accent), onboarding/pairing UX, empty/loading/
|
||||||
|
reconnecting/degraded states with honest copy.
|
||||||
|
- [ ] Full E2E suite (`13-testing.md` scenarios 1–12) automated where possible.
|
||||||
|
- [ ] Docs: `docs/protocol/frames.schema.json` finalized; `docs/setup.md`
|
||||||
|
(user-facing pairing + FCM/ntfy + remote access); root README.
|
||||||
|
- [ ] Security hardening checklist (`09-pairing-security.md`) verified.
|
||||||
|
- **Demo:** end-to-end on phone + desktop simultaneously; cron into a channel;
|
||||||
|
push; media; search.
|
||||||
|
- **Accept:** all feature-checklist items pass on-device; E2E green; docs
|
||||||
|
complete; `hermes-agent/` still never committed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sequencing notes
|
||||||
|
- **M1/M2 depend on the `ws_probe.py` harness** to lock frame shapes early —
|
||||||
|
build it in M1.
|
||||||
|
- **M3 (cron) and M5 (push) both touch the outbox** — build the outbox in M3,
|
||||||
|
extend for push in M5.
|
||||||
|
- **M6 (desktop) reuses M1–M5 shared code** — do it after the Android features
|
||||||
|
are stable so the shared module is settled.
|
||||||
|
- Parallelizable: plugin (Python) and app (Kotlin) can be worked on
|
||||||
|
concurrently once the protocol (`04-wire-protocol.md`) is agreed; the probe
|
||||||
|
harness is the integration seam.
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
# 15 — hermes-agent Source Reference Map
|
||||||
|
|
||||||
|
A cheat-sheet of the **exact hermes-agent files/lines** to read for each
|
||||||
|
integration point. Paths are relative to `hermes-agent/` (the read-only
|
||||||
|
reference). This lets a coder jump straight to the right code instead of
|
||||||
|
re-deriving the architecture.
|
||||||
|
|
||||||
|
> ⚠️ Read-only. We **install** our plugin into `~/.hermes/plugins/android`; we
|
||||||
|
> never edit these files.
|
||||||
|
|
||||||
|
## Plugin / platform registration
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| How to add a platform (Plugin Path) | `gateway/platforms/ADDING_A_PLATFORM.md` |
|
||||||
|
| Canonical plugin-platform example | `plugins/platforms/irc/adapter.py` (esp. `register(ctx)` at :953, `_env_enablement` :677, `_standalone_send` :743, `_get_scoped_secret` :42) |
|
||||||
|
| ntfy plugin example (push-ish) | `plugins/platforms/ntfy/adapter.py`, `plugin.yaml` |
|
||||||
|
| `register_platform()` (PluginContext) | `hermes_cli/plugins.py:2774` |
|
||||||
|
| `PlatformEntry` dataclass (all fields) | `gateway/platform_registry.py:63` |
|
||||||
|
| `Platform` enum | `gateway/config.py` |
|
||||||
|
| Plugin discovery (`PluginManager`) | `hermes_cli/plugins.py` |
|
||||||
|
|
||||||
|
## Base adapter contract
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| `BasePlatformAdapter` (ABC) | `gateway/platforms/base.py:2890` |
|
||||||
|
| `MessageEvent` (inbound) | `gateway/platforms/base.py:2300` |
|
||||||
|
| `MessageType` | `gateway/platforms/base.py:2278` |
|
||||||
|
| `SendResult` | `gateway/platforms/base.py:2466` |
|
||||||
|
| `build_source(...)` (SessionSource) | `gateway/platforms/base.py:7047` |
|
||||||
|
| `handle_message(event)` | `gateway/platforms/base.py:5981` |
|
||||||
|
| `send()` / `edit_message()` / `delete_message()` | `base.py:3920 / 3976 / 4005` |
|
||||||
|
| `send_typing` / `stop_typing` | `base.py:4298 / 4307` |
|
||||||
|
| Media send: `send_image/video/document/voice/animation/image_file/multiple_images` | `base.py:4396 / 4632 / 4659 / 4486 / 4415 / 4339 / 7834(tg)` |
|
||||||
|
| `extract_images` / `extract_media` / `extract_local_files` | `base.py:4439 / 4884 / 5020` |
|
||||||
|
| Media cache helpers: `cache_image/audio/video/document_from_bytes` | `base.py:854 / 1005 / 1122 / 2126` |
|
||||||
|
| Inbound media size: `get_inbound_media_max_bytes` / `validate_inbound_media_size` | `base.py:758 / 779` |
|
||||||
|
| Media delivery security: `validate_media_delivery_path` + roots/recency/denied | `base.py:1684 / 1312-1480` |
|
||||||
|
| Interactive: `send_slash_confirm` / `send_clarify` / `send_private_notice` | `base.py:4169 / 4204 / 4278` |
|
||||||
|
| `create_handoff_thread` | `base.py:3949` |
|
||||||
|
| Streaming hooks: `supports_draft_streaming` / `send_draft` / `render_message_event` / `format_tool_event` | `base.py:3215 / 3274 / 3316 / 3337` |
|
||||||
|
| Scoped secret read pattern (profile-safe) | `plugins/platforms/irc/adapter.py:42` |
|
||||||
|
| Scoped lock (profile-safe bind) | `gateway/status.py` (`acquire_scoped_lock`) |
|
||||||
|
|
||||||
|
## Streaming (legacy callback path — what the main gateway uses)
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| `GatewayStreamConsumer` (the sink) | `gateway/stream_consumer.py:156` |
|
||||||
|
| `on_delta` / `on_commentary` / `on_segment_break` / `finish` | `stream_consumer.py:611 / 518 / 514 / 623` |
|
||||||
|
| Consumer `run()` loop (edit cadence) | `stream_consumer.py:781` |
|
||||||
|
| Structured events (ACP-only, NOT main gateway) | `gateway/stream_events.py`, `gateway/stream_dispatch.py:40` |
|
||||||
|
| Callback wiring (agent → consumer) | `gateway/run.py:5642-5704` (`tool_progress_callback`, `stream_delta_callback`, `interim_assistant_callback`, `status_callback`, `event_callback`) |
|
||||||
|
| `send_progress_messages` (tool progress → `send`) | `gateway/run.py:4603` |
|
||||||
|
| Progress metadata | `gateway/run.py:28089` |
|
||||||
|
|
||||||
|
## Reasoning display
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Reasoning prepended to final response | `gateway/run.py:20089-20127` |
|
||||||
|
| `show_reasoning` / `reasoning_style` defaults + per-platform | `gateway/display_config.py:33-181` |
|
||||||
|
| `resolve_display_setting()` | `gateway/display_config.py:187` |
|
||||||
|
| `last_reasoning` in agent result | `gateway/run.py:6471` |
|
||||||
|
| Think-tag filtering in consumer | `stream_consumer.py:175-185, 627+` |
|
||||||
|
|
||||||
|
## Display settings (tool progress, interim, streaming, etc.)
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| All overrideable display keys + defaults | `gateway/display_config.py:33` (`tool_progress`, `show_reasoning`, `reasoning_style`, `tool_preview_length`, `streaming`, `interim_assistant_messages`, `long_running_notifications`, `cleanup_progress`, `live_status`) |
|
||||||
|
| Per-platform tiers | `gateway/display_config.py:81-181` |
|
||||||
|
|
||||||
|
## Slash commands
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| `COMMAND_REGISTRY` / `CommandDef` (all commands) | `hermes_cli/commands.py:144+` |
|
||||||
|
| `GATEWAY_KNOWN_COMMANDS` / `is_gateway_known_command` / `resolve_command` | `hermes_cli/commands.py` |
|
||||||
|
| Gateway command dispatch (alias, access, hooks) | `gateway/run.py:16974-17099` |
|
||||||
|
| `send_model_picker` / `send_choice_picker` (Telegram ref) | `plugins/platforms/telegram/adapter.py:6350 / 6424` |
|
||||||
|
| Picker invocation from gateway | `gateway/slash_commands.py:1859, 2157, 3622-3657` |
|
||||||
|
| Button-callback id conventions (`cl:`, `appr:`, `sc:`) | `gateway/platforms/ADDING_A_PLATFORM.md` (Interactive UX) |
|
||||||
|
|
||||||
|
## Cron delivery
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Resolve a delivery target (`platform:chat_id[:thread_id]`) | `cron/scheduler.py:2148` (`_resolve_single_delivery_target`) |
|
||||||
|
| `_resolve_delivery_targets` / routing tokens (`all`) | `cron/scheduler.py:2296 / 2278` |
|
||||||
|
| `cron_deliver_env_var` handling (home channel) | `cron/scheduler.py:1903+` |
|
||||||
|
| `deliver` param normalization | `cron/scheduler.py:2251` |
|
||||||
|
| `send_message` tool target resolution (`resolve_send_target`, `prepare_send_message_platforms`) | `tools/send_message_tool.py` |
|
||||||
|
| `parse_target_ref_fn` usage | `tools/send_message_tool.py` (`_parse_target_ref`) |
|
||||||
|
| Cron mirror delivery (default off) | `cron/scheduler.py:1520` |
|
||||||
|
| `cronjob` tool schema (`deliver` description) | `tools/cronjob_tools.py` |
|
||||||
|
|
||||||
|
## Search (FTS5)
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Session store (SQLite + FTS5) | `hermes_state.py` |
|
||||||
|
| Search implementation | `hermes_state_search.py` |
|
||||||
|
| Schema | `hermes_state_schema.py` |
|
||||||
|
|
||||||
|
## Config / env / profiles
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| `get_hermes_home()` / `display_hermes_home()` (profile-safe paths) | `hermes_constants.py` |
|
||||||
|
| `DEFAULT_CONFIG` / `OPTIONAL_ENV_VARS` | `hermes_cli/config.py` |
|
||||||
|
| Gateway config load (`load_gateway_config`, `_apply_env_overrides`) | `gateway/config.py` |
|
||||||
|
| Profile override (`_apply_profile_override`) | `hermes_cli/main.py` |
|
||||||
|
| Secret scope (multiplex fail-closed) | `agent/secret_scope.py` |
|
||||||
|
| PII redaction | `agent/redact.py` |
|
||||||
|
|
||||||
|
## Dependencies (confirm zero new deps)
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| `websockets==15.0.1` (core) | `pyproject.toml:111` |
|
||||||
|
| `httpx[socks]==0.28.1` (core) | `pyproject.toml:44` |
|
||||||
|
| `aiohttp` (messaging extra, NOT core) | `pyproject.toml:185` |
|
||||||
|
| Dependency pinning policy | `pyproject.toml:19-39`, root `AGENTS.md` |
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Hermetic test runner (use this, not bare pytest) | `scripts/run_tests.sh` |
|
||||||
|
| `_isolate_hermes_home` fixture | `tests/conftest.py` |
|
||||||
|
| Example platform tests | `tests/gateway/test_*.py` (e.g. `test_google_chat.py`, `test_line_plugin.py`) |
|
||||||
|
| Stream-event tests (dispatcher) | `tests/gateway/test_stream_events.py` |
|
||||||
|
|
||||||
|
## Existing desktop app (for reference only — we do NOT reuse its backend)
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Electron desktop app | `apps/desktop/` (its own `AGENTS.md`, `DESIGN.md`) |
|
||||||
|
| Shared JSON-RPC WS client (tui_gateway protocol) | `apps/shared/src/json-rpc-gateway.ts` |
|
||||||
|
| tui_gateway WS transport (mobile-client-ready) | `tui_gateway/ws.py` |
|
||||||
|
| tui_gateway method catalog | `tui_gateway/server.py` |
|
||||||
|
|
||||||
|
> Note: the existing desktop app talks to the **`tui_gateway`** backend
|
||||||
|
> (`hermes serve`), a *different* process from the messaging gateway. Our apps
|
||||||
|
> talk to the **messaging gateway** via our own plugin + protocol. We borrow
|
||||||
|
> naming conventions only.
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
# 16 — Decisions & Open Questions
|
||||||
|
|
||||||
|
## Locked decisions (from planning, 2026-08-19)
|
||||||
|
|
||||||
|
| # | Decision | Choice | Rationale |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | Desktop app tech | **Compose Multiplatform** | Desktop = "the Android app, tweaked"; share protocol/state/UI. |
|
||||||
|
| 2 | Push backend | **Both — FCM primary, ntfy fallback** | FCM is standard/reliable; ntfy for self-hosters with no Firebase. `ANDROID_PUSH_BACKEND`. |
|
||||||
|
| 3 | Media transport | **Over the WebSocket** | One transport, zero new Python deps; chunked binary frames. |
|
||||||
|
| 4 | Phone default layout | **User-toggleable, single-pane default** | App-like on phones; auto two-pane on large screens; desktop defaults two-pane. |
|
||||||
|
|
||||||
|
## Additional decisions made during planning
|
||||||
|
|
||||||
|
| Decision | Choice | Note |
|
||||||
|
|---|---|---|
|
||||||
|
| Connection point | **Messaging gateway** (platform plugin `android`), not `tui_gateway` | Makes cron/`send_message`/slash/coexistence native. |
|
||||||
|
| Plugin style | **Community plugin** (`register(ctx)`) | Zero hermes-core changes. |
|
||||||
|
| Python deps | **None new** (`websockets` + `httpx` are core) | Respects hermes pinning policy. |
|
||||||
|
| Tool events | **Structured frames; app controls verbosity** | Gateway sends full data; app = everything/truncated/nothing. |
|
||||||
|
| Reasoning | **Adapter splits the `show_reasoning` prefix** | Clean `reasoning` field → collapsible block above message. |
|
||||||
|
| Channels/threads | **Map onto `chat_id`/`thread_id`** | Existing gateway primitives; cron targets them. |
|
||||||
|
| Offline | **SQLite outbox + sync cursor** | Catch-up on reconnect; push on disconnect. |
|
||||||
|
| Default chat id | `android:default` | Home channel + cron default. |
|
||||||
|
| WS port | `8790` (default) | Configurable. |
|
||||||
|
| minSdk | 26 (test device API 29) | Broad coverage. |
|
||||||
|
| Frame routing | **Broadcast to all connected devices** (no per-chat subscribe) | Single-user model; simpler. |
|
||||||
|
| Initial channel load | **`history` frame** (paginated) | `sync` only replays outbox; `history` loads full messages. |
|
||||||
|
| Slash menu | **`commands.catalog` + `commands.complete`** frames | Gateway serves catalog; app renders bottom sheet + autocomplete. |
|
||||||
|
| Agent lifecycle | **`agent.busy`/`agent.idle`** events + **`agent.stop`/`agent.steer`** requests | App shows thinking indicator; user can abort or steer mid-turn. |
|
||||||
|
| Local DB (KMP) | **SQLDelight** (not Room) | Room is Android-only; SQLDelight works in commonMain for both platforms. |
|
||||||
|
| Voice input | **Record → upload as audio media** (no client-side STT) | Agent's STT (if configured) handles transcription. |
|
||||||
|
|
||||||
|
## Open questions (resolve during implementation)
|
||||||
|
|
||||||
|
These are **implementation-time** details, not blockers. Each has a default we
|
||||||
|
will proceed with unless you say otherwise.
|
||||||
|
|
||||||
|
1. **Tool-progress vs commentary classification (M2).** The exact signal that
|
||||||
|
distinguishes a tool-progress `send()` from a regular `send()`/commentary in
|
||||||
|
the legacy path. *Default:* per-chat turn-state machine + line-format
|
||||||
|
heuristic, **verified empirically** with the `ws_probe.py` harness against the
|
||||||
|
live gateway. If a clean metadata marker exists, prefer it.
|
||||||
|
2. **Reasoning prefix format stability (M2).** We split on the `code`-style
|
||||||
|
`💭 **Reasoning:**\n```\n…\n```\n\n` prefix. *Default:* set
|
||||||
|
`reasoning_style: code` for android and split on that; fallback = no
|
||||||
|
reasoning field (full text) if the prefix isn't found. Verify in M2.
|
||||||
|
3. **Per-device tokens vs shared token (M1/M5).** *Default (v1):* shared
|
||||||
|
`ANDROID_TOKEN` + optional `ANDROID_ALLOWED_USERS` device allowlist.
|
||||||
|
Per-device revocable tokens are a stretch.
|
||||||
|
4. **Desktop video backend (M6).** *Default:* `libmpv`/`mpv`-backed Compose
|
||||||
|
surface, WebView fallback. Confirm `mpv` availability on target OSes during
|
||||||
|
M6.
|
||||||
|
5. **Remote access default (M1).** *Default:* document Tailscale as the
|
||||||
|
recommended remote path; WSS + reverse proxy as alternatives. No public bind
|
||||||
|
by default.
|
||||||
|
6. **Streaming cadence (M2).** If live updates look chunky, tune
|
||||||
|
`display.platforms.android.streaming` / consumer edit interval. *Default:*
|
||||||
|
follow global streaming config.
|
||||||
|
7. **App package name / branding.** *Default:* applicationId `dev.iris.app`,
|
||||||
|
app name "Iris". Confirm final product name + package + icon.
|
||||||
|
8. **ntfy in-app listener (M5).** *Default:* a foreground service maintaining the
|
||||||
|
ntfy subscription (no extra native lib) — or a lightweight ntfy client lib if
|
||||||
|
one is acceptable. Decide in M5.
|
||||||
|
9. **Auto-update for desktop (M7).** *Default:* out of scope (manual download).
|
||||||
|
Revisit post-v1.
|
||||||
|
10. **iOS port.** Out of scope for v1 (protocol is transport-agnostic, so it's a
|
||||||
|
future port). No action now.
|
||||||
|
|
||||||
|
## Explicit non-goals (v1)
|
||||||
|
|
||||||
|
- Multi-user / group chat (personal 1-user agent).
|
||||||
|
- End-to-end encryption (transport WSS only).
|
||||||
|
- Standalone-cron delivery while the gateway process is fully down (best-effort
|
||||||
|
push only).
|
||||||
|
- iOS.
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# Iris × Hermes — Implementation Reference Library
|
||||||
|
|
||||||
|
A coder-facing reference library for building a **native Android + Desktop
|
||||||
|
experience** for [hermes-agent](https://github.com/NousResearch/hermes-agent),
|
||||||
|
connected through a **gateway platform plugin**.
|
||||||
|
|
||||||
|
This folder is the single source of truth for *what to build and why*. Read it
|
||||||
|
top-to-bottom once, then use the numbered docs as a lookup while implementing.
|
||||||
|
|
||||||
|
> ⚠️ **READ FIRST — two hard rules**
|
||||||
|
> 1. **`hermes-agent/` (sibling of this folder) is a read-only research
|
||||||
|
> reference. It must NEVER be committed, pushed, or shipped.** It is
|
||||||
|
> git-ignored at the repo root. We only *install* our plugin into a live
|
||||||
|
> hermes install (`~/.hermes/plugins/`); we never modify hermes core.
|
||||||
|
> 2. **ADB is installed and a device is connected** (`a5ca2a4b`, Xiaomi MIX 2S,
|
||||||
|
> Android 10 / API 29). Use it to install/launch/debug the app on-device.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Reading order
|
||||||
|
|
||||||
|
| # | File | When to read |
|
||||||
|
|---|------|--------------|
|
||||||
|
| 0 | [`00-overview.md`](00-overview.md) | Always first. Vision, scope, disclaimers, locked decisions. |
|
||||||
|
| 1 | [`01-architecture.md`](01-architecture.md) | Before touching code. System shape + rationale. |
|
||||||
|
| 2 | [`02-monorepo.md`](02-monorepo.md) | When scaffolding the repo. |
|
||||||
|
| 3 | [`03-gateway-plugin.md`](03-gateway-plugin.md) | When building the Python plugin. |
|
||||||
|
| 4 | [`04-wire-protocol.md`](04-wire-protocol.md) | When implementing either side of the WS. |
|
||||||
|
| 5 | [`05-streaming.md`](05-streaming.md) | Streaming / reasoning / tools / intermediate. |
|
||||||
|
| 6 | [`06-channels-cron-search.md`](06-channels-cron-search.md) | Channels, threads, cron delivery, search. |
|
||||||
|
| 7 | [`07-media.md`](07-media.md) | Media upload/download + playback. |
|
||||||
|
| 8 | [`08-push.md`](08-push.md) | Push (FCM + ntfy), outbox, sync. |
|
||||||
|
| 9 | [`09-pairing-security.md`](09-pairing-security.md) | Pairing, auth, security model. |
|
||||||
|
| 10 | [`10-android-app.md`](10-android-app.md) | When building the Android app. |
|
||||||
|
| 11 | [`11-desktop-app.md`](11-desktop-app.md) | When building the Desktop app. |
|
||||||
|
| 12 | [`12-toolchain.md`](12-toolchain.md) | First time on a machine (JDK/SDK/uv/Firebase). |
|
||||||
|
| 13 | [`13-testing.md`](13-testing.md) | Writing tests + on-device ADB workflow. |
|
||||||
|
| 14 | [`14-milestones.md`](14-milestones.md) | Planning work / tracking progress. |
|
||||||
|
| 15 | [`15-hermes-reference.md`](15-hermes-reference.md) | **Cheat-sheet** of hermes-agent source to read. |
|
||||||
|
| 16 | [`16-open-questions.md`](16-open-questions.md) | Decisions made + open items. |
|
||||||
|
|
||||||
|
Machine-readable / diagrams:
|
||||||
|
- [`protocol/frames.schema.json`](protocol/frames.schema.json) — wire-frame schema.
|
||||||
|
- [`diagrams/architecture.mmd`](diagrams/architecture.mmd) — mermaid architecture.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The three deliverables (one monorepo)
|
||||||
|
|
||||||
|
1. **`gateway-plugin/`** — a Python hermes **platform plugin** named `android`.
|
||||||
|
Runs inside the `hermes gateway` process. Opens a WebSocket server the apps
|
||||||
|
connect to. Implements the full `BasePlatformAdapter` contract. **Zero new
|
||||||
|
Python dependencies, zero hermes-core changes.**
|
||||||
|
2. **`app/androidApp`** — native Kotlin + Jetpack Compose client.
|
||||||
|
3. **`app/desktopApp`** — Kotlin + Compose Multiplatform client that *shares*
|
||||||
|
the Android app's code and is "tweaked" for a big screen.
|
||||||
|
|
||||||
|
The Android and Desktop clients live in **one Compose Multiplatform Gradle
|
||||||
|
project** (`app/`) with a shared KMP module (`app/shared`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- **Phase:** Planning complete → ready to implement (Milestone M0).
|
||||||
|
- **Owner decisions locked:** see [`16-open-questions.md`](16-open-questions.md).
|
||||||
|
- **Last updated:** 2026-08-19.
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
%% Iris x Hermes — architecture (mermaid)
|
||||||
|
%% Render with any mermaid viewer (e.g. https://mermaid.live or `mmdc`).
|
||||||
|
flowchart TB
|
||||||
|
subgraph HOST["User's machine (home server / PC)"]
|
||||||
|
subgraph GW["hermes gateway (ONE process)"]
|
||||||
|
AGENT["Agent core<br/>(run_agent.py)"]
|
||||||
|
SESS["Sessions<br/>(SQLite + FTS5)"]
|
||||||
|
CRON["Cron scheduler"]
|
||||||
|
subgraph PLUGIN["android PLATFORM PLUGIN"]
|
||||||
|
ADAPTER["AndroidAdapter<br/>(BasePlatformAdapter)"]
|
||||||
|
OUTBOX["Outbox (SQLite)<br/>+ sync cursor"]
|
||||||
|
PUSH["push.py<br/>FcmBackend / NtfyBackend"]
|
||||||
|
MEDIA["media.py<br/>cache + chunk stream"]
|
||||||
|
PAIR["pairing.py<br/>token + device registry"]
|
||||||
|
SEARCH["search.py<br/>FTS5 bridge"]
|
||||||
|
end
|
||||||
|
WSS["WebSocket SERVER<br/>(websockets) ws://host:8790/ws"]
|
||||||
|
end
|
||||||
|
AGENT -->|legacy stream callbacks| ADAPTER
|
||||||
|
CRON -->|deliver=android:chat:thread| ADAPTER
|
||||||
|
ADAPTER <--> WSS
|
||||||
|
ADAPTER <--> OUTBOX
|
||||||
|
ADAPTER <--> PUSH
|
||||||
|
ADAPTER <--> MEDIA
|
||||||
|
ADAPTER <--> PAIR
|
||||||
|
ADAPTER <--> SEARCH
|
||||||
|
ADAPTER <--> SESS
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph CLOUD["Cloud relays"]
|
||||||
|
FCM["Google FCM"]
|
||||||
|
NTFY["ntfy (self-host / ntfy.sh)"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph DEVICES["Clients"]
|
||||||
|
PHONE["ANDROID APP<br/>(Kotlin / Compose)<br/>WS client + ExoPlayer + FCM"]
|
||||||
|
DESKTOP["DESKTOP APP<br/>(Compose Multiplatform)<br/>WS client + tray + desktop player"]
|
||||||
|
end
|
||||||
|
|
||||||
|
WSS <-->|WSS JSON + binary media| PHONE
|
||||||
|
WSS <-->|WSS JSON + binary media| DESKTOP
|
||||||
|
PUSH -->|FCM HTTP v1| FCM
|
||||||
|
PUSH -->|ntfy publish| NTFY
|
||||||
|
FCM -->|wake (data msg)| PHONE
|
||||||
|
NTFY -->|subscribe| PHONE
|
||||||
|
|
||||||
|
classDef host fill:#eef,stroke:#333;
|
||||||
|
classDef plugin fill:#efe,stroke:#333;
|
||||||
|
classDef device fill:#fee,stroke:#333;
|
||||||
|
classDef cloud fill:#ffe,stroke:#333;
|
||||||
|
class GW,AGENT,SESS,CRON,OUTBOX,PUSH,MEDIA,PAIR,SEARCH,WSS host;
|
||||||
|
class ADAPTER plugin;
|
||||||
|
class PHONE,DESKTOP device;
|
||||||
|
class FCM,NTFY cloud;
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
{
|
||||||
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||||
|
"title": "Iris x Hermes wire protocol",
|
||||||
|
"description": "Machine-readable description of the JSON frames exchanged over the gateway WebSocket. Mirrors gateway-plugin/protocol.py and docs/04-wire-protocol.md. Media travels as binary WS frames referenced by header/end frames.",
|
||||||
|
"protocol_version": 1,
|
||||||
|
"envelope": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["v", "type"],
|
||||||
|
"properties": {
|
||||||
|
"v": { "type": "integer", "const": 1, "description": "Protocol version." },
|
||||||
|
"id": { "type": "integer", "description": "Request id; present on requests and their responses/acks. Absent on pure events." },
|
||||||
|
"type": { "type": "string", "description": "Frame type (see frame_types)." },
|
||||||
|
"chat_id": { "type": "string", "description": "Optional chat scope (e.g. android:default, android:chan_7)." },
|
||||||
|
"thread_id": { "type": "string", "description": "Optional thread scope within a chat_id." },
|
||||||
|
"payload": { "type": "object", "description": "Type-specific payload." }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"frame_types": {
|
||||||
|
"server_to_app": {
|
||||||
|
"hello.ack": {
|
||||||
|
"description": "Pairing succeeded.",
|
||||||
|
"payload": {
|
||||||
|
"server_caps": { "type": "object", "properties": { "streaming": {"type":"boolean"}, "reasoning": {"type":"boolean"}, "tools": {"type":"boolean"}, "media": {"type":"boolean"}, "search": {"type":"boolean"}, "push": {"type":"string","enum":["fcm","ntfy","none"]}, "pickers": {"type":"boolean"} } },
|
||||||
|
"sync_cursor": { "type": "integer" },
|
||||||
|
"channels": { "type": "array", "items": { "$ref": "#/definitions/channel" } }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"message": {
|
||||||
|
"description": "A final / standalone message.",
|
||||||
|
"payload": {
|
||||||
|
"message_id": { "type": "string" },
|
||||||
|
"role": { "type": "string", "enum": ["user", "assistant", "system", "cron"] },
|
||||||
|
"text": { "type": "string" },
|
||||||
|
"reasoning": { "type": "string", "description": "Optional; render ABOVE text." },
|
||||||
|
"media": { "type": "array", "items": { "$ref": "#/definitions/media_ref" } },
|
||||||
|
"reply_to": { "type": "string" },
|
||||||
|
"model": { "type": "string" },
|
||||||
|
"tokens": { "type": "integer" },
|
||||||
|
"ts": { "type": "integer", "description": "epoch millis" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"message.start": { "payload": { "message_id": { "type": "string" }, "role": { "type": "string" } } },
|
||||||
|
"message.update": { "payload": { "message_id": { "type": "string" }, "text": { "type": "string", "description": "Full current text (app replaces)." } } },
|
||||||
|
"message.stop": { "payload": { "message_id": { "type": "string" }, "final_text": { "type": "string" }, "reasoning": { "type": "string" }, "model": { "type": "string" }, "tokens": { "type": "integer" } } },
|
||||||
|
"commentary": { "description": "Intermediate assistant beat.", "payload": { "message_id": { "type": "string" }, "text": { "type": "string" } } },
|
||||||
|
"tool.start": { "payload": { "index": { "type": "integer" }, "name": { "type": "string" }, "preview": { "type": "string" }, "args": { "type": "object" } } },
|
||||||
|
"tool.progress": { "payload": { "index": { "type": "integer" }, "name": { "type": "string" }, "note": { "type": "string" } } },
|
||||||
|
"tool.end": { "payload": { "index": { "type": "integer" }, "name": { "type": "string" }, "ok": { "type": "boolean" }, "duration": { "type": "number" }, "output_preview": { "type": "string" } } },
|
||||||
|
"typing": { "payload": { "on": { "type": "boolean" } } },
|
||||||
|
"notification": { "payload": { "kind": { "type": "string", "enum": ["channel_renamed", "channel_created", "cron", "approval", "clarify", "generic"] }, "title": { "type": "string" }, "body": { "type": "string" }, "ts": { "type": "integer" } } },
|
||||||
|
"picker.model": { "payload": { "picker_id": { "type": "string" }, "current_model": { "type": "string" }, "current_provider": { "type": "string" }, "providers": { "type": "array", "items": { "type": "object", "properties": { "id": {"type":"string"}, "label": {"type":"string"}, "models": { "type": "array", "items": { "type": "object", "properties": { "id": {"type":"string"}, "label": {"type":"string"} } } } } } } } },
|
||||||
|
"picker.choice": { "payload": { "picker_id": { "type": "string" }, "title": { "type": "string" }, "choices": { "type": "array", "items": { "type": "object", "properties": { "value": {"type":"string"}, "label": {"type":"string"}, "is_current": {"type":"boolean"} } } } } },
|
||||||
|
"picker.clarify": { "payload": { "picker_id": { "type": "string" }, "question": { "type": "string" }, "choices": { "type": "array", "items": { "type": "object" } } } },
|
||||||
|
"picker.approval": { "payload": { "picker_id": { "type": "string" }, "command": { "type": "string" }, "description": { "type": "string" } } },
|
||||||
|
"picker.confirm": { "payload": { "picker_id": { "type": "string" }, "title": { "type": "string" }, "message": { "type": "string" } } },
|
||||||
|
"channel.list": { "payload": { "channels": { "type": "array", "items": { "$ref": "#/definitions/channel" } } } },
|
||||||
|
"channel.created": { "payload": { "$ref": "#/definitions/channel" } },
|
||||||
|
"channel.renamed": { "payload": { "chat_id": { "type": "string" }, "name": { "type": "string" } } },
|
||||||
|
"channel.deleted": { "payload": { "chat_id": { "type": "string" } } },
|
||||||
|
"history": { "description": "Response to history request; page of messages oldest→newest.", "payload": { "messages": { "type": "array", "items": { "type": "object", "properties": { "message_id": {"type":"string"}, "role": {"type":"string"}, "text": {"type":"string"}, "reasoning": {"type":"string"}, "model": {"type":"string"}, "tokens": {"type":"integer"}, "ts": {"type":"integer"} } } }, "has_more": { "type": "boolean" }, "oldest_message_id": { "type": "string" } } },
|
||||||
|
"commands.catalog": { "description": "Full slash-command catalog.", "payload": { "commands": { "type": "array", "items": { "type": "object", "properties": { "name": {"type":"string"}, "description": {"type":"string"}, "args_hint": {"type":"string"}, "category": {"type":"string"} } } } } },
|
||||||
|
"commands.complete": { "description": "Autocomplete matches for a typed prefix.", "payload": { "prefix": { "type": "string" }, "matches": { "type": "array", "items": { "type": "object", "properties": { "name": {"type":"string"}, "description": {"type":"string"}, "args_hint": {"type":"string"} } } } } },
|
||||||
|
"agent.busy": { "description": "Agent is processing; app shows thinking indicator.", "payload": { "reason": { "type": "string", "enum": ["processing", "tool", "waiting_input", "cron"] } } },
|
||||||
|
"agent.idle": { "description": "Agent turn complete; clear thinking indicator.", "payload": {} },
|
||||||
|
"search.results": { "payload": { "query": { "type": "string" }, "scope": { "type": "string", "enum": ["all", "chat"] }, "hits": { "type": "array", "items": { "type": "object", "properties": { "message_id": {"type":"string"}, "chat_id": {"type":"string"}, "thread_id": {"type":["string","null"]}, "role": {"type":"string"}, "snippet": {"type":"string"}, "ts": {"type":"integer"} } } } } },
|
||||||
|
"media.offer": { "description": "Agent-sent media available; app pulls bytes.", "payload": { "$ref": "#/definitions/media_ref" } },
|
||||||
|
"status": { "payload": { "state": { "type": "string", "enum": ["online", "restarting", "degraded"] }, "session": { "type": "object" } } },
|
||||||
|
"error": { "payload": { "code": { "type": "string", "enum": ["auth", "not_found", "rate_limited", "media_too_large", "unsupported", "internal"] }, "message": { "type": "string" } } },
|
||||||
|
"pong": { "payload": { "ts": { "type": "integer" } } },
|
||||||
|
"sync.done": { "payload": { "cursor": { "type": "integer" } } },
|
||||||
|
"media.pull.end": { "payload": { "ok": { "type": "boolean" } } }
|
||||||
|
},
|
||||||
|
"app_to_server": {
|
||||||
|
"hello": { "description": "First frame; auth + caps.", "payload": { "token": { "type": "string" }, "device_id": { "type": "string" }, "device_name": { "type": "string" }, "caps": { "type": "object", "properties": { "min_protocol": {"type":"integer"}, "media": {"type":"boolean"}, "push": {"type":"string"} } }, "fcm_token": { "type": "string" }, "ntfy_topic": { "type": "string" } } },
|
||||||
|
"message.send": { "payload": { "text": { "type": "string" }, "reply_to": { "type": "string" }, "media_refs": { "type": "array", "items": { "type": "string" } } } },
|
||||||
|
"media.upload.start": { "payload": { "media_ref": { "type": "string" }, "kind": { "$ref": "#/definitions/kind" }, "mime": { "type": "string" }, "size": { "type": "integer" }, "filename": { "type": "string" } } },
|
||||||
|
"media.upload.end": { "payload": { "media_ref": { "type": "string" }, "sha256": { "type": "string" } } },
|
||||||
|
"media.pull": { "payload": { "media_id": { "type": "string" } } },
|
||||||
|
"picker.select": { "payload": { "picker_id": { "type": "string" }, "value": { "type": "string" } } },
|
||||||
|
"channel.create": { "payload": { "name": { "type": "string" }, "kind": { "type": "string", "enum": ["channel", "thread"] }, "parent_chat_id": { "type": ["string", "null"] } } },
|
||||||
|
"channel.rename": { "payload": { "name": { "type": "string" } } },
|
||||||
|
"channel.set_default": { "payload": {} },
|
||||||
|
"channel.delete": { "payload": {} },
|
||||||
|
"search": { "payload": { "query": { "type": "string" }, "scope": { "type": "string", "enum": ["all", "chat"] }, "chat_id": { "type": "string" }, "thread_id": { "type": "string" } } },
|
||||||
|
"history": { "description": "Load a page of messages (initial open / scroll-up).", "payload": { "before_message_id": { "type": "string" }, "limit": { "type": "integer", "minimum": 1, "maximum": 200 } } },
|
||||||
|
"commands.catalog": { "description": "Fetch full slash-command catalog.", "payload": {} },
|
||||||
|
"commands.complete": { "description": "Autocomplete for typed /prefix.", "payload": { "prefix": { "type": "string" } } },
|
||||||
|
"agent.stop": { "description": "Abort current agent turn.", "payload": {} },
|
||||||
|
"agent.steer": { "description": "Inject steering message mid-turn.", "payload": { "text": { "type": "string" } } },
|
||||||
|
"read.receipt": { "description": "User viewed message; server stores + broadcasts to other devices.", "payload": { "chat_id": { "type": "string" }, "message_id": { "type": "string" } } },
|
||||||
|
"sync": { "description": "Reconnect catch-up; replays undelivered outbox frames only (not full history).", "payload": { "cursor": { "type": "integer" } } },
|
||||||
|
"fcm.register": { "payload": { "fcm_token": { "type": "string" }, "ntfy_topic": { "type": "string" } } },
|
||||||
|
"ping": { "payload": { "ts": { "type": "integer" } } }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"definitions": {
|
||||||
|
"kind": { "type": "string", "enum": ["image", "audio", "video", "document", "voice"] },
|
||||||
|
"channel": { "type": "object", "properties": { "chat_id": {"type":"string"}, "name": {"type":"string"}, "kind": {"type":"string","enum":["default","channel","thread"]}, "parent_chat_id": {"type":["string","null"]}, "is_default": {"type":"boolean"} } },
|
||||||
|
"media_ref": { "type": "object", "properties": { "media_id": {"type":"string"}, "kind": { "$ref": "#/definitions/kind" }, "mime": {"type":"string"}, "size": {"type":"integer"}, "filename": {"type":"string"} } }
|
||||||
|
},
|
||||||
|
"reliability": {
|
||||||
|
"ordering": "Per-connection (TCP/WS). message.update for a message_id is monotonic; app may coalesce to latest.",
|
||||||
|
"never_dropped": ["message", "message.stop", "tool.end", "notification", "picker.*", "channel.*", "agent.busy", "agent.idle", "history", "commands.catalog", "commands.complete", "search.results", "error"],
|
||||||
|
"coalescable_under_backpressure": ["message.update", "tool.progress"],
|
||||||
|
"offline": "Undelivered frames go to the outbox; replayed by sync. Terminal frames always outboxed."
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .adapter import register
|
||||||
|
|
||||||
|
__all__ = ["register"]
|
||||||
@@ -0,0 +1,471 @@
|
|||||||
|
"""
|
||||||
|
Android Platform Adapter for Hermes Agent (Iris x Hermes).
|
||||||
|
|
||||||
|
A plugin-based gateway adapter that runs a WebSocket server *inside* the
|
||||||
|
``hermes gateway`` process. The native Android / Desktop app connects to it
|
||||||
|
with a pairing token and talks to the agent over a single WS transport
|
||||||
|
(chat, streaming, tools, media, pairing, push-token).
|
||||||
|
|
||||||
|
Zero new Python dependencies: ``websockets`` and ``httpx`` are hermes core
|
||||||
|
deps. Zero hermes-core changes.
|
||||||
|
|
||||||
|
Milestone M0: this is a *skeleton* adapter. It registers the ``android``
|
||||||
|
platform, resolves its configuration, and implements the abstract adapter
|
||||||
|
contract as no-ops so that ``hermes gateway status`` lists ``android``. The
|
||||||
|
WebSocket server, pairing, streaming, media, outbox, push, and search are
|
||||||
|
wired in later milestones (see ``docs/14-milestones.md``).
|
||||||
|
|
||||||
|
Configuration in config.yaml::
|
||||||
|
|
||||||
|
gateway:
|
||||||
|
platforms:
|
||||||
|
android:
|
||||||
|
enabled: true
|
||||||
|
extra:
|
||||||
|
host: 127.0.0.1
|
||||||
|
port: 8790
|
||||||
|
home_channel: android:default
|
||||||
|
push_backend: fcm
|
||||||
|
outbox_retention_hours: 72
|
||||||
|
max_upload_bytes: 104857600
|
||||||
|
|
||||||
|
Or via environment variables (overrides config.yaml; secrets live in .env):
|
||||||
|
ANDROID_TOKEN, ANDROID_WS_HOST, ANDROID_WS_PORT, ANDROID_HOME_CHANNEL,
|
||||||
|
ANDROID_PUSH_BACKEND, ANDROID_FCM_SERVICE_ACCOUNT, NTFY_TOPIC, ...
|
||||||
|
"""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.secret_scope import UnscopedSecretError as _UnscopedSecretError
|
||||||
|
from agent.secret_scope import get_secret as _scoped_get_secret
|
||||||
|
|
||||||
|
|
||||||
|
def _get_scoped_secret(name, default=None):
|
||||||
|
"""Scope-aware credential read with the default-profile startup fallback.
|
||||||
|
|
||||||
|
Secondary profiles construct their adapters under a profile secret scope
|
||||||
|
-- the scope is authoritative and a scoped miss returns ``default`` (no
|
||||||
|
cross-profile borrow from ``os.environ``, which may hold another
|
||||||
|
profile's value). The DEFAULT profile's adapter constructs and sends
|
||||||
|
*unscoped* under multiplexing, where a bare ``get_secret`` would raise
|
||||||
|
``UnscopedSecretError`` and crash this path; there ``os.environ`` is that
|
||||||
|
profile's own value, so fall back to it. Same pattern as the IRC
|
||||||
|
``IRC_SERVER_PASSWORD`` read (``plugins/platforms/irc/adapter.py``).
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
val = _scoped_get_secret(name, default)
|
||||||
|
except _UnscopedSecretError:
|
||||||
|
val = os.getenv(name)
|
||||||
|
return val if val is not None else default
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Lazy import: BasePlatformAdapter and friends live in the main repo.
|
||||||
|
# We import at module level (as the bundled plugins do) but guard the heavy
|
||||||
|
# gateway imports so the plugin can be discovered before the gateway is fully
|
||||||
|
# initialised.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
from gateway.platforms.base import ( # noqa: E402
|
||||||
|
BasePlatformAdapter,
|
||||||
|
SendResult,
|
||||||
|
MessageEvent,
|
||||||
|
MessageType,
|
||||||
|
)
|
||||||
|
from gateway.config import Platform # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Defaults
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
DEFAULT_HOST = "127.0.0.1"
|
||||||
|
DEFAULT_PORT = 8790
|
||||||
|
DEFAULT_HOME_CHANNEL = "android:default"
|
||||||
|
DEFAULT_PUSH_BACKEND = "fcm"
|
||||||
|
DEFAULT_OUTBOX_RETENTION_HOURS = 72
|
||||||
|
DEFAULT_MAX_UPLOAD_BYTES = 100 * 1024 * 1024 # 100 MB
|
||||||
|
|
||||||
|
|
||||||
|
def _truthy(value: Optional[str]) -> bool:
|
||||||
|
return (value or "").strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Passive / config probes (called from status displays -- no side effects)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def check_requirements() -> bool:
|
||||||
|
"""PASSIVE dependency probe: ``websockets`` importable + token set.
|
||||||
|
|
||||||
|
Must be side-effect free (called from ``hermes setup`` / ``status`` /
|
||||||
|
dashboard readiness). Never installs.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import websockets # noqa: F401 (core dep)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
return bool(_get_scoped_secret("ANDROID_TOKEN"))
|
||||||
|
|
||||||
|
|
||||||
|
def validate_config(config) -> bool:
|
||||||
|
"""Given a PlatformConfig, is the platform properly configured?"""
|
||||||
|
extra = getattr(config, "extra", {}) or {}
|
||||||
|
token = _get_scoped_secret("ANDROID_TOKEN") or extra.get("token", "")
|
||||||
|
return bool(token)
|
||||||
|
|
||||||
|
|
||||||
|
def is_connected(config) -> bool:
|
||||||
|
"""Is the platform configured (env or config.yaml)?"""
|
||||||
|
return validate_config(config)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Env-driven auto-configuration (seeds PlatformConfig.extra pre-adapter)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _env_enablement() -> Optional[dict]:
|
||||||
|
"""Seed ``PlatformConfig.extra`` from env vars during gateway config load.
|
||||||
|
|
||||||
|
Called by the platform registry's env-enablement hook BEFORE adapter
|
||||||
|
construction, so ``gateway status`` and ``get_connected_platforms()``
|
||||||
|
reflect env-only configuration without instantiating the adapter.
|
||||||
|
Returns ``None`` when the platform isn't minimally configured (no token);
|
||||||
|
the caller then skips auto-enabling.
|
||||||
|
|
||||||
|
The special ``home_channel`` key in the returned dict is handled by the
|
||||||
|
core hook -- it becomes a proper ``HomeChannel`` dataclass on the
|
||||||
|
``PlatformConfig`` rather than being merged into ``extra``.
|
||||||
|
"""
|
||||||
|
token = _get_scoped_secret("ANDROID_TOKEN", "")
|
||||||
|
if not token:
|
||||||
|
return None
|
||||||
|
|
||||||
|
seed: Dict[str, Any] = {
|
||||||
|
"host": os.getenv("ANDROID_WS_HOST", "").strip() or DEFAULT_HOST,
|
||||||
|
"port": _parse_port(os.getenv("ANDROID_WS_PORT", "")),
|
||||||
|
"push_backend": (
|
||||||
|
os.getenv("ANDROID_PUSH_BACKEND", "").strip().lower()
|
||||||
|
or DEFAULT_PUSH_BACKEND
|
||||||
|
),
|
||||||
|
}
|
||||||
|
home = os.getenv("ANDROID_HOME_CHANNEL", "").strip() or DEFAULT_HOME_CHANNEL
|
||||||
|
seed["home_channel"] = {
|
||||||
|
"chat_id": home,
|
||||||
|
"name": os.getenv("ANDROID_HOME_CHANNEL_NAME", "").strip() or "Default",
|
||||||
|
}
|
||||||
|
return seed
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_port(raw: str) -> int:
|
||||||
|
try:
|
||||||
|
return int((raw or "").strip())
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return DEFAULT_PORT
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Target parsing: "android:<chat>[:<thread>]"
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _parse_target_ref(target_ref: str) -> Optional[tuple]:
|
||||||
|
"""Parse a raw target string into ``(chat_id, thread_id)`` or ``None``.
|
||||||
|
|
||||||
|
Recognises the native syntax ``android:<chat>[:<thread>]``. Returns
|
||||||
|
``None`` for anything else so the target proceeds to channel-directory
|
||||||
|
resolution.
|
||||||
|
"""
|
||||||
|
if not target_ref or not target_ref.startswith("android:"):
|
||||||
|
return None
|
||||||
|
body = target_ref[len("android:"):]
|
||||||
|
if not body:
|
||||||
|
return None
|
||||||
|
if ":" in body:
|
||||||
|
chat_id, thread_id = body.split(":", 1)
|
||||||
|
thread_id = thread_id or None
|
||||||
|
else:
|
||||||
|
chat_id, thread_id = body, None
|
||||||
|
chat_id = chat_id.strip()
|
||||||
|
if not chat_id:
|
||||||
|
return None
|
||||||
|
return (chat_id, thread_id)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Standalone (out-of-process) send -- best-effort, stretch for v1
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async def _standalone_send(
|
||||||
|
pconfig,
|
||||||
|
chat_id: str,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
thread_id: Optional[str] = None,
|
||||||
|
media_files: Optional[List[str]] = None,
|
||||||
|
force_document: bool = False,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Out-of-process delivery for cron jobs that run separately from the
|
||||||
|
gateway.
|
||||||
|
|
||||||
|
The outbox is served by the *running* gateway, so standalone delivery
|
||||||
|
while the gateway process is fully down is best-effort only (see
|
||||||
|
``docs/00-overview.md`` "Out of scope"). For M0 this is a stub that
|
||||||
|
reports the gateway is required; the real implementation lands with the
|
||||||
|
outbox (M3/M5).
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"error": (
|
||||||
|
"android standalone send: the running gateway is required to serve "
|
||||||
|
"the outbox (standalone delivery is best-effort only)"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Interactive setup (hermes gateway setup flow) -- full version in M1
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def interactive_setup() -> None:
|
||||||
|
"""Prompt for the pairing token / host / port / push backend.
|
||||||
|
|
||||||
|
M0: minimal. M1 adds token generation, QR payload, and a live ``hello``
|
||||||
|
connectivity test.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from hermes_cli.config import (
|
||||||
|
get_env_value,
|
||||||
|
save_env_value,
|
||||||
|
prompt,
|
||||||
|
print_info,
|
||||||
|
print_success,
|
||||||
|
print_warning,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
print("android: setup helpers unavailable; set ANDROID_TOKEN in ~/.hermes/.env")
|
||||||
|
return
|
||||||
|
|
||||||
|
print_info("📱 Android / Desktop (Iris x Hermes)")
|
||||||
|
token = get_env_value("ANDROID_TOKEN") or ""
|
||||||
|
if not token:
|
||||||
|
generated = uuid.uuid4().hex + uuid.uuid4().hex # 64 hex chars
|
||||||
|
save_env_value("ANDROID_TOKEN", generated)
|
||||||
|
print_success(f"Generated pairing token: {generated}")
|
||||||
|
print_warning("Keep this secret -- the app presents it on connect.")
|
||||||
|
else:
|
||||||
|
print_info("Existing ANDROID_TOKEN found (not shown).")
|
||||||
|
|
||||||
|
host = prompt("WS bind host", default=get_env_value("ANDROID_WS_HOST") or DEFAULT_HOST)
|
||||||
|
save_env_value("ANDROID_WS_HOST", host or DEFAULT_HOST)
|
||||||
|
port = prompt("WS port", default=str(_parse_port(get_env_value("ANDROID_WS_PORT") or "")))
|
||||||
|
save_env_value("ANDROID_WS_PORT", str(_parse_port(port)))
|
||||||
|
backend = prompt("Push backend (fcm/ntfy)", default=get_env_value("ANDROID_PUSH_BACKEND") or DEFAULT_PUSH_BACKEND)
|
||||||
|
save_env_value("ANDROID_PUSH_BACKEND", (backend or DEFAULT_PUSH_BACKEND).strip().lower())
|
||||||
|
|
||||||
|
print_success("Android configuration saved to ~/.hermes/.env")
|
||||||
|
print_info("Restart the gateway for changes to take effect: hermes gateway restart")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Android Adapter
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class AndroidAdapter(BasePlatformAdapter):
|
||||||
|
"""WebSocket-backed adapter for the native Iris Android / Desktop app.
|
||||||
|
|
||||||
|
M0: skeleton. Implements the abstract adapter contract as no-ops and
|
||||||
|
resolves configuration. The WebSocket server, connection registry,
|
||||||
|
pairing, streaming, media, outbox, push, and search are added in later
|
||||||
|
milestones.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, config, **kwargs):
|
||||||
|
platform = Platform("android")
|
||||||
|
super().__init__(config=config, platform=platform)
|
||||||
|
|
||||||
|
extra = getattr(config, "extra", {}) or {}
|
||||||
|
|
||||||
|
# Connection settings (env vars override config.yaml)
|
||||||
|
self.host = os.getenv("ANDROID_WS_HOST", "").strip() or extra.get("host", DEFAULT_HOST)
|
||||||
|
self.port = _parse_port(os.getenv("ANDROID_WS_PORT", "") or str(extra.get("port", DEFAULT_PORT)))
|
||||||
|
self.token = _get_scoped_secret("ANDROID_TOKEN") or extra.get("token", "")
|
||||||
|
self.home_channel = extra.get("home_channel", DEFAULT_HOME_CHANNEL)
|
||||||
|
self.push_backend = (
|
||||||
|
os.getenv("ANDROID_PUSH_BACKEND", "").strip().lower()
|
||||||
|
or extra.get("push_backend", DEFAULT_PUSH_BACKEND)
|
||||||
|
)
|
||||||
|
self.outbox_retention_hours = int(
|
||||||
|
extra.get("outbox_retention_hours", DEFAULT_OUTBOX_RETENTION_HOURS)
|
||||||
|
)
|
||||||
|
self.max_upload_bytes = int(
|
||||||
|
extra.get("max_upload_bytes", DEFAULT_MAX_UPLOAD_BYTES)
|
||||||
|
)
|
||||||
|
|
||||||
|
# TLS (optional)
|
||||||
|
self.ws_cert = _get_scoped_secret("ANDROID_WS_CERT") or extra.get("ws_cert", "")
|
||||||
|
self.ws_key = _get_scoped_secret("ANDROID_WS_KEY") or extra.get("ws_key", "")
|
||||||
|
|
||||||
|
# Auth
|
||||||
|
allowed = os.getenv("ANDROID_ALLOWED_USERS", "").strip()
|
||||||
|
self.allowed_users: List[str] = (
|
||||||
|
[u.strip() for u in allowed.split(",") if u.strip()] if allowed else []
|
||||||
|
)
|
||||||
|
self.allow_all = _truthy(os.getenv("ANDROID_ALLOW_ALL_USERS"))
|
||||||
|
|
||||||
|
# Runtime state (populated by the WS server in M1)
|
||||||
|
self._ws_server = None
|
||||||
|
self._connections: Dict[str, Any] = {}
|
||||||
|
self._connected = False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "Android"
|
||||||
|
|
||||||
|
# ── Connection lifecycle ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
async def connect(self, *, is_reconnect: bool = False) -> bool:
|
||||||
|
"""Bring the platform up.
|
||||||
|
|
||||||
|
M0: no WebSocket server yet -- just validate config and mark
|
||||||
|
connected so ``hermes gateway status`` reflects the platform. M1
|
||||||
|
starts the ``websockets`` server here.
|
||||||
|
"""
|
||||||
|
if not self.token:
|
||||||
|
logger.error("android: ANDROID_TOKEN must be set")
|
||||||
|
self._set_fatal_error(
|
||||||
|
"config_missing",
|
||||||
|
"ANDROID_TOKEN must be set",
|
||||||
|
retryable=False,
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Prevent two profiles from binding the same port/identity.
|
||||||
|
try:
|
||||||
|
from gateway.status import acquire_scoped_lock
|
||||||
|
lock_key = f"{self.host}:{self.port}"
|
||||||
|
if not acquire_scoped_lock("android", lock_key):
|
||||||
|
logger.error("android: %s:%s already in use by another profile", self.host, self.port)
|
||||||
|
self._set_fatal_error(
|
||||||
|
"lock_conflict",
|
||||||
|
"WS port in use by another profile",
|
||||||
|
retryable=False,
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
self._lock_key = lock_key
|
||||||
|
except ImportError:
|
||||||
|
self._lock_key = None # status module not available (e.g. tests)
|
||||||
|
|
||||||
|
# M1: start the websockets server on host:port (TLS if cert/key set).
|
||||||
|
self._connected = True
|
||||||
|
self._mark_connected()
|
||||||
|
logger.info("android: connected (skeleton; WS server starts in M1) on %s:%s", self.host, self.port)
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def disconnect(self) -> None:
|
||||||
|
"""Tear down the platform."""
|
||||||
|
try:
|
||||||
|
from gateway.status import release_scoped_lock
|
||||||
|
if getattr(self, "_lock_key", None):
|
||||||
|
release_scoped_lock("android", self._lock_key)
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
# M1: stop the server and close all device sockets.
|
||||||
|
self._connected = False
|
||||||
|
self._mark_disconnected()
|
||||||
|
logger.info("android: disconnected")
|
||||||
|
|
||||||
|
# ── Outbound (agent -> app) ───────────────────────────────────────────
|
||||||
|
|
||||||
|
async def send(
|
||||||
|
self,
|
||||||
|
chat_id: str,
|
||||||
|
content: str,
|
||||||
|
reply_to: Optional[str] = None,
|
||||||
|
metadata: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> SendResult:
|
||||||
|
"""Send a message to a chat.
|
||||||
|
|
||||||
|
M0: no live devices yet -- log and report success with a minted id.
|
||||||
|
M1: broadcast a ``message`` frame to connected devices, else fall to
|
||||||
|
the outbox + fire push.
|
||||||
|
"""
|
||||||
|
message_id = f"msg_{uuid.uuid4().hex}"
|
||||||
|
logger.debug("android: send to %s (%d chars) [skeleton no-op]", chat_id, len(content or ""))
|
||||||
|
return SendResult(success=True, message_id=message_id)
|
||||||
|
|
||||||
|
async def send_typing(self, chat_id: str, metadata: Optional[Dict[str, Any]] = None) -> None:
|
||||||
|
"""Send a typing indicator. M0: no-op (M1 emits a ``typing`` frame)."""
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def send_image(
|
||||||
|
self,
|
||||||
|
chat_id: str,
|
||||||
|
image_url: str,
|
||||||
|
caption: Optional[str] = None,
|
||||||
|
reply_to: Optional[str] = None,
|
||||||
|
metadata: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> SendResult:
|
||||||
|
"""Send an image. M0: not implemented (M4)."""
|
||||||
|
return SendResult(success=False, error="android: media not implemented yet (M4)")
|
||||||
|
|
||||||
|
# ── Chat info ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||||
|
"""Return ``{name, type, chat_id}`` for a chat.
|
||||||
|
|
||||||
|
M0: the channel directory is not persisted yet, so report the home
|
||||||
|
channel name for the default chat and a generic name otherwise.
|
||||||
|
"""
|
||||||
|
name = "Default" if chat_id in (self.home_channel, DEFAULT_HOME_CHANNEL) else (chat_id or "chat")
|
||||||
|
return {"name": name, "type": "channel", "chat_id": chat_id}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def register(ctx):
|
||||||
|
"""Plugin entry point: called by the Hermes plugin system."""
|
||||||
|
ctx.register_platform(
|
||||||
|
name="android",
|
||||||
|
label="Android",
|
||||||
|
adapter_factory=lambda cfg: AndroidAdapter(cfg),
|
||||||
|
check_fn=check_requirements,
|
||||||
|
validate_config=validate_config,
|
||||||
|
is_connected=is_connected,
|
||||||
|
required_env=["ANDROID_TOKEN"],
|
||||||
|
install_hint="No extra packages needed (websockets + httpx are core deps)",
|
||||||
|
setup_fn=interactive_setup,
|
||||||
|
# Env-driven auto-configuration: seeds PlatformConfig.extra with
|
||||||
|
# host/port/push_backend + home_channel so env-only setups show up in
|
||||||
|
# gateway status without instantiating the adapter.
|
||||||
|
env_enablement_fn=_env_enablement,
|
||||||
|
# Cron home-channel delivery support (deliver=android:<chat>[:<thread>]).
|
||||||
|
cron_deliver_env_var="ANDROID_HOME_CHANNEL",
|
||||||
|
# Out-of-process cron delivery (best-effort; outbox is gateway-served).
|
||||||
|
standalone_sender_fn=_standalone_send,
|
||||||
|
# Native target syntax: "android:<chat>[:<thread>]".
|
||||||
|
parse_target_ref_fn=_parse_target_ref,
|
||||||
|
# Auth env vars for _is_user_authorized() integration.
|
||||||
|
allowed_users_env="ANDROID_ALLOWED_USERS",
|
||||||
|
allow_all_env="ANDROID_ALLOW_ALL_USERS",
|
||||||
|
# WS has no message-size limit.
|
||||||
|
max_message_length=0,
|
||||||
|
# Display.
|
||||||
|
emoji="📱",
|
||||||
|
pii_safe=False,
|
||||||
|
allow_update_command=True,
|
||||||
|
# LLM guidance.
|
||||||
|
platform_hint=(
|
||||||
|
"You are chatting with the user through their native Iris app "
|
||||||
|
"(Android/Desktop). It renders Markdown, inline code, images, "
|
||||||
|
"audio and video, and shows your reasoning and tool activity. "
|
||||||
|
"Conversations are organized into channels and optional threads. "
|
||||||
|
"Keep formatting rich but readable."
|
||||||
|
),
|
||||||
|
)
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
"""Inbound media cache + outbound chunked streaming.
|
||||||
|
|
||||||
|
Inbound: ``media.upload`` (chunked binary frames) -> ``cache_*_from_bytes``
|
||||||
|
-> a ``media_ref`` the adapter attaches to the ``MessageEvent``. Enforces
|
||||||
|
size limit + sha256 + MIME re-sniff.
|
||||||
|
|
||||||
|
Outbound: ``send_*`` -> stage the file in the media cache, mint a
|
||||||
|
``media_id``, emit ``media.offer {media_id, mime, size, filename, kind}``;
|
||||||
|
serve bytes on ``media.pull`` as chunked binary frames. Delivery-path
|
||||||
|
security via ``validate_media_delivery_path``.
|
||||||
|
|
||||||
|
Reuses hermes ``cache_image/audio/video/document_from_bytes`` where possible.
|
||||||
|
All paths under ``get_hermes_home()/"android"/media``.
|
||||||
|
|
||||||
|
Milestone M4.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
"""SQLite offline outbox + monotonic sync cursor.
|
||||||
|
|
||||||
|
Undelivered frames are appended per ``chat_id`` with a monotonic cursor so a
|
||||||
|
reconnecting app can ``sync {cursor}`` the delta without re-reading full
|
||||||
|
history. Retention prunes old entries (``outbox_retention_hours``).
|
||||||
|
|
||||||
|
Storage: ``get_hermes_home()/"android"/outbox.db``.
|
||||||
|
|
||||||
|
Milestone M3 (built), extended in M5 (push integration).
|
||||||
|
"""
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
"""Pairing: token generation/verification + device registry.
|
||||||
|
|
||||||
|
Token generation (64-hex) and constant-time verification. Device registry
|
||||||
|
(SQLite) tracks ``device_id``, name, caps, fcm_token, ntfy_topic, last_seen,
|
||||||
|
created. QR payload for the pairing flow (``interactive_setup``).
|
||||||
|
|
||||||
|
Storage: ``get_hermes_home()/"android"/devices.db``.
|
||||||
|
|
||||||
|
Milestone M1.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
name: android-platform
|
||||||
|
label: Android
|
||||||
|
kind: platform
|
||||||
|
version: 0.1.0
|
||||||
|
description: >
|
||||||
|
Native Android / Desktop client gateway adapter for Hermes Agent.
|
||||||
|
Runs a WebSocket server inside the gateway; the app connects with a
|
||||||
|
pairing token. Supports streaming, reasoning, structured tool events,
|
||||||
|
channels/threads, media, FTS5 search, and FCM/ntfy push.
|
||||||
|
author: Iris x Hermes
|
||||||
|
# ``requires_env`` / ``optional_env`` entries are surfaced in the
|
||||||
|
# ``hermes config`` / ``hermes gateway setup`` UI via the platform-plugin
|
||||||
|
# env var injector in ``hermes_cli/config.py``.
|
||||||
|
requires_env:
|
||||||
|
- name: ANDROID_TOKEN
|
||||||
|
description: "Shared pairing token the app presents on connect"
|
||||||
|
prompt: "Android pairing token"
|
||||||
|
password: true
|
||||||
|
optional_env:
|
||||||
|
- name: ANDROID_WS_HOST
|
||||||
|
description: "WS bind host (default 127.0.0.1; use 0.0.0.0 for LAN)"
|
||||||
|
prompt: "WS host"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_PORT
|
||||||
|
description: "WS port (default 8790)"
|
||||||
|
prompt: "WS port"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_HOME_CHANNEL
|
||||||
|
description: "Default chat id for cron/notification delivery (default android:default)"
|
||||||
|
prompt: "Home channel"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_ALLOWED_USERS
|
||||||
|
description: "Comma-separated allowed device_ids (empty = token-only auth)"
|
||||||
|
prompt: "Allowed device ids"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_ALLOW_ALL_USERS
|
||||||
|
description: "Allow any paired device (dev only)"
|
||||||
|
prompt: "Allow all devices? (true/false)"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_PUSH_BACKEND
|
||||||
|
description: "Push backend: fcm (default) or ntfy"
|
||||||
|
prompt: "Push backend"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_FCM_SERVICE_ACCOUNT
|
||||||
|
description: "Path to Firebase service-account JSON (FCM HTTP v1)"
|
||||||
|
prompt: "FCM service account path"
|
||||||
|
password: true
|
||||||
|
- name: ANDROID_FCM_SERVER_KEY
|
||||||
|
description: "Legacy FCM server key (fallback if no service account)"
|
||||||
|
prompt: "FCM server key"
|
||||||
|
password: true
|
||||||
|
- name: NTFY_TOPIC
|
||||||
|
description: "ntfy topic for push (when ANDROID_PUSH_BACKEND=ntfy)"
|
||||||
|
prompt: "ntfy topic"
|
||||||
|
password: false
|
||||||
|
- name: NTFY_SERVER_URL
|
||||||
|
description: "ntfy server URL (default https://ntfy.sh)"
|
||||||
|
prompt: "ntfy server URL"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_CERT
|
||||||
|
description: "TLS cert path for WSS (optional)"
|
||||||
|
prompt: "WSS cert"
|
||||||
|
password: false
|
||||||
|
- name: ANDROID_WS_KEY
|
||||||
|
description: "TLS key path for WSS (optional)"
|
||||||
|
prompt: "WSS key"
|
||||||
|
password: false
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""Frame schemas -- the single source of truth for the wire protocol.
|
||||||
|
|
||||||
|
Every frame the plugin sends/receives is modelled here as a dataclass +
|
||||||
|
constants. ``docs/protocol/frames.schema.json`` is generated/mirrored from
|
||||||
|
this module, and the Kotlin side mirrors these shapes (see
|
||||||
|
``docs/04-wire-protocol.md``).
|
||||||
|
|
||||||
|
Milestone M1: hello/hello.ack, message, error, ping/pong.
|
||||||
|
Milestone M2: message.start/update/stop, reasoning, tool.*, commentary, typing.
|
||||||
|
Milestone M3: channel.*, search, sync.
|
||||||
|
Milestone M4: media.*.
|
||||||
|
Milestone M5: notification, fcm.register, read.receipt.
|
||||||
|
"""
|
||||||
|
|
||||||
|
PROTOCOL_VERSION = 1
|
||||||
|
|
||||||
|
# Frame type constants (the ``type`` field of every frame).
|
||||||
|
TYPE_HELLO = "hello"
|
||||||
|
TYPE_HELLO_ACK = "hello.ack"
|
||||||
|
TYPE_ERROR = "error"
|
||||||
|
TYPE_PING = "ping"
|
||||||
|
TYPE_PONG = "pong"
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
"""Push backends: FCM (primary) + ntfy (fallback).
|
||||||
|
|
||||||
|
``PushBackend`` interface with two implementations:
|
||||||
|
- ``FcmBackend``: FCM HTTP v1 via ``httpx`` + a Firebase service account
|
||||||
|
(``ANDROID_FCM_SERVICE_ACCOUNT``), or a legacy server key
|
||||||
|
(``ANDROID_FCM_SERVER_KEY``).
|
||||||
|
- ``NtfyBackend``: reuses hermes ntfy publish (``NTFY_TOPIC`` /
|
||||||
|
``NTFY_SERVER_URL``).
|
||||||
|
|
||||||
|
Selected by ``ANDROID_PUSH_BACKEND`` (``fcm`` default, ``ntfy`` fallback).
|
||||||
|
Fired when a frame has no live subscriber; data payload drives a silent sync
|
||||||
|
on the device.
|
||||||
|
|
||||||
|
Milestone M5.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
"""FTS5 session search bridge.
|
||||||
|
|
||||||
|
Bridges the ``search {query, scope, chat_id?, thread_id?}`` frame to the
|
||||||
|
hermes session store (SQLite + FTS5, ``hermes_state_search.py``) and returns
|
||||||
|
``search.results``. Scope: "all" (everywhere) or "chat" (this chat/channel).
|
||||||
|
|
||||||
|
Milestone M3.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
# Tests for the android gateway plugin.
|
||||||
|
|
||||||
|
Run via hermes's hermetic runner (never bare pytest)::
|
||||||
|
|
||||||
|
scripts/run_tests.sh tests/gateway/test_android.py
|
||||||
|
|
||||||
|
See ``docs/13-testing.md`` for the scenario list.
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
"""WebSocket server, connection registry, and frame routing.
|
||||||
|
|
||||||
|
Runs on the gateway's asyncio loop (started in ``AndroidAdapter.connect()``).
|
||||||
|
Uses the ``websockets`` core dep (v15): ``websockets.serve(handler, host,
|
||||||
|
port, ssl=ctx)``.
|
||||||
|
|
||||||
|
Per-connection handler:
|
||||||
|
1. Await first frame; must be ``hello {token, device_id, device_name,
|
||||||
|
caps, fcm_token?}``. Verify token (constant-time) + allowlist. On
|
||||||
|
failure: send ``error {code:"auth"}`` and close.
|
||||||
|
2. On success: register in the connection registry (``device_id ->
|
||||||
|
{ws, caps, fcm_token}``), send ``hello.ack {server_caps, sync_cursor,
|
||||||
|
channels[]}``.
|
||||||
|
3. Loop: decode frames, dispatch to adapter inbound handlers.
|
||||||
|
4. On close: deregister; if no devices remain, ensure pending outbox
|
||||||
|
frames have push fired.
|
||||||
|
|
||||||
|
Routing: ``emit(chat_id, frame)`` broadcasts to ALL connected devices
|
||||||
|
(single-user model). Heartbeat via WS ping/pong + app-level ping/pong.
|
||||||
|
Backpressure: bounded per-connection send queue; coalesce ``message.update``
|
||||||
|
under pressure, never drop ``message``/``tool.end``/``notification``.
|
||||||
|
|
||||||
|
Milestone M1.
|
||||||
|
"""
|
||||||
Executable
+37
@@ -0,0 +1,37 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Guard: hermes-agent/ is a read-only research reference and must NEVER be
|
||||||
|
# committed, pushed, or shipped. This fails the commit/CI if any path under
|
||||||
|
# hermes-agent/ is staged.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# - pre-commit hook: scripts/guard_hermes_agent.sh --staged
|
||||||
|
# - CI / manual: scripts/guard_hermes_agent.sh --staged
|
||||||
|
#
|
||||||
|
# Exit 0 = clean, exit 1 = hermes-agent/ paths detected.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
MODE="${1:---staged}"
|
||||||
|
|
||||||
|
if [[ "$MODE" == "--staged" ]]; then
|
||||||
|
# Every path currently in the index. Using `git ls-files --cached` (rather
|
||||||
|
# than `git diff --cached`) makes this robust on the very first commit,
|
||||||
|
# where no HEAD exists yet and `git diff --cached` is unreliable. We never
|
||||||
|
# want hermes-agent/ in the index at all, so blocking on its presence is
|
||||||
|
# both correct and safe.
|
||||||
|
mapfile -t BAD < <(git ls-files --cached | grep -E '^hermes-agent/' || true)
|
||||||
|
else
|
||||||
|
# Fallback: any tracked file under hermes-agent/.
|
||||||
|
mapfile -t BAD < <(git ls-files | grep -E '^hermes-agent/' || true)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ${#BAD[@]} -gt 0 ]]; then
|
||||||
|
echo "ERROR: the following hermes-agent/ paths are staged. This directory is a" >&2
|
||||||
|
echo "read-only research reference and must NEVER be committed (see .gitignore" >&2
|
||||||
|
echo "and docs/00-overview.md 'Disclaimers'). Unstage them with:" >&2
|
||||||
|
echo " git restore --staged <path>" >&2
|
||||||
|
printf ' %s\n' "${BAD[@]}" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
exit 0
|
||||||
Reference in new issue
Block a user