feat: same-turn memory retrieval in provider mode

prefetch() now reuses the in-flight background search when it matches
the current query, or falls back to a synchronous search (capped at 3s,
split across both requests in case-insensitive mode) so memories are
injected on the same turn instead of one turn behind.

- Track _prefetch_query to scope cached results to their query
- Reset _prefetch_result when starting a new background search to
  prevent stale cross-turn injection after a timed-out join
- Guard background thread writes against superseded queries
- Add optional per-request timeout override to LocalMem0Client
- Update AGENTS.md and README to describe the same-turn flow
This commit is contained in:
2026-08-15 19:52:25 +02:00
parent fad5eb7f1e
commit c79cf43ff8
4 changed files with 64 additions and 25 deletions
+5 -6
View File
@@ -5,7 +5,7 @@ Self-hosted Mem0 memory provider for Hermes-Agent. Provides semantic memory sear
## Features
- **Local Mem0 server** — No cloud dependency, full data privacy
- **Async prefetch** — Memory retrieval happens in background (~40ms)
- **Same-turn prefetch** — Memory retrieved synchronously before the LLM call (~40ms)
- **Context injection** — Relevant memories injected directly into LLM prompt
- **Automatic fact extraction** — Server-side LLM extracts facts from conversations
- **Semantic search** — Find memories by meaning, not keywords
@@ -138,16 +138,15 @@ hermes gateway restart
### How It Works
1. **User message received** → `queue_prefetch()` spawns background thread
2. **Mem0 search** → Semantic search for relevant memories (~40ms)
3. **Context injection** → Results injected via `pre_llm_call` hook
4. **LLM receives** → User message + memory context (no tool call needed!)
1. **User message received** → `prefetch()` searches Mem0 synchronously (~40ms)
2. **Context injection** → Results injected before the LLM call
3. **LLM receives** → User message + memory context (no tool call needed!)
**Example**:
```
User: "Hey, is a new episode out from my favorite anime?"
↓ [Background: mem0.prefetch() searches for "favorite anime"]
↓ [mem0.prefetch() searches for "favorite anime"]
LLM receives:
"""