Files
agent-estate-wiki/incidents/2026-10-22-conduit-memory-start-timeout.md
T

110 lines
5.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Conduit Voice Call Loses Its Memory Context (Mnemosyne Start Timeout)
type: incident
status: resolved
created: 2026-10-24
updated: 2026-10-25
verified_on: 2026-10-25
review_after: 2026-11-08
confidence: high
tags: [conduit, mnemosyne, timeout, plugin, voice, incident, resolved]
sources: [live-log, plugin-tree, live-sqlite-conduit, git-diff]
---
# Conduit Voice Call Loses Its Memory Context — 2026-10-22 → 2026-10-23
**Window:** 2026-10-22 11:28 → 2026-10-23 18:29 AWST (three occurrences)
**Status:** Resolved 2026-10-23 18:54 (patched and verified)
**Impact:** A Conduit voice call that arrived before the Mnemosyne provider was warm lost its
memory context — and, in the pre-fix code path, the *call itself* was abandoned by the plugin's
memory layer.
## Symptom
Three identical warnings in `~/.hermes/logs/agent.log` (also in `errors.log`), one per call attempt
that hit a cold memory provider:
```
2026-10-22 11:28:17,137 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
2026-10-22 13:16:37,867 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
2026-10-23 18:29:16,830 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
2026-10-22 13:16:37,875 WARNING hermes_dashboard_plugin_conduit_push: Memory context for Conduit failed: Memory start timed out
```
## Root cause
`dashboard/plugin_api.py` in the `conduit_push` plugin hard-coded `MEMORY_TIMEOUT_S = 10.0` — a
budget for **one request including the provider start**. Mnemosyne's cold start for the
`conduit-voice` session takes roughly **10–45 s**, so any call landing on a cold provider blew the
budget. The plugin then ran the call **without memory context**, and the failure was logged as
`Memory start timed out` — the agent speaks to the user with no recall of prior sessions and no
error surfaced to the caller.
This is not an OmniRoute/model fault, and not a Mnemosyne data fault: the store was healthy and
the provider simply needed longer than the plugin allowed. → [[systems/mnemosyne]]
## Fix applied (2026-10-23, live before 18:54:43)
A single-file change to `~/.hermes/plugins/conduit_push/dashboard/plugin_api.py`, **still
uncommitted in the working tree**:
```diff
-MEMORY_TIMEOUT_S = 10.0
+# Mnemosyne (the memory provider) can take ~10-45s to cold-start for the
+# conduit-voice session, so give it enough room; tunable via env.
+MEMORY_TIMEOUT_S = float(os.environ.get("CONDUIT_MEMORY_TIMEOUT_S", "60.0"))
```
and a hard-failure guard so a token/start error degrades instead of aborting:
```diff
- entry = _memory_providers.get(key, name, deadline)
+ try:
+ entry = _memory_providers.get(key, name, deadline)
+ except TokenError:
+ logger.warning("Memory provider %r failed to start for Conduit; proceeding without recall", name)
+ entry = None
```
The new default is used by every deadline in the memory layer (6 call sites) plus the
`+ 2.0 s` backstop around each provider call. `CONDUIT_MEMORY_TIMEOUT_S` overrides it without
touching the file.
## Verification
- Dashboard process on `:9119` restarted **2026-10-23 18:54:43** — after the 18:29 abort and
therefore running the patched code.
- **Zero** `abandoning the call` / `Memory context for Conduit failed` lines exist after that
restart (`grep` across `~/.hermes/logs/*.log`, deduped).
- ~~Two Conduit call sessions closed cleanly after the restart — **2026-10-24 00:03:23** and
**00:06:57**, both `outcome: done`.~~ **Corrected 2026-10-25.** Those two
`conduit-calls.json → ends` entries are the **cron** sessions of that night (Mnemosyne Embedding
Coverage Monitor 00:01, Health Monitor 00:04), not calls; `ends` records *every* turn-end that
fires a watch (`MAX_RECENT_ENDS = 64`). **`history` — real call records — is empty**, so the fix
has **no positive call evidence** and no call has been placed since it landed. The surviving
evidence is the absence of abort lines after the 18:54:43 restart, and nothing has falsified it.
- **Caveat, stated plainly:** the plugin logs only *failures*, so the absence of a warning is
not positive proof that recall was populated on those calls. The proven part is that the call
is no longer abandoned by the memory layer.
## Follow-ups
- ⚠ **The fix is still unexercised by a call** (see the correction above): `conduit-calls.json →
history` is empty at 2026-10-25 00:2x, and the plugin is now **five releases behind HEAD**
(`0.64.0` → `0.69.0`).
- **Follow-on failure (2026-10-24):** three requests to be phoned were refused by the agent, which
asserted the capability did not exist **without calling the tool** →
[[incidents/2026-10-24-conduit-call-request-refused]].
- **The patch is uncommitted.** A plugin update that resets
`~/.hermes/plugins/conduit_push` reverts it to the 10 s budget. Commit it or send it upstream
(`kaishi00/hermes-conduit-notifier`).
- The underlying Mnemosyne cold start (10–45 s) is unchanged — only the budget was raised.
Consumers with tight timeouts elsewhere will hit the same wall. → [[systems/mnemosyne]]
## Related
- [[incidents/2026-10-24-conduit-call-request-refused]]
- [[systems/conduit]]
- [[systems/mnemosyne]]
- [[incidents/2026-10-09-memory-recall-relevance]]