110 lines
5.3 KiB
Markdown
110 lines
5.3 KiB
Markdown
---
|
||
title: Conduit Voice Call Loses Its Memory Context (Mnemosyne Start Timeout)
|
||
type: incident
|
||
status: resolved
|
||
created: 2026-10-24
|
||
updated: 2026-10-25
|
||
verified_on: 2026-10-25
|
||
review_after: 2026-11-08
|
||
confidence: high
|
||
tags: [conduit, mnemosyne, timeout, plugin, voice, incident, resolved]
|
||
sources: [live-log, plugin-tree, live-sqlite-conduit, git-diff]
|
||
---
|
||
|
||
# Conduit Voice Call Loses Its Memory Context — 2026-10-22 → 2026-10-23
|
||
|
||
**Window:** 2026-10-22 11:28 → 2026-10-23 18:29 AWST (three occurrences)
|
||
**Status:** Resolved 2026-10-23 18:54 (patched and verified)
|
||
**Impact:** A Conduit voice call that arrived before the Mnemosyne provider was warm lost its
|
||
memory context — and, in the pre-fix code path, the *call itself* was abandoned by the plugin's
|
||
memory layer.
|
||
|
||
## Symptom
|
||
|
||
Three identical warnings in `~/.hermes/logs/agent.log` (also in `errors.log`), one per call attempt
|
||
that hit a cold memory provider:
|
||
|
||
```
|
||
2026-10-22 11:28:17,137 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
|
||
2026-10-22 13:16:37,867 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
|
||
2026-10-23 18:29:16,830 WARNING hermes_dashboard_plugin_conduit_push: Memory provider 'mnemosyne' start for Conduit timed out; abandoning the call
|
||
2026-10-22 13:16:37,875 WARNING hermes_dashboard_plugin_conduit_push: Memory context for Conduit failed: Memory start timed out
|
||
```
|
||
|
||
## Root cause
|
||
|
||
`dashboard/plugin_api.py` in the `conduit_push` plugin hard-coded `MEMORY_TIMEOUT_S = 10.0` — a
|
||
budget for **one request including the provider start**. Mnemosyne's cold start for the
|
||
`conduit-voice` session takes roughly **10–45 s**, so any call landing on a cold provider blew the
|
||
budget. The plugin then ran the call **without memory context**, and the failure was logged as
|
||
`Memory start timed out` — the agent speaks to the user with no recall of prior sessions and no
|
||
error surfaced to the caller.
|
||
|
||
This is not an OmniRoute/model fault, and not a Mnemosyne data fault: the store was healthy and
|
||
the provider simply needed longer than the plugin allowed. → [[systems/mnemosyne]]
|
||
|
||
## Fix applied (2026-10-23, live before 18:54:43)
|
||
|
||
A single-file change to `~/.hermes/plugins/conduit_push/dashboard/plugin_api.py`, **still
|
||
uncommitted in the working tree**:
|
||
|
||
```diff
|
||
-MEMORY_TIMEOUT_S = 10.0
|
||
+# Mnemosyne (the memory provider) can take ~10-45s to cold-start for the
|
||
+# conduit-voice session, so give it enough room; tunable via env.
|
||
+MEMORY_TIMEOUT_S = float(os.environ.get("CONDUIT_MEMORY_TIMEOUT_S", "60.0"))
|
||
```
|
||
|
||
and a hard-failure guard so a token/start error degrades instead of aborting:
|
||
|
||
```diff
|
||
- entry = _memory_providers.get(key, name, deadline)
|
||
+ try:
|
||
+ entry = _memory_providers.get(key, name, deadline)
|
||
+ except TokenError:
|
||
+ logger.warning("Memory provider %r failed to start for Conduit; proceeding without recall", name)
|
||
+ entry = None
|
||
```
|
||
|
||
The new default is used by every deadline in the memory layer (6 call sites) plus the
|
||
`+ 2.0 s` backstop around each provider call. `CONDUIT_MEMORY_TIMEOUT_S` overrides it without
|
||
touching the file.
|
||
|
||
## Verification
|
||
|
||
- Dashboard process on `:9119` restarted **2026-10-23 18:54:43** — after the 18:29 abort and
|
||
therefore running the patched code.
|
||
- **Zero** `abandoning the call` / `Memory context for Conduit failed` lines exist after that
|
||
restart (`grep` across `~/.hermes/logs/*.log`, deduped).
|
||
- ~~Two Conduit call sessions closed cleanly after the restart — **2026-10-24 00:03:23** and
|
||
**00:06:57**, both `outcome: done`.~~ **Corrected 2026-10-25.** Those two
|
||
`conduit-calls.json → ends` entries are the **cron** sessions of that night (Mnemosyne Embedding
|
||
Coverage Monitor 00:01, Health Monitor 00:04), not calls; `ends` records *every* turn-end that
|
||
fires a watch (`MAX_RECENT_ENDS = 64`). **`history` — real call records — is empty**, so the fix
|
||
has **no positive call evidence** and no call has been placed since it landed. The surviving
|
||
evidence is the absence of abort lines after the 18:54:43 restart, and nothing has falsified it.
|
||
- **Caveat, stated plainly:** the plugin logs only *failures*, so the absence of a warning is
|
||
not positive proof that recall was populated on those calls. The proven part is that the call
|
||
is no longer abandoned by the memory layer.
|
||
|
||
## Follow-ups
|
||
|
||
- ⚠ **The fix is still unexercised by a call** (see the correction above): `conduit-calls.json →
|
||
history` is empty at 2026-10-25 00:2x, and the plugin is now **five releases behind HEAD**
|
||
(`0.64.0` → `0.69.0`).
|
||
- **Follow-on failure (2026-10-24):** three requests to be phoned were refused by the agent, which
|
||
asserted the capability did not exist **without calling the tool** →
|
||
[[incidents/2026-10-24-conduit-call-request-refused]].
|
||
- **The patch is uncommitted.** A plugin update that resets
|
||
`~/.hermes/plugins/conduit_push` reverts it to the 10 s budget. Commit it or send it upstream
|
||
(`kaishi00/hermes-conduit-notifier`).
|
||
- The underlying Mnemosyne cold start (10–45 s) is unchanged — only the budget was raised.
|
||
Consumers with tight timeouts elsewhere will hit the same wall. → [[systems/mnemosyne]]
|
||
|
||
## Related
|
||
|
||
- [[incidents/2026-10-24-conduit-call-request-refused]]
|
||
- [[systems/conduit]]
|
||
- [[systems/mnemosyne]]
|
||
- [[incidents/2026-10-09-memory-recall-relevance]]
|