Files
agent-estate-wiki/incidents/2026-08-01-mnemosyne-capture-failure.md
T

89 lines
4.8 KiB
Markdown

---
title: Mnemosyne Automatic Capture Failure
type: incident
status: active
created: 2026-08-01
updated: 2026-10-20
resolved_on: 2026-10-20
verified_on: 2026-10-20
confidence: high
tags: [memory, mnemosyne, hermes, incident]
sources: [live-config, mnemosyne-db, hermes-cli-e2e]
---
# Mnemosyne Automatic Capture Failure — 2026-08-01
## Impact
Ordinary Hermes conversations were not being written to Mnemosyne `working_memory`. Explicit memory tools, SQLite integrity, vectors, FTS indexes, consolidation, backups, and cron monitoring remained operational. The newest automatic/explicit working row before repair was approximately fourteen hours old despite active Hermes sessions.
The always-injected legacy files were also effectively full: `MEMORY.md` was 5,055 bytes against a 5,000-character limit and `USER.md` was 2,958 bytes against 3,000. Legacy writes had begun failing.
## Root Cause
Structured values under `memory.mnemosyne` in `~/.hermes/config.yaml` had been serialized as strings instead of native YAML types. The critical value was:
```yaml
sync_roles: "['user']"
```
The provider interpreted this as the literal invalid role `['user']`, so turn synchronization completed without persisting a user row. Other malformed values included `skip_contexts`, `reflect`, `sleep_threshold`, and `ignore_patterns`.
## Repair
- Backed up config to `/home/hermes/.hermes/backups/mnemosyne-capture-fix-20260801_180249/config.yaml.before`.
- Rewrote structured Mnemosyne settings with native YAML types.
- Set `profile_isolation: true` because the installation has multiple Hermes profiles.
- Preserved `sync_roles: [user]` and the intended skipped contexts.
- Ran `hermes config check` successfully.
- Created authoritative Mnemosyne canonical slots for user preferences and key workflows, correcting stale light-theme, numbered-Obsidian, and CT460 route entries.
- Backed up legacy files to `/home/hermes/.hermes/backups/legacy-memory-slim-20260801_180702/`.
- Reduced legacy injection from 8,013 bytes to 1,080 bytes while retaining essential always-on rules.
## Verification
A pre-fix CLI turn using marker `MNEMO_E2E_20260801_180113_27053` completed but produced no database row.
A post-fix CLI turn using marker `MNEMO_FIXED_20260801_180310_16583` wrote:
```text
[USER] Reply with exactly OK. Memory capture test marker: MNEMO_FIXED_20260801_180310_16583
```
The row was verified directly in `~/.hermes/mnemosyne/data/mnemosyne.db`. Final diagnostics reported zero failed checks, complete working-vector coverage, zero orphan embeddings, 1,482 working rows, and 463 episodic vectors.
## Remaining Activation Step
The already-running Telegram gateway/provider instance was initialized before the config repair. Run `hermes gateway restart` from an external terminal to load the corrected typed configuration for messaging-platform turns. The gateway cannot safely restart itself from one of its own child tool calls.
## Monitoring Note
The health script checks capture staleness only after 24 hours. It therefore remained green during this fourteen-hour active-capture outage. A future improvement should compare recent non-cron conversation activity with the newest `working_memory` row rather than lowering the threshold blindly and creating idle-period false positives.
## Resolution confirmed (2026-10-20)
The remaining "Activation Step" on this page — restarting the gateway to load the corrected typed
configuration — **has been completed and the config is still correct 50 days later.** Verified live
in `/home/hermes/.hermes/config.yaml` under `memory.mnemosyne`:
| Key | Live value | Was it a string? |
|---|---|---|
| `sync_roles` | `['user']` (real list) | no — now a native list |
| `skip_contexts` | `['cron', 'flush', 'subagent', 'background', 'skill_loop']` | no — native list |
| `reflect` | `{disabled_for_cron: True, max_calls_per_session: 3}` (dict) | no — native dict |
| `sleep_threshold` | `50` (int) | no — int |
| `ignore_patterns` | `[]` (list) | no — native list |
| `profile_isolation` | `True` (bool) | no — bool |
| `auto_sleep` / `default_scope` / `vector_type` | `True` / `session` / `int8` | no |
**None of these are stringified.** The `hermes config set` stringification bug that caused this
incident has not recurred for this block.
**One drift to note:** `skip_contexts` now includes `cron`, which was not in the original repair
list. Mnemosyne capture is therefore **deliberately skipped in cron contexts** — scheduled jobs
produce no working memories by design. That explains why cron-driven runs of this wiki job leave
no Mnemosyne trace.
**Status: `resolved`.** The back-up copies named in the repair section still exist:
`~/.hermes/backups/mnemosyne-capture-fix-20260801_180249/` and
`~/.hermes/backups/legacy-memory-slim-20260801_180702/`.
## Related
- [[systems/mnemosyne]]
- [[systems/memory-backend]]
- [[incidents/2026-07-29-mnemosyne-relay]]