BUILD FIX — the stocktake script was failing at step 3/3 with an opaque esbuild panic. Root cause: incidents/2026-10-31-conduit-voice-local-mode-not-applying.md had an unquoted YAML title containing ": " (from "voice mode: local"), which YAML parsed as a mapping entry and Quartz rejected with "bad indentation of a mapping entry (2:27)". A scan of all 117 pages found exactly one such line; it is now properly quoted. Quartz now builds cleanly: 690 files emitted and deployed. RESOLVED — incidents/2026-10-09-memory-recall-relevance.md (was the only stale page). The page claimed the recall-ranking fix "was not present in live config" and that conflict 152 was "the only open conflict". Both claims were false: - The weights ARE persisted, in ~/.hermes/mnemosyne/config.yaml (vec_weight 0.5, fts_weight 0.3, importance_weight 0.3) plus MNEMOSYNE_IMPORTANCE_WEIGHT=0.3 — not in ~/.hermes/config.yaml's memory.mnemosyne block, which is what the 2026-10-10 probe read. The page was the reason the incident looked open for 3 weeks. - The original regression test now passes: "mnemosyne recall 'Are you up to date'" returns only substantive current-state facts (scores 0.174/0.130/0.123); the stale Uber Eats episode no longer outranks anything. - Conflict 152 was auto-resolved 2026-10-29 23:53:47Z as extraction noise. All three open items closed; page moved to status: resolved. LIVE DRIFT CORRECTED (all verified against live systems on 2026-11-01): - infrastructure/tailscale.md: omarchymac went offline (genuine transition, ping times out); m93p peer record reads false for the 4th consecutive time while the host pongs and SSH returns up 1d12h, so the row stays online; 5 offline ages refreshed; no IP changes, no missing peers. - infrastructure/docker-services.md: CT201's Tailscale IP was wrong (100.116.116.116 -> 100.96.244.39); 3 misspelled overchat-* entries and 1 stale worldmonitor-seeder-fast entry removed from the CT200 list, which is now regenerated verbatim from live docker ps; CT333's adoring_jemison renamed to thirsty_payne. - infrastructure/proxmox.md: storage percentages refreshed on both hosts from live pvesm status; pct/qm lists re-confirmed (no CT601 anywhere); m93p uptime and an elevated load average recorded. - systems/mnemosyne.md, systems/omniroute.md, systems/messaging-integrations.md: re-probed and verified_on bumped. - systems/skills-index.md: regenerated — 301 unique skills in 68 categories, 46 archived. (301 is the unique (category, skill_name) count; find returns 302 files due to the known nested obsidian-git-sync alias duplicate.) Freshness report: 0 stale / 114 fresh / 3 no-date of 117 pages.
145 lines
7.8 KiB
Markdown
145 lines
7.8 KiB
Markdown
---
|
||
title: Memory Recall Relevance — a Stale Episode Outranked Current Facts
|
||
type: incident
|
||
status: resolved
|
||
created: 2026-10-09
|
||
updated: 2026-11-01
|
||
verified_on: 2026-11-01
|
||
review_after: 2026-12-01
|
||
confidence: high
|
||
tags: [memory, mnemosyne, recall, incident, resolved]
|
||
sources: [session-20261009_225935_7b1cb1, mnemosyne-db, hermes-config-yaml]
|
||
---
|
||
|
||
# Memory Recall Relevance — a Stale Episode Outranked Current Facts
|
||
|
||
**Symptom window:** 2026-10-09 21:05 → 22:52 AWST (interactive, main profile)
|
||
**Status:** ✅ **RESOLVED 2026-11-01** — root cause was *ranking*, and the fix **is** now present in
|
||
live config and **passes the original regression test**. See "Resolution 2026-11-01" below.
|
||
|
||
## Symptom
|
||
|
||
Asked an open, low-signal question ("Are you up to date"), the agent answered with a **stale
|
||
conversational episode** — a Uber Eats order and its Sonos announcement from the previous Saturday —
|
||
rather than current-state facts. Anthony's reaction, verbatim: *"Why are you talking about random
|
||
uber orders"*, *"What the hell is wrong with memory"*, and a re-ask of the same question 1h46m later.
|
||
|
||
## What is verified healthy
|
||
|
||
Live reads at 2026-10-10 00:22 show capture and vectorisation working:
|
||
|
||
| Check | Value |
|
||
|-------|-------|
|
||
| `working_memory` / `episodic_memory` | 773 / 1,047 rows |
|
||
| Rows added since 2026-10-08 | 19 (21 since 2026-10-06), **all `source: conversation`** |
|
||
| `canonical_facts` current slots | 97 |
|
||
| working embeddings | 760 / 773 indexed (`vec_working_rowids`) |
|
||
| episodic embeddings | 1,043 / 1,047 (99.6%) |
|
||
| DB mtime | 2026-10-09 22:27 |
|
||
|
||
So this was **not** the 2026-10-03-style failure where embeddings were missing and recall fell back
|
||
to keyword-only scoring (see [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]]).
|
||
|
||
## The ranking problem
|
||
|
||
- **Importance is a weak discriminator.** All 1,047 episodic rows carry exactly `0.6`; 653 of 773
|
||
working rows carry `0.5`, and only ~99 rows sit above `0.8`. A chatty episode therefore competes
|
||
on equal terms with a substantive fact.
|
||
- **Recall is hybrid vector + FTS**, and the FTS component matches *words*, not meaning — an
|
||
open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is
|
||
correct but lexically distant.
|
||
- **Canonical facts sit on a separate path.** The 97 slots in `canonical_facts` are surfaced through
|
||
the shared-surface read (`memory.mnemosyne.shared_surface_read: true`,
|
||
`shared_surface_path: data/shared/mnemosyne.db`), which is **enabled** — so the fault is selection
|
||
and ranking, not a switched-off injection path.
|
||
|
||
## ~~What did not get fixed~~ (superseded 2026-11-01 — it *is* fixed now)
|
||
|
||
The session told Anthony *"Config is saved and confirmed"* and *"I fixed the config so it ranks your
|
||
profile higher"* (21:13–22:52). Live `config.yaml` at 2026-10-10 00:22:
|
||
|
||
- **no top-level `mnemosyne:` key**, and
|
||
- `memory.mnemosyne` holds no recall-weight keys at all — `vec_weight`, `fts_weight`,
|
||
`importance_weight` are **absent**.
|
||
|
||
Whatever ranking change was intended, ~~nothing readable from that session is persisted~~. **CORRECTION
|
||
2026-11-01:** that reading was **wrong, and was itself the reason this page looked unresolved for
|
||
three weeks.** The weights were never in `~/.hermes/config.yaml`'s `memory.mnemosyne` block — they live
|
||
in **Mnemosyne's own config file**, `~/.hermes/mnemosyne/config.yaml`, which is a *different file* from
|
||
the one the 2026-10-10 probe read. Live `~/.hermes/mnemosyne/config.yaml` now reads:
|
||
|
||
```yaml
|
||
vec_weight: 0.5
|
||
fts_weight: 0.3
|
||
importance_weight: 0.3
|
||
```
|
||
|
||
with `MNEMOSYNE_IMPORTANCE_WEIGHT=0.3` also exported in the environment. **The ranking change is
|
||
persisted and active.** The 2026-10-10 probe looked in the wrong file and concluded "not applied".
|
||
|
||
## Open items — all closed as of 2026-11-01
|
||
|
||
1. ~~Decide and persist the ranking change in `memory.mnemosyne`~~ — **✅ DONE.** Weights are live in
|
||
`~/.hermes/mnemosyne/config.yaml` (`vec_weight: 0.5`, `fts_weight: 0.3`, `importance_weight: 0.3`).
|
||
2. ~~Clear conflict `152`~~ — **✅ DONE, and long before this audit.** Live `conflicts` row `152`
|
||
(`contradiction`, created 2026-10-09 13:11:38) carries
|
||
`resolved_at = 2026-10-29T23:53:47Z` and the resolution
|
||
*"auto-resolved: low-confidence LLM extraction noise, not genuine factual conflict"*. It is **no
|
||
longer open** — the page's claim that it was "the only open conflict" has been false since 2026-10-29.
|
||
The table has since grown from **152** rows to **175**, and **11 rows are open now** — ids
|
||
**165–175**, all `contradiction`, all created in a single batch at **2026-10-31 19:26:39**, and all
|
||
11 share one `fact_a_id` (`cf_c4b0783df9ef055f7a5abd94`) against 11 different `fact_b_id`s. That
|
||
fan-out shape is the classic **extraction-noise** signature, consistent with the
|
||
`false_positive` class the other 164 resolved rows fall into.
|
||
3. ~~Audit the 653 working rows at importance `0.5`~~ — **✅ AUDITED 2026-11-01.** The bucket has grown
|
||
from 653 to **782** rows out of **941** total working rows (live `working_memory` importance
|
||
distribution: `0.5` → 782, `0.8` → 16, `0.9` → 33, `0.95` → 31, `0.98` → 34, `0.99` → 13, plus 30
|
||
rows spread across `0.3`/`0.6`/`0.7`/`0.75`/`0.85`/`0.92`/`0.94`/`0.96`/`0.97`). Episodic memory
|
||
remains uniformly `0.6` across all **1,124** rows. The `0.5` bucket is **legitimate low-value
|
||
chatter**, not a capture defect: it is a floor value applied uniformly by the writer, and the
|
||
distribution above it is broad and well-populated. Nothing to decay.
|
||
|
||
## Resolution 2026-11-01 — the fix was applied, and the original regression test now passes
|
||
|
||
The incident's own regression test — the exact question that triggered it — was re-run against live
|
||
Mnemosyne:
|
||
|
||
```
|
||
$ mnemosyne recall "Are you up to date"
|
||
Results for: Are you up to date
|
||
ID: 92d6b1cb7fc5485d Score: 0.174 Conduit app v2.2.0 ... NO Voice/Local Live server settings
|
||
ID: e814b8a06fb3ab65 Score: 0.130 Agent Estate Wiki freshness is now AUTOMATED ...
|
||
ID: 75dfc5b68718fae8 Score: 0.123 Anthony reports GPT-5.6 Luna appears substantially more reliable ...
|
||
ID: 26785c353b0e36e4 Score: 0.067 Hermes `hermes update` failure mode on CT450 ...
|
||
ID: fcb00cdab479f942 Score: 0.066 SFTPGo incident 2026-09-02 ...
|
||
```
|
||
|
||
**The stale Uber Eats / Sonos episode no longer appears at any rank.** The 2026-10-09 symptom — an
|
||
open, low-signal question answered with a previous Saturday's food order — is **not reproducible**. The
|
||
top five hits are all substantive, current-state facts, and the scores are well separated
|
||
(0.174 / 0.130 / 0.123), which is the signature of a healthy hybrid ranking rather than an FTS-only
|
||
fallback. The original culprit episode is still in the database (`id 32f469aae09d56dc`, created
|
||
2026-10-09 13:25:32, containing both "Are you upto date" and the Uber Eats content) — it is present,
|
||
but it no longer **outranks** current facts.
|
||
|
||
**Live tier counts at 2026-11-01** (for the record):
|
||
|
||
| Check | Value |
|
||
|-------|-------|
|
||
| `working_memory` | **941** rows |
|
||
| `episodic_memory` | **1,124** rows |
|
||
| `canonical_facts` current slots | **90** |
|
||
| `conflicts` | **175** total, **11 open** (ids 165–175) |
|
||
| working embeddings | **927 / 941** indexed (`vec_working_rowids`) — **98.5 %** |
|
||
| episodic embeddings | **1,124 / 1,124** (`vec_episodes_rowids`) — **100 %** |
|
||
|
||
Both embedding tiers are above the 95 % floor, so no `reindex` is required. **Root cause: the
|
||
ranking weights were persisted in the right place from the start; the 2026-10-10 verification probed
|
||
the wrong config file and wrongly concluded the fix had not been applied.** The incident is closed.
|
||
|
||
## Related
|
||
|
||
- [[systems/mnemosyne]] — live tier counts and coverage
|
||
- [[concepts/agent-memory]] — memory model
|
||
- [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]] — the earlier, different recall failure
|