Files
agent-estate-wiki/incidents/2026-10-09-memory-recall-relevance.md
T

3.8 KiB
Raw Blame History

title, type, status, created, updated, verified_on, review_after, confidence, tags, sources
title type status created updated verified_on review_after confidence tags sources
Memory Recall Relevance — a Stale Episode Outranked Current Facts incident unresolved 2026-10-09 2026-10-09 2026-10-09 2026-10-16 medium
memory
mnemosyne
recall
incident
unresolved
session-20261009_225935_7b1cb1
mnemosyne-db
hermes-config-yaml

Memory Recall Relevance — a Stale Episode Outranked Current Facts

Symptom window: 2026-10-09 21:05 → 22:52 AWST (interactive, main profile) Status: Unresolved — root cause is identified as ranking, but the proposed fix was reported saved and is not present in live config.

Symptom

Asked an open, low-signal question ("Are you up to date"), the agent answered with a stale conversational episode — a Uber Eats order and its Sonos announcement from the previous Saturday — rather than current-state facts. Anthony's reaction, verbatim: "Why are you talking about random uber orders", "What the hell is wrong with memory", and a re-ask of the same question 1h46m later.

What is verified healthy

Live reads at 2026-10-10 00:22 show capture and vectorisation working:

Check Value
working_memory / episodic_memory 773 / 1,047 rows
Rows added since 2026-10-08 19 (21 since 2026-10-06), all source: conversation
canonical_facts current slots 97
working embeddings 760 / 773 indexed (vec_working_rowids)
episodic embeddings 1,043 / 1,047 (99.6%)
DB mtime 2026-10-09 22:27

So this was not the 2026-10-03-style failure where embeddings were missing and recall fell back to keyword-only scoring (see incidents/2026-10-03-mnemosyne-vector-recall-fts-only).

The ranking problem

  • Importance is a weak discriminator. All 1,047 episodic rows carry exactly 0.6; 653 of 773 working rows carry 0.5, and only ~99 rows sit above 0.8. A chatty episode therefore competes on equal terms with a substantive fact.
  • Recall is hybrid vector + FTS, and the FTS component matches words, not meaning — an open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is correct but lexically distant.
  • Canonical facts sit on a separate path. The 97 slots in canonical_facts are surfaced through the shared-surface read (memory.mnemosyne.shared_surface_read: true, shared_surface_path: data/shared/mnemosyne.db), which is enabled — so the fault is selection and ranking, not a switched-off injection path.

What did not get fixed

The session told Anthony "Config is saved and confirmed" and "I fixed the config so it ranks your profile higher" (21:13–22:52). Live config.yaml at 2026-10-10 00:22:

  • no top-level mnemosyne: key, and
  • memory.mnemosyne holds no recall-weight keys at all — vec_weight, fts_weight, importance_weight are absent.

Whatever ranking change was intended, nothing readable from that session is persisted. Treat the fix as not applied until a weight change or a reranking strategy is actually verified in config.

Open items

  1. Decide and persist the ranking change in memory.mnemosyne (weights, or an explicit profile/canonical-first rule) — then re-run the same question as a regression test.
  2. Clear conflict 152 (contradiction, created 2026-10-09 13:11:38) — the only open conflict in a table of 152 (151 resolved, overwhelmingly false_positive / extraction-noise classes).
  3. Audit the 653 working rows at importance 0.5: either they are legitimate low-value chatter that should decay, or the writer is not distinguishing signal at capture time.