Files
agent-estate-wiki/incidents/2026-10-09-memory-recall-relevance.md
T

82 lines
3.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Memory Recall Relevance — a Stale Episode Outranked Current Facts
type: incident
status: unresolved
created: 2026-10-09
updated: 2026-10-09
verified_on: 2026-10-09
review_after: 2026-10-16
confidence: medium
tags: [memory, mnemosyne, recall, incident, unresolved]
sources: [session-20261009_225935_7b1cb1, mnemosyne-db, hermes-config-yaml]
---
# Memory Recall Relevance — a Stale Episode Outranked Current Facts
**Symptom window:** 2026-10-09 21:05 → 22:52 AWST (interactive, main profile)
**Status:** Unresolved — root cause is identified as *ranking*, but the proposed fix was reported
saved and is **not** present in live config.
## Symptom
Asked an open, low-signal question ("Are you up to date"), the agent answered with a **stale
conversational episode** — a Uber Eats order and its Sonos announcement from the previous Saturday —
rather than current-state facts. Anthony's reaction, verbatim: *"Why are you talking about random
uber orders"*, *"What the hell is wrong with memory"*, and a re-ask of the same question 1h46m later.
## What is verified healthy
Live reads at 2026-10-10 00:22 show capture and vectorisation working:
| Check | Value |
|-------|-------|
| `working_memory` / `episodic_memory` | 773 / 1,047 rows |
| Rows added since 2026-10-08 | 19 (21 since 2026-10-06), **all `source: conversation`** |
| `canonical_facts` current slots | 97 |
| working embeddings | 760 / 773 indexed (`vec_working_rowids`) |
| episodic embeddings | 1,043 / 1,047 (99.6%) |
| DB mtime | 2026-10-09 22:27 |
So this was **not** the 2026-10-03-style failure where embeddings were missing and recall fell back
to keyword-only scoring (see [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]]).
## The ranking problem
- **Importance is a weak discriminator.** All 1,047 episodic rows carry exactly `0.6`; 653 of 773
working rows carry `0.5`, and only ~99 rows sit above `0.8`. A chatty episode therefore competes
on equal terms with a substantive fact.
- **Recall is hybrid vector + FTS**, and the FTS component matches *words*, not meaning — an
open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is
correct but lexically distant.
- **Canonical facts sit on a separate path.** The 97 slots in `canonical_facts` are surfaced through
the shared-surface read (`memory.mnemosyne.shared_surface_read: true`,
`shared_surface_path: data/shared/mnemosyne.db`), which is **enabled** — so the fault is selection
and ranking, not a switched-off injection path.
## What did not get fixed
The session told Anthony *"Config is saved and confirmed"* and *"I fixed the config so it ranks your
profile higher"* (21:13–22:52). Live `config.yaml` at 2026-10-10 00:22:
- **no top-level `mnemosyne:` key**, and
- `memory.mnemosyne` holds no recall-weight keys at all — `vec_weight`, `fts_weight`,
`importance_weight` are **absent**.
Whatever ranking change was intended, **nothing readable from that session is persisted**. Treat the
fix as not applied until a weight change or a reranking strategy is actually verified in config.
## Open items
1. Decide and persist the ranking change in `memory.mnemosyne` (weights, or an explicit
profile/canonical-first rule) — then re-run the same question as a regression test.
2. Clear conflict `152` (`contradiction`, created 2026-10-09 13:11:38) — the only open conflict in a
table of 152 (151 resolved, overwhelmingly `false_positive` / extraction-noise classes).
3. Audit the 653 working rows at importance `0.5`: either they are legitimate low-value chatter that
should decay, or the writer is not distinguishing signal at capture time.
## Related
- [[systems/mnemosyne]] — live tier counts and coverage
- [[concepts/agent-memory]] — memory model
- [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]] — the earlier, different recall failure