wiki audit 2026-10-09: fleet-wide cron timeout storm (15 failures/10 jobs) + memory recall-relevance incident; backfill 2026-10-08 window; commit stranded stocktake artifacts
This commit is contained in:
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: Memory Recall Relevance — a Stale Episode Outranked Current Facts
|
||||
type: incident
|
||||
status: unresolved
|
||||
created: 2026-10-09
|
||||
updated: 2026-10-09
|
||||
verified_on: 2026-10-09
|
||||
review_after: 2026-10-16
|
||||
confidence: medium
|
||||
tags: [memory, mnemosyne, recall, incident, unresolved]
|
||||
sources: [session-20261009_225935_7b1cb1, mnemosyne-db, hermes-config-yaml]
|
||||
---
|
||||
|
||||
# Memory Recall Relevance — a Stale Episode Outranked Current Facts
|
||||
|
||||
**Symptom window:** 2026-10-09 21:05 → 22:52 AWST (interactive, main profile)
|
||||
**Status:** Unresolved — root cause is identified as *ranking*, but the proposed fix was reported
|
||||
saved and is **not** present in live config.
|
||||
|
||||
## Symptom
|
||||
|
||||
Asked an open, low-signal question ("Are you up to date"), the agent answered with a **stale
|
||||
conversational episode** — a Uber Eats order and its Sonos announcement from the previous Saturday —
|
||||
rather than current-state facts. Anthony's reaction, verbatim: *"Why are you talking about random
|
||||
uber orders"*, *"What the hell is wrong with memory"*, and a re-ask of the same question 1h46m later.
|
||||
|
||||
## What is verified healthy
|
||||
|
||||
Live reads at 2026-10-10 00:22 show capture and vectorisation working:
|
||||
|
||||
| Check | Value |
|
||||
|-------|-------|
|
||||
| `working_memory` / `episodic_memory` | 773 / 1,047 rows |
|
||||
| Rows added since 2026-10-08 | 19 (21 since 2026-10-06), **all `source: conversation`** |
|
||||
| `canonical_facts` current slots | 97 |
|
||||
| working embeddings | 760 / 773 indexed (`vec_working_rowids`) |
|
||||
| episodic embeddings | 1,043 / 1,047 (99.6%) |
|
||||
| DB mtime | 2026-10-09 22:27 |
|
||||
|
||||
So this was **not** the 2026-10-03-style failure where embeddings were missing and recall fell back
|
||||
to keyword-only scoring (see [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]]).
|
||||
|
||||
## The ranking problem
|
||||
|
||||
- **Importance is a weak discriminator.** All 1,047 episodic rows carry exactly `0.6`; 653 of 773
|
||||
working rows carry `0.5`, and only ~99 rows sit above `0.8`. A chatty episode therefore competes
|
||||
on equal terms with a substantive fact.
|
||||
- **Recall is hybrid vector + FTS**, and the FTS component matches *words*, not meaning — an
|
||||
open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is
|
||||
correct but lexically distant.
|
||||
- **Canonical facts sit on a separate path.** The 97 slots in `canonical_facts` are surfaced through
|
||||
the shared-surface read (`memory.mnemosyne.shared_surface_read: true`,
|
||||
`shared_surface_path: data/shared/mnemosyne.db`), which is **enabled** — so the fault is selection
|
||||
and ranking, not a switched-off injection path.
|
||||
|
||||
## What did not get fixed
|
||||
|
||||
The session told Anthony *"Config is saved and confirmed"* and *"I fixed the config so it ranks your
|
||||
profile higher"* (21:13–22:52). Live `config.yaml` at 2026-10-10 00:22:
|
||||
|
||||
- **no top-level `mnemosyne:` key**, and
|
||||
- `memory.mnemosyne` holds no recall-weight keys at all — `vec_weight`, `fts_weight`,
|
||||
`importance_weight` are **absent**.
|
||||
|
||||
Whatever ranking change was intended, **nothing readable from that session is persisted**. Treat the
|
||||
fix as not applied until a weight change or a reranking strategy is actually verified in config.
|
||||
|
||||
## Open items
|
||||
|
||||
1. Decide and persist the ranking change in `memory.mnemosyne` (weights, or an explicit
|
||||
profile/canonical-first rule) — then re-run the same question as a regression test.
|
||||
2. Clear conflict `152` (`contradiction`, created 2026-10-09 13:11:38) — the only open conflict in a
|
||||
table of 152 (151 resolved, overwhelmingly `false_positive` / extraction-noise classes).
|
||||
3. Audit the 653 working rows at importance `0.5`: either they are legitimate low-value chatter that
|
||||
should decay, or the writer is not distinguishing signal at capture time.
|
||||
|
||||
## Related
|
||||
|
||||
- [[systems/mnemosyne]] — live tier counts and coverage
|
||||
- [[concepts/agent-memory]] — memory model
|
||||
- [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]] — the earlier, different recall failure
|
||||
Reference in New Issue
Block a user