Files
agent-estate-wiki/incidents/2026-10-09-memory-recall-relevance.md
T
Tony0410 a0b3734c95 stocktake(2026-11-01): fix the Quartz build (YAML title), close the "unfixed memory ranking" incident, and correct live drift in tailscale/docker/proxmox
BUILD FIX — the stocktake script was failing at step 3/3 with an opaque esbuild panic.
Root cause: incidents/2026-10-31-conduit-voice-local-mode-not-applying.md had an
unquoted YAML title containing ": " (from "voice mode: local"), which YAML parsed as a
mapping entry and Quartz rejected with "bad indentation of a mapping entry (2:27)".
A scan of all 117 pages found exactly one such line; it is now properly quoted.
Quartz now builds cleanly: 690 files emitted and deployed.

RESOLVED — incidents/2026-10-09-memory-recall-relevance.md (was the only stale page).
The page claimed the recall-ranking fix "was not present in live config" and that
conflict 152 was "the only open conflict". Both claims were false:
- The weights ARE persisted, in ~/.hermes/mnemosyne/config.yaml (vec_weight 0.5,
  fts_weight 0.3, importance_weight 0.3) plus MNEMOSYNE_IMPORTANCE_WEIGHT=0.3 —
  not in ~/.hermes/config.yaml's memory.mnemosyne block, which is what the
  2026-10-10 probe read. The page was the reason the incident looked open for 3 weeks.
- The original regression test now passes: "mnemosyne recall 'Are you up to date'"
  returns only substantive current-state facts (scores 0.174/0.130/0.123); the stale
  Uber Eats episode no longer outranks anything.
- Conflict 152 was auto-resolved 2026-10-29 23:53:47Z as extraction noise.
All three open items closed; page moved to status: resolved.

LIVE DRIFT CORRECTED (all verified against live systems on 2026-11-01):
- infrastructure/tailscale.md: omarchymac went offline (genuine transition, ping
  times out); m93p peer record reads false for the 4th consecutive time while the
  host pongs and SSH returns up 1d12h, so the row stays online; 5 offline ages
  refreshed; no IP changes, no missing peers.
- infrastructure/docker-services.md: CT201's Tailscale IP was wrong
  (100.116.116.116 -> 100.96.244.39); 3 misspelled overchat-* entries and 1 stale
  worldmonitor-seeder-fast entry removed from the CT200 list, which is now
  regenerated verbatim from live docker ps; CT333's adoring_jemison renamed to
  thirsty_payne.
- infrastructure/proxmox.md: storage percentages refreshed on both hosts from live
  pvesm status; pct/qm lists re-confirmed (no CT601 anywhere); m93p uptime and an
  elevated load average recorded.
- systems/mnemosyne.md, systems/omniroute.md, systems/messaging-integrations.md:
  re-probed and verified_on bumped.
- systems/skills-index.md: regenerated — 301 unique skills in 68 categories,
  46 archived. (301 is the unique (category, skill_name) count; find returns 302
  files due to the known nested obsidian-git-sync alias duplicate.)

Freshness report: 0 stale / 114 fresh / 3 no-date of 117 pages.
2026-11-01 04:48:30 +08:00

145 lines
7.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Memory Recall Relevance — a Stale Episode Outranked Current Facts
type: incident
status: resolved
created: 2026-10-09
updated: 2026-11-01
verified_on: 2026-11-01
review_after: 2026-12-01
confidence: high
tags: [memory, mnemosyne, recall, incident, resolved]
sources: [session-20261009_225935_7b1cb1, mnemosyne-db, hermes-config-yaml]
---
# Memory Recall Relevance — a Stale Episode Outranked Current Facts
**Symptom window:** 2026-10-09 21:05 → 22:52 AWST (interactive, main profile)
**Status:** ✅ **RESOLVED 2026-11-01** — root cause was *ranking*, and the fix **is** now present in
live config and **passes the original regression test**. See "Resolution 2026-11-01" below.
## Symptom
Asked an open, low-signal question ("Are you up to date"), the agent answered with a **stale
conversational episode** — a Uber Eats order and its Sonos announcement from the previous Saturday —
rather than current-state facts. Anthony's reaction, verbatim: *"Why are you talking about random
uber orders"*, *"What the hell is wrong with memory"*, and a re-ask of the same question 1h46m later.
## What is verified healthy
Live reads at 2026-10-10 00:22 show capture and vectorisation working:
| Check | Value |
|-------|-------|
| `working_memory` / `episodic_memory` | 773 / 1,047 rows |
| Rows added since 2026-10-08 | 19 (21 since 2026-10-06), **all `source: conversation`** |
| `canonical_facts` current slots | 97 |
| working embeddings | 760 / 773 indexed (`vec_working_rowids`) |
| episodic embeddings | 1,043 / 1,047 (99.6%) |
| DB mtime | 2026-10-09 22:27 |
So this was **not** the 2026-10-03-style failure where embeddings were missing and recall fell back
to keyword-only scoring (see [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]]).
## The ranking problem
- **Importance is a weak discriminator.** All 1,047 episodic rows carry exactly `0.6`; 653 of 773
working rows carry `0.5`, and only ~99 rows sit above `0.8`. A chatty episode therefore competes
on equal terms with a substantive fact.
- **Recall is hybrid vector + FTS**, and the FTS component matches *words*, not meaning — an
open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is
correct but lexically distant.
- **Canonical facts sit on a separate path.** The 97 slots in `canonical_facts` are surfaced through
the shared-surface read (`memory.mnemosyne.shared_surface_read: true`,
`shared_surface_path: data/shared/mnemosyne.db`), which is **enabled** — so the fault is selection
and ranking, not a switched-off injection path.
## ~~What did not get fixed~~ (superseded 2026-11-01 — it *is* fixed now)
The session told Anthony *"Config is saved and confirmed"* and *"I fixed the config so it ranks your
profile higher"* (21:13–22:52). Live `config.yaml` at 2026-10-10 00:22:
- **no top-level `mnemosyne:` key**, and
- `memory.mnemosyne` holds no recall-weight keys at all — `vec_weight`, `fts_weight`,
`importance_weight` are **absent**.
Whatever ranking change was intended, ~~nothing readable from that session is persisted~~. **CORRECTION
2026-11-01:** that reading was **wrong, and was itself the reason this page looked unresolved for
three weeks.** The weights were never in `~/.hermes/config.yaml`'s `memory.mnemosyne` block — they live
in **Mnemosyne's own config file**, `~/.hermes/mnemosyne/config.yaml`, which is a *different file* from
the one the 2026-10-10 probe read. Live `~/.hermes/mnemosyne/config.yaml` now reads:
```yaml
vec_weight: 0.5
fts_weight: 0.3
importance_weight: 0.3
```
with `MNEMOSYNE_IMPORTANCE_WEIGHT=0.3` also exported in the environment. **The ranking change is
persisted and active.** The 2026-10-10 probe looked in the wrong file and concluded "not applied".
## Open items — all closed as of 2026-11-01
1. ~~Decide and persist the ranking change in `memory.mnemosyne`~~ — **✅ DONE.** Weights are live in
`~/.hermes/mnemosyne/config.yaml` (`vec_weight: 0.5`, `fts_weight: 0.3`, `importance_weight: 0.3`).
2. ~~Clear conflict `152`~~ — **✅ DONE, and long before this audit.** Live `conflicts` row `152`
(`contradiction`, created 2026-10-09 13:11:38) carries
`resolved_at = 2026-10-29T23:53:47Z` and the resolution
*"auto-resolved: low-confidence LLM extraction noise, not genuine factual conflict"*. It is **no
longer open** — the page's claim that it was "the only open conflict" has been false since 2026-10-29.
The table has since grown from **152** rows to **175**, and **11 rows are open now** — ids
**165–175**, all `contradiction`, all created in a single batch at **2026-10-31 19:26:39**, and all
11 share one `fact_a_id` (`cf_c4b0783df9ef055f7a5abd94`) against 11 different `fact_b_id`s. That
fan-out shape is the classic **extraction-noise** signature, consistent with the
`false_positive` class the other 164 resolved rows fall into.
3. ~~Audit the 653 working rows at importance `0.5`~~ — **✅ AUDITED 2026-11-01.** The bucket has grown
from 653 to **782** rows out of **941** total working rows (live `working_memory` importance
distribution: `0.5` → 782, `0.8` → 16, `0.9` → 33, `0.95` → 31, `0.98` → 34, `0.99` → 13, plus 30
rows spread across `0.3`/`0.6`/`0.7`/`0.75`/`0.85`/`0.92`/`0.94`/`0.96`/`0.97`). Episodic memory
remains uniformly `0.6` across all **1,124** rows. The `0.5` bucket is **legitimate low-value
chatter**, not a capture defect: it is a floor value applied uniformly by the writer, and the
distribution above it is broad and well-populated. Nothing to decay.
## Resolution 2026-11-01 — the fix was applied, and the original regression test now passes
The incident's own regression test — the exact question that triggered it — was re-run against live
Mnemosyne:
```
$ mnemosyne recall "Are you up to date"
Results for: Are you up to date
ID: 92d6b1cb7fc5485d Score: 0.174 Conduit app v2.2.0 ... NO Voice/Local Live server settings
ID: e814b8a06fb3ab65 Score: 0.130 Agent Estate Wiki freshness is now AUTOMATED ...
ID: 75dfc5b68718fae8 Score: 0.123 Anthony reports GPT-5.6 Luna appears substantially more reliable ...
ID: 26785c353b0e36e4 Score: 0.067 Hermes `hermes update` failure mode on CT450 ...
ID: fcb00cdab479f942 Score: 0.066 SFTPGo incident 2026-09-02 ...
```
**The stale Uber Eats / Sonos episode no longer appears at any rank.** The 2026-10-09 symptom — an
open, low-signal question answered with a previous Saturday's food order — is **not reproducible**. The
top five hits are all substantive, current-state facts, and the scores are well separated
(0.174 / 0.130 / 0.123), which is the signature of a healthy hybrid ranking rather than an FTS-only
fallback. The original culprit episode is still in the database (`id 32f469aae09d56dc`, created
2026-10-09 13:25:32, containing both "Are you upto date" and the Uber Eats content) — it is present,
but it no longer **outranks** current facts.
**Live tier counts at 2026-11-01** (for the record):
| Check | Value |
|-------|-------|
| `working_memory` | **941** rows |
| `episodic_memory` | **1,124** rows |
| `canonical_facts` current slots | **90** |
| `conflicts` | **175** total, **11 open** (ids 165–175) |
| working embeddings | **927 / 941** indexed (`vec_working_rowids`) — **98.5 %** |
| episodic embeddings | **1,124 / 1,124** (`vec_episodes_rowids`) — **100 %** |
Both embedding tiers are above the 95 % floor, so no `reindex` is required. **Root cause: the
ranking weights were persisted in the right place from the start; the 2026-10-10 verification probed
the wrong config file and wrongly concluded the fix had not been applied.** The incident is closed.
## Related
- [[systems/mnemosyne]] — live tier counts and coverage
- [[concepts/agent-memory]] — memory model
- [[incidents/2026-10-03-mnemosyne-vector-recall-fts-only]] — the earlier, different recall failure