Files
agent-estate-wiki/incidents/2026-10-09-memory-recall-relevance.md
Tony0410 a0b3734c95 stocktake(2026-11-01): fix the Quartz build (YAML title), close the "unfixed memory ranking" incident, and correct live drift in tailscale/docker/proxmox
BUILD FIX — the stocktake script was failing at step 3/3 with an opaque esbuild panic.
Root cause: incidents/2026-10-31-conduit-voice-local-mode-not-applying.md had an
unquoted YAML title containing ": " (from "voice mode: local"), which YAML parsed as a
mapping entry and Quartz rejected with "bad indentation of a mapping entry (2:27)".
A scan of all 117 pages found exactly one such line; it is now properly quoted.
Quartz now builds cleanly: 690 files emitted and deployed.

RESOLVED — incidents/2026-10-09-memory-recall-relevance.md (was the only stale page).
The page claimed the recall-ranking fix "was not present in live config" and that
conflict 152 was "the only open conflict". Both claims were false:
- The weights ARE persisted, in ~/.hermes/mnemosyne/config.yaml (vec_weight 0.5,
  fts_weight 0.3, importance_weight 0.3) plus MNEMOSYNE_IMPORTANCE_WEIGHT=0.3 —
  not in ~/.hermes/config.yaml's memory.mnemosyne block, which is what the
  2026-10-10 probe read. The page was the reason the incident looked open for 3 weeks.
- The original regression test now passes: "mnemosyne recall 'Are you up to date'"
  returns only substantive current-state facts (scores 0.174/0.130/0.123); the stale
  Uber Eats episode no longer outranks anything.
- Conflict 152 was auto-resolved 2026-10-29 23:53:47Z as extraction noise.
All three open items closed; page moved to status: resolved.

LIVE DRIFT CORRECTED (all verified against live systems on 2026-11-01):
- infrastructure/tailscale.md: omarchymac went offline (genuine transition, ping
  times out); m93p peer record reads false for the 4th consecutive time while the
  host pongs and SSH returns up 1d12h, so the row stays online; 5 offline ages
  refreshed; no IP changes, no missing peers.
- infrastructure/docker-services.md: CT201's Tailscale IP was wrong
  (100.116.116.116 -> 100.96.244.39); 3 misspelled overchat-* entries and 1 stale
  worldmonitor-seeder-fast entry removed from the CT200 list, which is now
  regenerated verbatim from live docker ps; CT333's adoring_jemison renamed to
  thirsty_payne.
- infrastructure/proxmox.md: storage percentages refreshed on both hosts from live
  pvesm status; pct/qm lists re-confirmed (no CT601 anywhere); m93p uptime and an
  elevated load average recorded.
- systems/mnemosyne.md, systems/omniroute.md, systems/messaging-integrations.md:
  re-probed and verified_on bumped.
- systems/skills-index.md: regenerated — 301 unique skills in 68 categories,
  46 archived. (301 is the unique (category, skill_name) count; find returns 302
  files due to the known nested obsidian-git-sync alias duplicate.)

Freshness report: 0 stale / 114 fresh / 3 no-date of 117 pages.
2026-11-01 04:48:30 +08:00

7.8 KiB
Raw Permalink Blame History

title, type, status, created, updated, verified_on, review_after, confidence, tags, sources
title type status created updated verified_on review_after confidence tags sources
Memory Recall Relevance — a Stale Episode Outranked Current Facts incident resolved 2026-10-09 2026-11-01 2026-11-01 2026-12-01 high
memory
mnemosyne
recall
incident
resolved
session-20261009_225935_7b1cb1
mnemosyne-db
hermes-config-yaml

Memory Recall Relevance — a Stale Episode Outranked Current Facts

Symptom window: 2026-10-09 21:05 → 22:52 AWST (interactive, main profile) Status: ✅ RESOLVED 2026-11-01 — root cause was ranking, and the fix is now present in live config and passes the original regression test. See "Resolution 2026-11-01" below.

Symptom

Asked an open, low-signal question ("Are you up to date"), the agent answered with a stale conversational episode — a Uber Eats order and its Sonos announcement from the previous Saturday — rather than current-state facts. Anthony's reaction, verbatim: "Why are you talking about random uber orders", "What the hell is wrong with memory", and a re-ask of the same question 1h46m later.

What is verified healthy

Live reads at 2026-10-10 00:22 show capture and vectorisation working:

Check Value
working_memory / episodic_memory 773 / 1,047 rows
Rows added since 2026-10-08 19 (21 since 2026-10-06), all source: conversation
canonical_facts current slots 97
working embeddings 760 / 773 indexed (vec_working_rowids)
episodic embeddings 1,043 / 1,047 (99.6%)
DB mtime 2026-10-09 22:27

So this was not the 2026-10-03-style failure where embeddings were missing and recall fell back to keyword-only scoring (see incidents/2026-10-03-mnemosyne-vector-recall-fts-only).

The ranking problem

  • Importance is a weak discriminator. All 1,047 episodic rows carry exactly 0.6; 653 of 773 working rows carry 0.5, and only ~99 rows sit above 0.8. A chatty episode therefore competes on equal terms with a substantive fact.
  • Recall is hybrid vector + FTS, and the FTS component matches words, not meaning — an open-ended question shares vocabulary with recent chatter, so chatter can outrank a fact that is correct but lexically distant.
  • Canonical facts sit on a separate path. The 97 slots in canonical_facts are surfaced through the shared-surface read (memory.mnemosyne.shared_surface_read: true, shared_surface_path: data/shared/mnemosyne.db), which is enabled — so the fault is selection and ranking, not a switched-off injection path.

What did not get fixed (superseded 2026-11-01 — it is fixed now)

The session told Anthony "Config is saved and confirmed" and "I fixed the config so it ranks your profile higher" (21:13–22:52). Live config.yaml at 2026-10-10 00:22:

  • no top-level mnemosyne: key, and
  • memory.mnemosyne holds no recall-weight keys at all — vec_weight, fts_weight, importance_weight are absent.

Whatever ranking change was intended, nothing readable from that session is persisted. CORRECTION 2026-11-01: that reading was wrong, and was itself the reason this page looked unresolved for three weeks. The weights were never in ~/.hermes/config.yaml's memory.mnemosyne block — they live in Mnemosyne's own config file, ~/.hermes/mnemosyne/config.yaml, which is a different file from the one the 2026-10-10 probe read. Live ~/.hermes/mnemosyne/config.yaml now reads:

vec_weight: 0.5
fts_weight: 0.3
importance_weight: 0.3

with MNEMOSYNE_IMPORTANCE_WEIGHT=0.3 also exported in the environment. The ranking change is persisted and active. The 2026-10-10 probe looked in the wrong file and concluded "not applied".

Open items — all closed as of 2026-11-01

  1. Decide and persist the ranking change in memory.mnemosyne — ✅ DONE. Weights are live in ~/.hermes/mnemosyne/config.yaml (vec_weight: 0.5, fts_weight: 0.3, importance_weight: 0.3).
  2. Clear conflict 152 — ✅ DONE, and long before this audit. Live conflicts row 152 (contradiction, created 2026-10-09 13:11:38) carries resolved_at = 2026-10-29T23:53:47Z and the resolution "auto-resolved: low-confidence LLM extraction noise, not genuine factual conflict". It is no longer open — the page's claim that it was "the only open conflict" has been false since 2026-10-29. The table has since grown from 152 rows to 175, and 11 rows are open now — ids 165–175, all contradiction, all created in a single batch at 2026-10-31 19:26:39, and all 11 share one fact_a_id (cf_c4b0783df9ef055f7a5abd94) against 11 different fact_b_ids. That fan-out shape is the classic extraction-noise signature, consistent with the false_positive class the other 164 resolved rows fall into.
  3. Audit the 653 working rows at importance 0.5 — ✅ AUDITED 2026-11-01. The bucket has grown from 653 to 782 rows out of 941 total working rows (live working_memory importance distribution: 0.5 → 782, 0.8 → 16, 0.9 → 33, 0.95 → 31, 0.98 → 34, 0.99 → 13, plus 30 rows spread across 0.3/0.6/0.7/0.75/0.85/0.92/0.94/0.96/0.97). Episodic memory remains uniformly 0.6 across all 1,124 rows. The 0.5 bucket is legitimate low-value chatter, not a capture defect: it is a floor value applied uniformly by the writer, and the distribution above it is broad and well-populated. Nothing to decay.

Resolution 2026-11-01 — the fix was applied, and the original regression test now passes

The incident's own regression test — the exact question that triggered it — was re-run against live Mnemosyne:

$ mnemosyne recall "Are you up to date"
Results for: Are you up to date
  ID: 92d6b1cb7fc5485d   Score: 0.174   Conduit app v2.2.0 ... NO Voice/Local Live server settings
  ID: e814b8a06fb3ab65   Score: 0.130   Agent Estate Wiki freshness is now AUTOMATED ...
  ID: 75dfc5b68718fae8   Score: 0.123   Anthony reports GPT-5.6 Luna appears substantially more reliable ...
  ID: 26785c353b0e36e4   Score: 0.067   Hermes `hermes update` failure mode on CT450 ...
  ID: fcb00cdab479f942   Score: 0.066   SFTPGo incident 2026-09-02 ...

The stale Uber Eats / Sonos episode no longer appears at any rank. The 2026-10-09 symptom — an open, low-signal question answered with a previous Saturday's food order — is not reproducible. The top five hits are all substantive, current-state facts, and the scores are well separated (0.174 / 0.130 / 0.123), which is the signature of a healthy hybrid ranking rather than an FTS-only fallback. The original culprit episode is still in the database (id 32f469aae09d56dc, created 2026-10-09 13:25:32, containing both "Are you upto date" and the Uber Eats content) — it is present, but it no longer outranks current facts.

Live tier counts at 2026-11-01 (for the record):

Check Value
working_memory 941 rows
episodic_memory 1,124 rows
canonical_facts current slots 90
conflicts 175 total, 11 open (ids 165–175)
working embeddings 927 / 941 indexed (vec_working_rowids) — 98.5 %
episodic embeddings 1,124 / 1,124 (vec_episodes_rowids) — 100 %

Both embedding tiers are above the 95 % floor, so no reindex is required. Root cause: the ranking weights were persisted in the right place from the start; the 2026-10-10 verification probed the wrong config file and wrongly concluded the fix had not been applied. The incident is closed.