7.0 KiB
title, type, status, created, updated, verified_on, review_after, confidence, tags, sources
| title | type | status | created | updated | verified_on | review_after | confidence | tags | sources | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mnemosyne Watchdog red for 210 consecutive 15-minute slots — 59 unresolved conflicts against a threshold of 50 (from 2026-11-18 20:01 AWST) | incident | active | 2026-11-20 | 2026-11-21 | 2026-11-21 | 2026-11-22 | high |
|
|
Mnemosyne Watchdog red for 210 consecutive slots (from 2026-11-18 20:01 AWST) — 115 at the 2026-11-20 read
Open. 96eb95ce12f2 (Mnemosyne Watchdog, 15-minute cadence) has failed every slot since
2026-11-18 20:01:06 AWST — 115 consecutive failures through 2026-11-20 00:31:16 AWST, the last
recorded before this page was written (it was 114 when the audit began; the streak gained a slot mid-audit).
cron/jobs.json reads last_status: error, failure_streak: 115 (that is the 2026-11-20 reading — this page was written then; the re-read below carries the streak to 210, last_seen_at 2026-11-21 00:31:16).
This is the estate's only red cron row with a large streak, and it is a single durable condition re-reported every 15 minutes, not a series of separate faults.
The failure is a fixed, self-describing threshold trip
Script exited with code 1
stdout:
🚨 Many unresolved conflicts: 59 (threshold: 50)
cron_incidents row 96eb95_98ade422d195 — state: alerted, failure_type: script,
first_seen_at 2026-11-18T20:01:15.906837+08:00, last_seen_at 2026-11-20T00:31:16.326357+08:00,
alerted_at 2026-11-19T20:46:23.428143+08:00, closed_at: None, output file
/home/hermes/.hermes/cron/output/96eb95ce12f2/2026-11-20_00-16-15.md.
Unlike the health monitor's rows (whose signatures embed a changing float and so fragment into a new row per
slot → incidents/2026-11-14-mnemosyne-capture-stall), this one fixes on the constant count 59 and
therefore stays a single row. The cost of that stability is the mirror image: it never closes, and it
re-alerts forever.
Alerting works — and that is the problem
deliveries.db, window 2026-11-18 → 2026-11-20 00:0x — the watchdog's own failure notices:
| delivered | execution |
|---|---|
| 2026-11-18 20:01:16 | f9865723b4f64d9fbfca7711ab053056 |
| 2026-11-19 02:16:16 | 7aa35681fd914a1e8db64b8dd0e09c84 |
| 2026-11-19 08:31:17 | 1ccc7e280d234287973edf5e0b35a5b7 |
| 2026-11-19 12:01:59 | 396940d9514b419eb024b83b3c10198d |
| 2026-11-19 14:31:35 | 0766953f92dc4fc4b0d0404dca51683f |
| 2026-11-19 20:46:19 | db010a8294bf4cd78457a265a5fb167c |
| 2026-11-20 00:03:37 | 64c6b5202463487c906f860bb746138b |
All status: delivered, all for_failure: 1. So detection, the failure notice and the push path are all
working. The output is an alert roughly every 3–6 h for a threshold breach that has not moved in two days —
which is exactly how a red signal stops meaning anything: while this row is red, a new Mnemosyne fault
arriving on the same job is indistinguishable from the standing one.
What the 59 conflicts actually are
Live read of ~/.hermes/mnemosyne/data/mnemosyne.db (2026-11-20 00:3x AWST): 223 conflicts total, 59
unresolved. Unresolved by creation day:
| created | unresolved |
|---|---|
| 2026-10-31 | 11 |
| 2026-11-02 | 2 |
| 2026-11-03 | 12 |
| 2026-11-06 | 1 |
| 2026-11-07 | 2 |
| 2026-11-08 | 1 |
| 2026-11-12 | 13 |
| 2026-11-13 | 3 |
| 2026-11-18 | 14 |
| total | 59 |
Two things stand out:
- Most of the backlog is old. 42 of the 59 predate 2026-11-14; the oldest batch is from 2026-10-31,
which is the same batch recorded on systems/mnemosyne as "all created in one batch at 2026-10-31
19:26:39 and sharing a single
fact_a_id, the extraction-noise signature". The documented remediation for this class is to bulk-resolve them asfalse_positive(see themnemosyne-operationsskill) — not applied here. - The threshold has been raised before. The 2026-10-18 and 2026-11-01 reads on systems/mnemosyne both record "1 unresolved conflict (benign, threshold 10)". It now reads 50. So the tripwire was already loosened 5×, and the backlog has since grown past even that. UNVERIFIED: when the threshold was changed from 10 to 50, and by whom; basis: two dated page reads vs the live alert text — the config line itself was not opened in this window.
The 2026-11-18 batch coincides with the migration session
The 14 conflicts created on 2026-11-18 are the newest batch, and 2026-11-18 is also the day of the only
interactive session in this window — 20261118_191808_5c9e08 (desktop, 9 messages, title "Helping migrate
context AI assistant"), whose turns were captured into working_memory at 19:25:03 AWST and summarised
episodically at 19:55:20 AWST. Consistent with the standing assessment that most conflicts in this bank are
LLM-extraction artefacts rather than real contradictions, but UNVERIFIED: that the 14 rows are extraction
noise from that session's turns specifically; basis: same-day coincidence plus the page's prior noise analysis —
the fact pairs were not inspected.
Re-read 2026-11-21 00:0x–00:5x AWST — 210 consecutive slots; no signature fragmentation
failure_streak= 210 (was 115 at the 11-20 read; the job's cadence is 96 slots/day, so this is 11-18 20:01 → 11-21 00:16 ≈ 2 days 4 h of continuous red).- Incident
96eb95_98ade422d195is one row, not a series:first_seen_at2026-11-18 20:01:15,last_seen_at2026-11-21 00:31:16 (00:16:15 at the first read),alerted_at2026-11-20 21:31:24,closed_atempty. The 11-20 page raised signature fragmentation as a risk; on this evidence the watchdog's de-duplication holds. - The contrast is the health monitor, which re-forms a new incident id every slot for one standing
condition (
ea79a2_f2be2db3bfce11-19 12:01:58 →ea79a2_aae4d914f3c911-20 00:03:36 →ea79a2_ace0f263e18a11-21 00:03:12). Same estate, same period, opposite behaviour — worth knowing when reading the ledger for "new" versus "chronic". - Error text unchanged: 59 unresolved conflicts against a threshold of 50. Nothing was resolved, reclassified or deleted at this read either.
- Still the standing question: a level-based test re-reports on every slot. A state-change alert would have fired once on 11-18 20:01 and then stayed quiet.
Not applied / open
- Nothing was resolved, reclassified or deleted. No conflict row was touched; no threshold was changed.
- The watchdog's cadence, signature construction and alert path were left as-is.
- Open question: should a standing threshold breach alert on state change rather than on every slot? Same family as the zero-item-consolidation streak problem on incidents/2026-11-14-mnemosyne-capture-stall — both are conditions that re-report continuously because the gate tests a level, not a change.