Stocktake 2026-11-12: tailscale live-vs-page drift corrected; sftpgo incident re-verified; pre-dispatch-filter signature found recurring on a second job/pin
- infrastructure/tailscale.md: node table rebuilt from live tailscale status --json. mailflow/qwenpaw/springroll read Online:false in the peer record but all three ping (CT1262 + CT465 running; springroll via DERP(sin)) - stale peer records, rows stay online. omarchymac genuinely stale in the other direction (offline 2d/11-08 -> offline 4d, ping timeout). 9 named offline ages + 6 codex ephemeral peers refreshed. New guest CT1007 homebridge (running, tailscaled inactive). DEAD Serve targets 8769/9123 re-confirmed dead; CT201 table unchanged; container counts 17 LXC+4 VM (pve) / 20 LXC+2 VM (m93p) confirmed. - incidents/2026-09-02-sftpgo-outage.md: 22d stale, re-probed live (pct list running, peer Online:true, pong, healthz 401; swap now 512 not 43Gi). Cleared. - incidents/2026-10-21-omniroute-pre-dispatch-filter-skip.md: signature recurred 2026-11-10 on cbfe434a3b5e (custom:omni-101/free-stack, HTTP 429 credits 100%); incident row resolved 2026-11-12T00:25. Original subject clean 4 slots. Retention-floor caveat recorded. - Threshold re-verifications (19d/18d): mnemosyne-relay (tools:None gone, 0 log hits), work-outlook (himalaya still login not supported), omniroute-free-deepseek (good-free/free-stack both HTTP 200, non-deepseek resolution). - Correction recorded: previous entry mis-described the 2026-11-12 stale page as log.md. - wiki-maintain.sh re-run after all edits: freshness 0 stale / 130 fresh / 3 no-date of 133.
This commit is contained in:
@@ -3,9 +3,8 @@ title: 'Mnemosyne Relay Scope Corruption + `tools: None` — 2026-07-29'
|
||||
type: incident
|
||||
status: resolved
|
||||
created: 2026-07-29
|
||||
updated: 2026-10-24
|
||||
verified_on: 2026-10-24
|
||||
review_after: 2026-11-07
|
||||
updated: 2026-11-12
|
||||
verified_on: 2026-11-12
|
||||
confidence: high
|
||||
tags: [mnemosyne, config, relay, incident, resolved]
|
||||
sources: [live-probe, config, session-20260729]
|
||||
@@ -51,4 +50,21 @@ Scope push/pop ordering mismatch. Something pushes a scope that isn't at the top
|
||||
Filed (or should be filed) under NousResearch/hermes-agent issues. No token available to auto-create.
|
||||
|
||||
## MONITORING
|
||||
Watch for recurrence. If persists, check whether the affected sessions share a common tool call pattern (subagent delegation, tool failure, etc.).
|
||||
Watch for recurrence. If persists, check whether the affected sessions share a common tool call pattern (subagent delegation, tool failure, etc.).
|
||||
|
||||
## Re-verified 2026-11-12 03:3x AWST (stocktake — page was 19d old, inside the staleness window)
|
||||
Both halves of this page re-checked against live state and **the fix holds; neither symptom has recurred**
|
||||
in ~3.5 months and 106 days of logs:
|
||||
- `grep -n "tools:" ~/.hermes/config.yaml` → the `memory.mnemosyne` block no longer carries `tools: None`
|
||||
(the literal `None` key is absent), so the `Unknown Mnemosyne tool(s)… None` schema warning has no
|
||||
trigger left.
|
||||
- `grep -c "Unknown Mnemosyne tool" ~/.hermes/logs/gateway.log` and `.log.1`–`.log.3` → **0 in every
|
||||
rotated file** (4 generations, 0 occurrences).
|
||||
- `grep -ah "scope handle is not at the top" ~/.hermes/logs/gateway.log*` → **no occurrences**; the
|
||||
relay-scope corruption has not repeated since 2026-07-29 08:10 UTC.
|
||||
- Mnemosyne itself is healthy on the current build: `mnemosyne stats` → **2782 memories (1062 working /
|
||||
1217 episodic)** and `mnemosyne recall "tailscale serve routes"` returns substantive ranked results, so
|
||||
the relay/memory path is functioning.
|
||||
|
||||
The `REPORTED` action above still stands (no token to auto-create a NousResearch issue); the **MONITORING**
|
||||
condition is now satisfied — no recurrence across four log generations.
|
||||
Reference in New Issue
Block a user