stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)

- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
This commit is contained in:
Hermes
2026-11-11 03:52:48 +08:00
parent 7ceaec1a2f
commit 341dda95dd
41 changed files with 579 additions and 102 deletions
@@ -3,9 +3,9 @@ title: Mnemosyne Automatic Capture Failure
type: incident
status: active
created: 2026-08-01
updated: 2026-10-20
updated: 2026-11-11
resolved_on: 2026-10-20
verified_on: 2026-10-20
verified_on: 2026-11-11
confidence: high
tags: [memory, mnemosyne, hermes, incident]
sources: [live-config, mnemosyne-db, hermes-cli-e2e]
@@ -82,6 +82,14 @@ no Mnemosyne trace.
`~/.hermes/backups/mnemosyne-capture-fix-20260801_180249/` and
`~/.hermes/backups/legacy-memory-slim-20260801_180702/`.
> **Resolution re-confirmed 2026-11-11.** The whole `memory.mnemosyne` block in
> `~/.hermes/config.yaml` is still natively typed — `sync_roles: [user]` (list),
> `skip_contexts: [cron, flush, subagent, background, skill_loop]` (list),
> `reflect: {disabled_for_cron: true, max_calls_per_session: 3}` (dict), `sleep_threshold: 50`
> (int), `ignore_patterns: []` (list), `profile_isolation: true` (bool), `auto_sleep: true`,
> `default_scope: session`, `vector_type: int8`. No stringification regression. Both backup
> directories named in the repair still exist. `status: resolved` stands.
## Related
- [[systems/mnemosyne]]
- [[systems/memory-backend]]
+21 -2
View File
@@ -3,8 +3,8 @@ title: "Gmail Read-State Incident (2026-08-29)"
type: incident
status: active
created: 2026-08-29
updated: 2026-10-20
verified_on: 2026-10-20
updated: 2026-11-11
verified_on: 2026-11-11
confidence: high
tags: [email, gmail, cron, himalaya, incident]
sources: [session:default/20260829_204103_9e1087]
@@ -80,6 +80,25 @@ Despite using `--preview`, Himalaya may still mark messages as read depending on
> **not** read-state errors — but the read-state exposure persists regardless, because the
> newsletter fetch path still issues `message read`.
>
> **Re-verified 2026-11-11 — the cron error state has cleared; the read-state root cause has not.**
>
> | Item | State on 2026-10-20 | State on 2026-11-11 |
> |---|---|---|
> | `goimapnotify.service` | active (PID 383) | **still active**, now PID **441**, same binary/conf path |
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False`, `last_status: ok` |
> | `fetch_newsletters.py` body-fetch call | `himalaya message read --preview …` | **identical, still unfixed** — the unsafe call is still in the file |
> | `85edf6772bca` Tech & AI Newsletter Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 07:05 |
> | `08eb86f02d37` Gmail Action Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 09:18 |
> | `06a0a1c8704f` AI Newsletter Curation for Patrick | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-05 10:41 |
>
> So 74 days after the incident: **both** original mitigations that were applied on 2026-08-29 are
> now **inert** — the monitor is still paused, but `goimapnotify` is back up, so IMAP IDLE is live
> again — and the root cause (`himalaya message read` on Anthony's INBOX) is untouched. The three
> email cron jobs that were in `error` on 2026-10-20 are now green, but that says nothing about
> the read-state exposure: a job that marks emails read and *succeeds* still pollutes the mailbox.
> Per the skill's own rule (a delivered/successful run is not evidence of correct behaviour), the
> exposure is best judged from the code path, not the cron status — and the code path is unchanged.
>
> **Recommended action (not taken here — this is a wiki job, and the fix touches Anthony's mail):**
> rewrite `fetch_newsletters.py` to use readonly IMAP, and decide whether `goimapnotify` should be
> running. See [[systems/scheduled-tasks]] and the Gmail IMAP lessons in
@@ -3,9 +3,9 @@ title: Nous Portal Refresh Token Invalid — Context Summaries Unavailable
type: incident
status: open
created: 2026-10-26
updated: 2026-10-26
verified_on: 2026-10-26
review_after: 2026-11-09
updated: 2026-11-11
verified_on: 2026-11-11
review_after: 2026-12-11
confidence: high
tags: [hermes, router, memory, incident, unresolved]
sources: [gateway-log, live-config, live-probe]
@@ -42,6 +42,30 @@ sources: [gateway-log, live-config, live-probe]
- Re-authenticate the Nous Portal credential, or remove `nous` from `fallback_providers`.
- Until then, a failover event will attempt a dead credential silently.
> **Re-verified 2026-11-11 — the refresh-token failure is gone from the logs, and the exposure has
> changed shape rather than closed.**
>
> - **Zero `invalid_grant` / refresh-token lines** now appear in `~/.hermes/logs/errors.log` (which
> covers the window through 2026-11-10 19:19) — so the credential is no longer failing at *auth*.
> `~/.hermes/.env` carries no `NOUS_API_KEY`; `~/.hermes/auth.json` is present and live.
> - What appears instead is **404 `insufficient_credits_for_paid_model`** for
> `anthropic/claude-haiku-4.5` — 3 occurrences, most recent **2026-11-10 19:19:33**, from session
> `20261031_190813_95fbde`.
> - **A ninth auxiliary slot now points back at `nous`.** The 2026-10-25 remediation moved all
> *eight* slots off `nous` onto `custom:omni-101` / `no-think/freeapi/auto`, but
> `auxiliary.pulse_memory` (`provider: nous`, `model: anthropic/claude-haiku-4.5`) was added
> afterwards and **is** the slot producing those 404s. So the masking is intact for the original
> eight slots but a new, single-slot nous dependency exists and is **failing on credits**, not on
> auth.
> - `nous` is **still the first entry** in `fallback_providers`
> (`meituan/longcat-2.5-preview:free`) — that is a *free* model and has not been observed failing,
> so the failover risk described below is now "may burn an attempt", not "guaranteed dead".
>
> **Revised open item:** decide whether `auxiliary.pulse_memory` should be moved onto
> `custom:omni-101` like the other eight slots (it is the only active nous path left, and it is
> currently erroring), and/or add credits for paid nous models. The original re-authenticate item
> is **superseded** — auth works now.
## Related
- [[systems/model-providers]]
- [[systems/current-profile]]