stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13) - skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299) - runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated - comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest - incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised - incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused) - 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists) - log.md addendum + index.md date bump; no page deleted, no secrets written
This commit is contained in:
@@ -3,9 +3,9 @@ title: Mnemosyne Automatic Capture Failure
|
||||
type: incident
|
||||
status: active
|
||||
created: 2026-08-01
|
||||
updated: 2026-10-20
|
||||
updated: 2026-11-11
|
||||
resolved_on: 2026-10-20
|
||||
verified_on: 2026-10-20
|
||||
verified_on: 2026-11-11
|
||||
confidence: high
|
||||
tags: [memory, mnemosyne, hermes, incident]
|
||||
sources: [live-config, mnemosyne-db, hermes-cli-e2e]
|
||||
@@ -82,6 +82,14 @@ no Mnemosyne trace.
|
||||
`~/.hermes/backups/mnemosyne-capture-fix-20260801_180249/` and
|
||||
`~/.hermes/backups/legacy-memory-slim-20260801_180702/`.
|
||||
|
||||
> **Resolution re-confirmed 2026-11-11.** The whole `memory.mnemosyne` block in
|
||||
> `~/.hermes/config.yaml` is still natively typed — `sync_roles: [user]` (list),
|
||||
> `skip_contexts: [cron, flush, subagent, background, skill_loop]` (list),
|
||||
> `reflect: {disabled_for_cron: true, max_calls_per_session: 3}` (dict), `sleep_threshold: 50`
|
||||
> (int), `ignore_patterns: []` (list), `profile_isolation: true` (bool), `auto_sleep: true`,
|
||||
> `default_scope: session`, `vector_type: int8`. No stringification regression. Both backup
|
||||
> directories named in the repair still exist. `status: resolved` stands.
|
||||
|
||||
## Related
|
||||
- [[systems/mnemosyne]]
|
||||
- [[systems/memory-backend]]
|
||||
|
||||
@@ -3,8 +3,8 @@ title: "Gmail Read-State Incident (2026-08-29)"
|
||||
type: incident
|
||||
status: active
|
||||
created: 2026-08-29
|
||||
updated: 2026-10-20
|
||||
verified_on: 2026-10-20
|
||||
updated: 2026-11-11
|
||||
verified_on: 2026-11-11
|
||||
confidence: high
|
||||
tags: [email, gmail, cron, himalaya, incident]
|
||||
sources: [session:default/20260829_204103_9e1087]
|
||||
@@ -80,6 +80,25 @@ Despite using `--preview`, Himalaya may still mark messages as read depending on
|
||||
> **not** read-state errors — but the read-state exposure persists regardless, because the
|
||||
> newsletter fetch path still issues `message read`.
|
||||
>
|
||||
> **Re-verified 2026-11-11 — the cron error state has cleared; the read-state root cause has not.**
|
||||
>
|
||||
> | Item | State on 2026-10-20 | State on 2026-11-11 |
|
||||
> |---|---|---|
|
||||
> | `goimapnotify.service` | active (PID 383) | **still active**, now PID **441**, same binary/conf path |
|
||||
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False`, `last_status: ok` |
|
||||
> | `fetch_newsletters.py` body-fetch call | `himalaya message read --preview …` | **identical, still unfixed** — the unsafe call is still in the file |
|
||||
> | `85edf6772bca` Tech & AI Newsletter Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 07:05 |
|
||||
> | `08eb86f02d37` Gmail Action Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 09:18 |
|
||||
> | `06a0a1c8704f` AI Newsletter Curation for Patrick | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-05 10:41 |
|
||||
>
|
||||
> So 74 days after the incident: **both** original mitigations that were applied on 2026-08-29 are
|
||||
> now **inert** — the monitor is still paused, but `goimapnotify` is back up, so IMAP IDLE is live
|
||||
> again — and the root cause (`himalaya message read` on Anthony's INBOX) is untouched. The three
|
||||
> email cron jobs that were in `error` on 2026-10-20 are now green, but that says nothing about
|
||||
> the read-state exposure: a job that marks emails read and *succeeds* still pollutes the mailbox.
|
||||
> Per the skill's own rule (a delivered/successful run is not evidence of correct behaviour), the
|
||||
> exposure is best judged from the code path, not the cron status — and the code path is unchanged.
|
||||
>
|
||||
> **Recommended action (not taken here — this is a wiki job, and the fix touches Anthony's mail):**
|
||||
> rewrite `fetch_newsletters.py` to use readonly IMAP, and decide whether `goimapnotify` should be
|
||||
> running. See [[systems/scheduled-tasks]] and the Gmail IMAP lessons in
|
||||
|
||||
@@ -3,9 +3,9 @@ title: Nous Portal Refresh Token Invalid — Context Summaries Unavailable
|
||||
type: incident
|
||||
status: open
|
||||
created: 2026-10-26
|
||||
updated: 2026-10-26
|
||||
verified_on: 2026-10-26
|
||||
review_after: 2026-11-09
|
||||
updated: 2026-11-11
|
||||
verified_on: 2026-11-11
|
||||
review_after: 2026-12-11
|
||||
confidence: high
|
||||
tags: [hermes, router, memory, incident, unresolved]
|
||||
sources: [gateway-log, live-config, live-probe]
|
||||
@@ -42,6 +42,30 @@ sources: [gateway-log, live-config, live-probe]
|
||||
- Re-authenticate the Nous Portal credential, or remove `nous` from `fallback_providers`.
|
||||
- Until then, a failover event will attempt a dead credential silently.
|
||||
|
||||
> **Re-verified 2026-11-11 — the refresh-token failure is gone from the logs, and the exposure has
|
||||
> changed shape rather than closed.**
|
||||
>
|
||||
> - **Zero `invalid_grant` / refresh-token lines** now appear in `~/.hermes/logs/errors.log` (which
|
||||
> covers the window through 2026-11-10 19:19) — so the credential is no longer failing at *auth*.
|
||||
> `~/.hermes/.env` carries no `NOUS_API_KEY`; `~/.hermes/auth.json` is present and live.
|
||||
> - What appears instead is **404 `insufficient_credits_for_paid_model`** for
|
||||
> `anthropic/claude-haiku-4.5` — 3 occurrences, most recent **2026-11-10 19:19:33**, from session
|
||||
> `20261031_190813_95fbde`.
|
||||
> - **A ninth auxiliary slot now points back at `nous`.** The 2026-10-25 remediation moved all
|
||||
> *eight* slots off `nous` onto `custom:omni-101` / `no-think/freeapi/auto`, but
|
||||
> `auxiliary.pulse_memory` (`provider: nous`, `model: anthropic/claude-haiku-4.5`) was added
|
||||
> afterwards and **is** the slot producing those 404s. So the masking is intact for the original
|
||||
> eight slots but a new, single-slot nous dependency exists and is **failing on credits**, not on
|
||||
> auth.
|
||||
> - `nous` is **still the first entry** in `fallback_providers`
|
||||
> (`meituan/longcat-2.5-preview:free`) — that is a *free* model and has not been observed failing,
|
||||
> so the failover risk described below is now "may burn an attempt", not "guaranteed dead".
|
||||
>
|
||||
> **Revised open item:** decide whether `auxiliary.pulse_memory` should be moved onto
|
||||
> `custom:omni-101` like the other eight slots (it is the only active nous path left, and it is
|
||||
> currently erroring), and/or add credits for paid nous models. The original re-authenticate item
|
||||
> is **superseded** — auth works now.
|
||||
|
||||
## Related
|
||||
- [[systems/model-providers]]
|
||||
- [[systems/current-profile]]
|
||||
|
||||
Reference in New Issue
Block a user