Files
agent-estate-wiki/incidents/2026-10-22-nous-portal-refresh-token-invalid.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

3.8 KiB

title, type, status, created, updated, verified_on, review_after, confidence, tags, sources
title type status created updated verified_on review_after confidence tags sources
Nous Portal Refresh Token Invalid — Context Summaries Unavailable incident open 2026-10-26 2026-11-11 2026-11-11 2026-12-11 high
hermes
router
memory
incident
unresolved
gateway-log
live-config
live-probe

Nous Portal Refresh Token Invalid — Context Summaries Unavailable (2026-10-22 →)

Nous Portal runtime credentials unavailable: Invalid refresh token. Run hermes model to re-authenticate.

Occurrences (gateway logs, counted 2026-10-26)

Day Lines Notable
2026-10-22 1 11:32:20 tui_gateway.server: llm.oneshot failed
2026-10-24 3 —
2026-10-25 20 last at 14:53:04 — agent.context_compressor: Failed to generate context summary
2026-10-26 0 (to 00:1x) none since the 19:27:13 restart

Effect

  • Context compression failed outright. Sessions could not summarise, so working context grew unchecked — one cron fire reached 148 k tokens on 10-25 and ran 1 h 55 m. → incidents/2026-10-25-gmail-digest-runaway-and-refire
  • Every auxiliary LLM path pointed at nous failed with it.

What changed — and what did not

  • 2026-10-25, config.yaml written 19:13 AWST: all eight auxiliary: slots moved off nous to no-think/freeapi/auto on custom:omni-101, each with the chain no-think/freeapi/deepseek-v4-flash → no-think/kg/kilo-auto/free, free_only: true (compression timeout 90 s, vision timeout 180 s). → systems/model-providers
  • The credential is still invalid. The fault is masked, not fixed — no hermes model re-authentication has been run, and nous is still the first entry in fallback_providers.
  • Zero invalid_grant lines appear after the 2026-10-25 19:27:13 gateway start, so the masking is holding.

Open

  • Re-authenticate the Nous Portal credential, or remove nous from fallback_providers.
  • Until then, a failover event will attempt a dead credential silently.

Re-verified 2026-11-11 — the refresh-token failure is gone from the logs, and the exposure has changed shape rather than closed.

  • Zero invalid_grant / refresh-token lines now appear in ~/.hermes/logs/errors.log (which covers the window through 2026-11-10 19:19) — so the credential is no longer failing at auth. ~/.hermes/.env carries no NOUS_API_KEY; ~/.hermes/auth.json is present and live.
  • What appears instead is 404 insufficient_credits_for_paid_model for anthropic/claude-haiku-4.5 — 3 occurrences, most recent 2026-11-10 19:19:33, from session 20261031_190813_95fbde.
  • A ninth auxiliary slot now points back at nous. The 2026-10-25 remediation moved all eight slots off nous onto custom:omni-101 / no-think/freeapi/auto, but auxiliary.pulse_memory (provider: nous, model: anthropic/claude-haiku-4.5) was added afterwards and is the slot producing those 404s. So the masking is intact for the original eight slots but a new, single-slot nous dependency exists and is failing on credits, not on auth.
  • nous is still the first entry in fallback_providers (meituan/longcat-2.5-preview:free) — that is a free model and has not been observed failing, so the failover risk described below is now "may burn an attempt", not "guaranteed dead".

Revised open item: decide whether auxiliary.pulse_memory should be moved onto custom:omni-101 like the other eight slots (it is the only active nous path left, and it is currently erroring), and/or add credits for paid nous models. The original re-authenticate item is superseded — auth works now.