audit(2026-11-11): the audit's own 429 killed the 11-10 window; stocktake + radar recover with output
Window 2026-11-10 23:59 -> 2026-11-11 23:59 (two windows: the 11-10 audit failed). NEW incident: incidents/2026-11-10-wiki-audit-429-all-targets-skipped.md - cbfe434a3b5e ran 23:59:54.486572 -> 2026-11-11 00:12:29.646987 = 12m35s, failed, delivery suppressed - RuntimeError: HTTP 429 ... all targets skipped by pre-dispatch filters (credits 100% conn:3021bab6) reset after 6h24m31s; 6 candidates, attempted 0; tenant tn-5ud77abtl1 quota exceeded - incident row cbfe43_ed01845310d8 (rate_limit) alerted_at NULL, closed_at NULL = only open row - consequence: the 2026-11-10 window was never audited; non-clean last_status 1 -> 2 RESOLVED with output (dated Recovery sections added): - stocktake 1a8e90ea2271 03:00:56.081504 -> 04:11:38.724852 = 1h10m42s, 36 stale re-verified -> 0 - job radar stage1 07:01:08 -> 07:37:34 = 36m26s, first handoff since 2026-11-06 (22,682 B) - radar stage2 recovery fire 07:35:56 -> 08:03:47 closed row 5fc292_773822455856 resolved UPDATED pages: current-state.md, systems/scheduled-tasks.md, systems/opencode-go.md (opencode-zen 401 api_key_expired now observed 7x, paired with nous 404 credits 14x), incidents/2026-10-22-nous-portal-refresh-token-invalid.md, index.md, log.md
This commit is contained in:
+35
-3
@@ -3,8 +3,8 @@ title: OpenCode Go
|
||||
type: system
|
||||
status: active
|
||||
created: 2026-07-29
|
||||
updated: 2026-11-08
|
||||
verified_on: 2026-11-08
|
||||
updated: 2026-11-11
|
||||
verified_on: 2026-11-11
|
||||
confidence: high
|
||||
tags: [opencode, provider, models, subscription, limits]
|
||||
sources: [https://opencode.ai/docs/go/]
|
||||
@@ -160,12 +160,44 @@ both returned **HTTP 401** on the Go endpoint:
|
||||
use.
|
||||
- **To restore:** mint a new key in the OpenCode console, then replace it in **both** stores — the Hermes
|
||||
provider entry and `~/.local/share/opencode/auth.json` — and re-probe `…/zen/go/v1/models` for a 200.
|
||||
(Keys live in the vault; this page records only their state — [[references/hermes-api-vault]].)
|
||||
(Keys live in the vault; this page records only their state — the `hermes-api-vault` skill (vault-backed; no value is stored in this wiki).)
|
||||
|
||||
**Config format:** `opencode-go/<model-id>` (e.g. `opencode-go/kimi-k3`)
|
||||
|
||||
**Model list API:** `https://opencode.ai/zen/go/v1/models`
|
||||
|
||||
## 2026-11-11 — the expired credential is no longer latent: it is in the live fallback path
|
||||
|
||||
The 2026-11-08 record above closed with *"**No execution failed on an auth error** — nothing in the fleet
|
||||
was actually consuming this provider in-window. The risk is latent: anything pinned to it fails on first
|
||||
use."* **That latent risk has now fired, repeatedly.**
|
||||
|
||||
- `fallback_providers[1]` is `opencode-zen` / `deepseek-v4.1-flash` (`https://opencode.ai/zen/v1`,
|
||||
`api_mode: chat_completions`) — i.e. the **second hop of every session fallback chain**.
|
||||
- On **2026-11-11** it produced **7 × HTTP 401 `api_key_expired`**, each one *immediately* after the
|
||||
hop above it (nous `anthropic/claude-haiku-4.5`) returned **404 `insufficient_credits_for_paid_model`**:
|
||||
|
||||
| # | nous 404 | opencode-zen 401 |
|
||||
|---|---|---|
|
||||
| 1 | 07:01:11 | 07:01:13 |
|
||||
| 2 | 07:19:28 | 07:19:30 |
|
||||
| 3 | 08:03:38 | 08:03:40 |
|
||||
| 4 | 13:03:27 | 13:03:28 |
|
||||
| 5 | 13:19:31 | 13:19:33 |
|
||||
| 6 | 18:03:38 | 18:03:39 |
|
||||
| 7 | 19:19:37 | 19:19:39 |
|
||||
|
||||
- **Two consecutive dead hops** on one chain: a paid model with no credit, then an expired key. Every
|
||||
affected turn has to fall through to `custom:guacano` / `custom:litellm` / `kilocode` to succeed.
|
||||
- ⚠ Nothing in the fleet is *pinned* to `opencode-zen` — no cron job names it — so this never shows up as
|
||||
a job failure. It shows up only as `agent.chat_completion_helpers: Streaming failed before delivery:
|
||||
Error code: 401` in `~/.hermes/logs/errors.log`. → [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]
|
||||
|
||||
**To restore:** mint a new key in the OpenCode console and replace it in both stores (the Hermes provider
|
||||
entry and `~/.local/share/opencode/auth.json`), then re-probe `…/zen/go/v1/models` for a 200 — or
|
||||
**remove `opencode-zen` from `fallback_providers`**, since a dead second hop costs a round-trip on every
|
||||
retry. Keys live in the vault; this page records only their state — the `hermes-api-vault` skill (vault-backed; no value is stored in this wiki).
|
||||
|
||||
## Privacy
|
||||
|
||||
Zero-retention policy from providers — your data is not used for model training.
|
||||
|
||||
Reference in New Issue
Block a user