2 Commits
Author SHA1 Message Date
Hermes ab3652aa20 Stocktake 2026-11-12: tailscale live-vs-page drift corrected; sftpgo incident re-verified; pre-dispatch-filter signature found recurring on a second job/pin
- infrastructure/tailscale.md: node table rebuilt from live tailscale status --json.
  mailflow/qwenpaw/springroll read Online:false in the peer record but all three ping
  (CT1262 + CT465 running; springroll via DERP(sin)) - stale peer records, rows stay online.
  omarchymac genuinely stale in the other direction (offline 2d/11-08 -> offline 4d, ping timeout).
  9 named offline ages + 6 codex ephemeral peers refreshed. New guest CT1007 homebridge
  (running, tailscaled inactive). DEAD Serve targets 8769/9123 re-confirmed dead; CT201 table
  unchanged; container counts 17 LXC+4 VM (pve) / 20 LXC+2 VM (m93p) confirmed.
- incidents/2026-09-02-sftpgo-outage.md: 22d stale, re-probed live (pct list running, peer
  Online:true, pong, healthz 401; swap now 512 not 43Gi). Cleared.
- incidents/2026-10-21-omniroute-pre-dispatch-filter-skip.md: signature recurred 2026-11-10 on
  cbfe434a3b5e (custom:omni-101/free-stack, HTTP 429 credits 100%); incident row resolved
  2026-11-12T00:25. Original subject clean 4 slots. Retention-floor caveat recorded.
- Threshold re-verifications (19d/18d): mnemosyne-relay (tools:None gone, 0 log hits),
  work-outlook (himalaya still login not supported), omniroute-free-deepseek
  (good-free/free-stack both HTTP 200, non-deepseek resolution).
- Correction recorded: previous entry mis-described the 2026-11-12 stale page as log.md.
- wiki-maintain.sh re-run after all edits: freshness 0 stale / 130 fresh / 3 no-date of 133.
2026-11-12 03:19:12 +08:00
Hermes Wiki Audit 729bb597cf audit(2026-11-11): the audit's own 429 killed the 11-10 window; stocktake + radar recover with output
Window 2026-11-10 23:59 -> 2026-11-11 23:59 (two windows: the 11-10 audit failed).

NEW incident: incidents/2026-11-10-wiki-audit-429-all-targets-skipped.md
 - cbfe434a3b5e ran 23:59:54.486572 -> 2026-11-11 00:12:29.646987 = 12m35s, failed, delivery suppressed
 - RuntimeError: HTTP 429 ... all targets skipped by pre-dispatch filters (credits 100% conn:3021bab6)
   reset after 6h24m31s; 6 candidates, attempted 0; tenant tn-5ud77abtl1 quota exceeded
 - incident row cbfe43_ed01845310d8 (rate_limit) alerted_at NULL, closed_at NULL = only open row
 - consequence: the 2026-11-10 window was never audited; non-clean last_status 1 -> 2

RESOLVED with output (dated Recovery sections added):
 - stocktake 1a8e90ea2271 03:00:56.081504 -> 04:11:38.724852 = 1h10m42s, 36 stale re-verified -> 0
 - job radar stage1 07:01:08 -> 07:37:34 = 36m26s, first handoff since 2026-11-06 (22,682 B)
 - radar stage2 recovery fire 07:35:56 -> 08:03:47 closed row 5fc292_773822455856 resolved

UPDATED pages: current-state.md, systems/scheduled-tasks.md, systems/opencode-go.md
(opencode-zen 401 api_key_expired now observed 7x, paired with nous 404 credits 14x),
incidents/2026-10-22-nous-portal-refresh-token-invalid.md, index.md, log.md
2026-11-12 00:16:39 +08:00