wiki: nightly audit 2026-11-03 window — new Job Radar stream-timeout incident, hermes-hq-push system page; update 11-02 recurrence, scheduled-tasks, index, log

This commit is contained in:
Hermes Wiki Audit
2026-11-04 00:16:23 +08:00
parent ca014efd63
commit 785fbc40b2
8 changed files with 534 additions and 13 deletions
+53
View File
@@ -1907,3 +1907,56 @@ recorded wrong.**
- **DRIFT (context, not findings):** touched pages follow the vault's existing conventions and
SCHEMA.md frontmatter rules. **No secrets, passwords, tokens or keys written or quoted** — every job
id, PID, thread id and timestamp above is a non-sensitive operational identifier.
## [2026-11-04] audit — window 2026-11-03
**Verdict: one genuine red row, and the ledger produced it itself.** 2026-11-03 settled **170 `completed` /
1 `failed` / 1 `running` / 0 `unknown`** (172 rows) against 11-02's 177 / 2 — the **first `failed` row since
10-31**.
**New incident page:** `incidents/2026-11-03-job-radar-stream-active-timeout.md` — Job Radar Stage 1
(`dca8482e4f76`) ran **07:00:33.819190 → 13:28:52.305995 = 6 h 28 m 18 s** and died on
**`RuntimeError: stream active timeout`**. Its session (`local-7180cef7…`) compacted **five times** (08:04:35,
08:47:19, 09:31:50, 10:49:26, 11:39:21) and went **silent for 65 m 21 s** after its last tool result at
**12:23:28** before the harness recorded *"This turn did not complete."* at 13:28:49. Incident
`dca848_dfd4bea84f2a` (`failure_type: timeout`) opened 13:28:51.463929, **alerted** 13:28:55.141618, **still
open** → fleet unclosed rows **0 → 1**. The failure notice was delivered at **13:28:51.741724** and Pulse
carried it to Anthony (report `p34c8e899-1`, desktop session `20261031_190813_95fbde`: *"…dying on a stream
active timeout"*).
**Updated:** `incidents/2026-11-02-job-radar-stage1-iteration-cap-recurrence.md` — new 11-03 row in the
recurrence table plus a 2026-11-04 update section that **answers its open question 3**: output delivery
tracks the *terminal state*, not the run length. 11-02 ended `completed` → delivered nothing; 11-03 ended
`failed` → notice delivered and a row raised. Open question 2 narrows: the ledger **can** record a non-clean
state when the harness kills the turn; the iteration cap is the case that still escapes as `completed`.
Question 4 is answered against us: **2026-11-02's candidates and 11-03's discovery are both unrecovered.**
**Second lost Stage-1 day:** both Stage 2 slots declined silently again (`5fc292ff0f3b` 11:30:36 → 11:32:16,
14:30:32 → 14:33:11) and `~/.hermes/radar-ref/handoff/` still holds **only** 10-27, 10-29, 10-30.
**New page:** `systems/hermes-hq-push.md` — plugin **`hermes-hq-push` v0.9.27** (`kind: standalone`, author
*Hermes HQ*), tree mtime **2026-11-03 11:06**, **enabled** (`hermes plugins list` → `enabled`, state `user`;
`config.yaml` `plugins.enabled` line 7556). Its own manifest states it **replaces `dispatch-push`**, and the
replacement is clean: **no `dispatch-push` reference survives in `config.yaml`** and no such directory
remains. It is a *different* stack from the Cadu/Vory bridge on `systems/push-notifications.md` (untouched,
still `degraded`). **Operational note recorded:** the plugin's `pre_tool_call` guard rejects any tool call
whose arguments contain the literal text of a launchd one-off-restart command — a wiki write quoting it was
blocked and had to be rephrased.
**Fleet:** 29 jobs — 26 enabled / 3 disabled (unchanged). **Non-clean `last_status` 1 → 2:**
`dca8482e4f76` (`error`, streak 1) joins `066bb5191280` (unchanged `delivery_failed`). **Deliveries: 7 rows,
7/7 `delivered`** (07:13:49, 07:30:35, 08:39:25, 12:37:09, **13:28:51 `for_failure`**, 13:49:11, 18:45:39).
⚠ **The 03:00 stocktake ran long for the second day:** `1a8e90ea2271` 03:00:30.495724 → 10:41:13.851030 =
**7 h 40 m 43 s**, `completed` / `suppressed`, **delivered nothing**, and again left its artefacts
uncommitted (`systems/skills-index.md` modified, `raw/stocktake/freshness-20261103.txt` untracked) — this
audit committed them.
**Sessions in the window:** 16 (14 `cron`, 1 `ios` `20260908_031541_49c051`, 1 `desktop`
`20261031_190813_95fbde`) — 14 started, all `cron`. **No gateway restart:** PID **175269** is still the
2026-10-31 19:23:56 start (`hermes-dashboard` PID **176771**, 19:35:02).
**Not done / unchanged:** no page removed or archived; no job added, removed, disabled or repinned; no
decision reversed. **No secrets, passwords, tokens or keys written or quoted** — a Gitea remote URL with an
embedded credential was seen during verification and is not reproduced here ([REDACTED]).