audit corrections 2026-11-05: fleet count 29/26/3 (was 158/144/14), 11-04 closes 167 completed/2 failed/1 running, 11-03 re-settled 171/1, deepseek-paid-smart 503 on omniroute:20128 vs 200 on omniroute-1

This commit is contained in:
Tony0410
2026-11-05 01:38:41 +08:00
parent 3b5daccb4d
commit 20aa11b0c4
5 changed files with 49 additions and 8 deletions
+30 -1
View File
@@ -1978,7 +1978,7 @@ embedded credential was seen during verification and is not reproduced here ([RE
00:54:21 AWST: `LOCK: 0`, both `cgroup.freeze` flags `0`, guest `ALIVE`, `sftpgo: active`, 0 vzdump procs.
- **Updated [[infrastructure/tailscale]]** - the sftpgo row now records recovery from *two* freezes.
- **Updated [[systems/scheduled-tasks]]** - new ledger bullet: 11-04 settled **170 rows - 167 `completed` /
2 `failed` / 1 `item_suppressed`**; non-clean `last_status` **2 -> 3**; unclosed incident rows **1 -> 3**;
2 `failed` / 1 `running`**; non-clean `last_status` **2 -> 3**; unclosed incident rows **1 -> 3**;
deliveries **8/8 `delivered`**; the two failures are one upstream signature; stocktake overran for the third
day (5 h 41 m 46 s, `free-stack` pin).
- **Updated [[runbooks/job-radar]]** - cover-letter model route: `regenerate_cover_letter.py` defaults to
@@ -1998,3 +1998,32 @@ embedded credential was seen during verification and is not reproduced here ([RE
decision reversed. **No secrets, passwords, tokens or keys written or quoted.**
- *network-map.md re-verified (live probe 2026-11-05):* corrected the ct460 port table — **9377 is Camofox browser REST** (`/home/hermes/services/camofox-browser`), not Stratos; `8790` and `5174` are no longer listening; added `9120` (wiki serve), `9222` (Camofox CDP/WS), `7777` (OpenChamber), `6767` (Paseo), `8085` (AgentMail webhook), `9900` (gateway). Added the full Tailscale Serve table (12 routes on `:443`, `/` on `:9443` → 5173, Funnel `:8443` `/webhook` → 8085). Two routes are dead: `/agent-estate` → 8769 and `/synergy-prep` → 9123 have no listener.
## [2026-11-05] correction + live probe | fleet count, 11-04 ledger composition, `deepseek-paid-smart` dispatch
- **What was wrong.** The 2026-11-05 nightly-audit entry (written 00:5x AWST) recorded the fleet as **158 jobs
(144 enabled / 14 disabled)** and the 11-04 ledger as closing **1 `item_suppressed`**. Both were wrong. Live
re-reads - `~/.hermes/cron/jobs.json`, `hermes cron list`, and the in-flight job count - each give **29 jobs /
26 enabled / 3 disabled**; `executions.db` shows 11-04 at **170 rows - 167 `completed` / 2 `failed` / 1
`running`** (this audit), with **no `item_suppressed` status anywhere on 11-04**.
- **Where corrected:** `systems/scheduled-tasks.md` (Count bullet + correction clause),
`current-state.md` (11-05 block), `index.md` (11-05 ledger paragraph), and this entry. No other page carried
the bad number.
- **11-03 re-settle:** `171 completed / 1 failed` (172 rows) - its `running` row has since completed - so 11-03
and 11-04 are the only consecutive days with a `failed` row since 10-31.
- **Live probe - model dispatch, 2026-11-05 01:33:12 AWST (first-hand, keyed):**
`POST http://omniroute:20128/v1/chat/completions` with `deepseek-paid-smart` -> **HTTP 503**
`ALL_TARGETS_SKIPPED` (`poolSize: 5`, `attempted: 0`). The same request to
`http://100.78.63.20:20128/v1/chat/completions` (`omniroute-1`) -> **HTTP 200**, `model:
deepseek/deepseek-v4-flash`. Unkeyed `/v1/models` on both hosts returns **401** - auth-gating, not health. The
fleet's only `deepseek-paid-smart` pin is **this audit job** (`cbfe434a3b5e`, provider `custom:omni-101`, i.e.
the `omniroute-1` node), which is why the audit still completes while a direct CT600 probe 503s.
- **Re-verified and matching (no change needed):** `cron_incidents` - 3 unclosed rows, all `state = alerted`;
`deliveries.db` - 8 rows on 11-04, **8/8 `delivered`**, including both `for_failure` notices (09:41:43,
09:42:20); non-clean `last_status` **3**; pin distribution **14 `smart-route` / 6 `free-stack` / 8 unpinned /
1 `deepseek-paid-smart`**.
- **Files touched this pass:** `systems/scheduled-tasks.md`, `current-state.md`, `runbooks/job-radar.md`
(`CL_MODEL` caveat now carries the keyed probe), `index.md`, `log.md`. Frontmatter `updated` / `verified_on`
bumped to 2026-11-05 where stale.
- **Not done / unchanged:** no job added, removed, disabled or re-pinned; no page removed or archived; no
decision reversed. **No secrets, passwords, tokens or keys written or quoted** - the API key was read from
`~/.hermes/.env` into the shell for the probe only.