audit(2026-10-20): omni-101 503 burst 2 — incident back to active; scheduled-tasks rebuilt from jobs.json
- burst 2 (2026-10-19): 9 model calls across 8 jobs, 159/13 day - burst-2 table names/schedules/aliases corrected from live jobs.json - non-clean last_status 7 -> 12; unclosed incident rows 13 (query stated) - 2026-10-17 tail corrected 162/5 -> 163/5
This commit is contained in:
@@ -912,3 +912,31 @@ with a working tree carrying uncommitted 10-17/10-18 changes, and it has **no re
|
||||
pushed. **Not fixed** — a full mirror pass is a bigger call than an audit's mandate; flagged here.
|
||||
|
||||
No secrets written; host names, IPs, VMIDs, job/incident IDs, ports and probe status codes only.
|
||||
|
||||
## [2026-10-20] audit | Nightly wiki audit — `omni-101` 503 burst 2 (2026-10-19)
|
||||
|
||||
- **Incident re-opened.** [[incidents/2026-10-16-omniroute-503-resource-pressure]] went back to
|
||||
`status: active`. A second, identical `HTTP 503 … resource pressure` burst hit `omniroute-1` on 2026-10-19 —
|
||||
**9 model calls across 8 jobs** (06:01, then 07:01 → 12:01) — against burst 1's 9 across 7
|
||||
(2026-10-16 08:00 → 10-17 03:00). Whole day **159 completed / 13 failed**; 2026-10-18 was 168 / 1.
|
||||
- **Facts corrected from the source of truth** (`~/.hermes/cron/jobs.json`, `executions.db`) instead of being
|
||||
carried forward: the burst-2 table's job *names*, *schedules* and *model aliases* were wrong
|
||||
(`85edf6772bca` is **Tech & AI Newsletter Digest (Analytical)**, not "Fringe Festival Plan"; `ea79a2107e36`
|
||||
is the **Mnemosyne Health Monitor**, not a news digest; `ecc1fc13655a` is **FreshRSS Midday News Briefing**),
|
||||
`08eb86f02d37` fails at **08:31** (not 08:01, it is a `30 8 * * *` job), and 2026-10-17 reads **163 / 5**,
|
||||
not 162 / 5. **All eight failing jobs read `provider: custom:omni-101`, `model: smart-route`** — the burst-1
|
||||
note's wider alias mix (`deepseek-paid-smart`, `free-IA`, `dark`) no longer matches `jobs.json` and is now
|
||||
flagged in the page for confirmation rather than silently overwritten.
|
||||
- **Incident ledger read properly for the first time.** Unclosed rows = **13** (12 `alerted` / 1 `detected`),
|
||||
not the `7` the 10-19 entry recorded — the earlier figures used a narrower filter. A raw `closed_at IS NULL`
|
||||
count reads **12** at 10-19 00:20 and **13** now; the entry states the query so future entries compare like
|
||||
with like. 4 rows opened and 2 closed on 10-19, plus `ea79a2_747b27556fd0` closed 10-20 00:04:24.
|
||||
- **Pages updated:** `incidents/2026-10-16-omniroute-503-resource-pressure.md` (burst-2 table rebuilt from
|
||||
`jobs.json`, new findings, status back to active), `systems/scheduled-tasks.md` (new current-count bullet,
|
||||
non-clean `last_status` rewritten 7 → 12, Known Issues addition, 8 job-table rows corrected),
|
||||
`systems/omniroute.md` (new Recent Changes section), `current-state.md` (new Recent Changes section),
|
||||
`index.md` (scheduled-tasks and incident summaries, "Last updated" → 2026-10-20), `log.md`.
|
||||
- **Nothing removed or archived** — no page became obsolete and no one-way sync was run.
|
||||
- **Fleet:** 28 jobs / 25 enabled / 3 disabled (unchanged). **15 sessions in the window, all `cron`** — no
|
||||
interactive sessions, so there were no user-facing decisions to record.
|
||||
- No secrets written; host names, IPs, job/incident IDs, ports and probe status codes only.
|
||||
|
||||
Reference in New Issue
Block a user