2026-11-18 stocktake: re-verify 7 stale pages against live state, resolve gmail-digest refire incident

This commit is contained in:
Tony0410
2026-11-18 03:19:54 +08:00
parent 4574c4ebc3
commit 98610466cc
11 changed files with 199 additions and 20 deletions
@@ -3,8 +3,8 @@ title: OmniRoute `omni-101` Resource-Pressure 503s — Recurring Cron Failure Bu
type: incident
status: monitoring
created: 2026-10-18
updated: 2026-10-27
verified_on: 2026-10-27
updated: 2026-11-18
verified_on: 2026-11-18
review_after: 2026-11-03
confidence: high
tags: [cron, router, provider, infrastructure]
@@ -35,6 +35,8 @@ days** (10-16 → 10-17 and 10-19) with an identical signature on an identical n
> string on 2026-10-25 or 2026-10-26.** Status stays **`monitoring`** for the same reason as 10-21: the rows are
> shut, the single-node exposure is not.
> **Re-verified 2026-11-18 — 22 days clean.** Zero execution rows carry the resource-pressure string since 2026-10-27 (verified against `executions.db`). Both nodes healthy at probe time: CT600 `/api/health` → 200 with `/models` → 200 (**6,501 models**), `omni-101` `/api/health` → 200, zero unclosed `cron_incidents` rows estate-wide. Status stays **`monitoring`**: the single-node exposure and the invisible-to-`/api/health` failure mode are unchanged.
**Burst 1:** first 503 failure row **2026-10-16 08:00:37**, last **2026-10-17 03:00:28** AWST — **19 hours**.
10-16 = **165 completed / 8 failed**, 10-17 = **163 / 5** (all five pre-03:01).
**Burst 2:** **2026-10-19**, two clusters — **06:01** (single job) and **07:01 → 12:01** (six slots across seven