audit: 2026-10-26 nightly — planned restart orphaned a live cron run (new incident page); 503 burst fully closed; radar cover-letter PDF pipeline moved to Chromium-over-HTML
This commit is contained in:
@@ -3,9 +3,9 @@ title: Gateway Exited Uncleanly on 2026-10-20 — One Cron Run Orphaned (status
|
||||
type: incident
|
||||
status: active
|
||||
created: 2026-10-21
|
||||
updated: 2026-10-21
|
||||
verified_on: 2026-10-21
|
||||
review_after: 2026-10-28
|
||||
updated: 2026-10-27
|
||||
verified_on: 2026-10-27
|
||||
review_after: 2026-11-03
|
||||
confidence: high
|
||||
tags: [gateway, cron, scheduler, lifecycle, restart, silent-failure]
|
||||
sources: [live-logs, live-gateway-ledger, live-sqlite-cron-executions, live-sqlite-sessions, live-probe]
|
||||
@@ -87,6 +87,7 @@ sqlite3 ~/.hermes/.hermes/cron/deliveries.db \
|
||||
|
||||
## Related
|
||||
|
||||
- [[incidents/2026-10-26-planned-restart-orphaned-cron-run]] — **recurrence on 2026-10-26, and the loud-orphan gap here is now partly closed.** This one was a **planned `SIGTERM`** (not an unclean exit) and the drain still **timed out after 1.35 s**, interrupting `ea79a2107e36` (Mnemosyne Health Monitor) 2 h 34 m in. Unlike this page's case it closed **`failed`** on a **new** string (`lost its durable fire claim ownership`), **raised two `cron_incidents` rows**, and **delivered** the failure — but the work still did not happen. Any triage keyed on this page's `unknown` / "owner exited before a durable terminal state" wording will miss it.
|
||||
- [[systems/scheduled-tasks]]
|
||||
- [[incidents/2026-10-18-degenerate-stream-canned-substitute]] — the other 2026-10-20 non-delivery (a canned substitute served to a user).
|
||||
- [[incidents/2026-08-08-gitea-config-backup-push-dead]] — the other "the row is green, the work did not happen" case found in this audit.
|
||||
|
||||
Reference in New Issue
Block a user