wiki audit 2026-11-07: Job Radar concurrent-fire claim discard (new incident page), scheduled-tasks re-verified section, index+log+runbook updates, freshness 2026-11-07
This commit is contained in:
+30
-4
@@ -3,16 +3,42 @@ title: Current State
|
||||
type: concept
|
||||
status: active
|
||||
created: 2026-07-22
|
||||
updated: 2026-11-06
|
||||
verified_on: 2026-11-06
|
||||
review_after: 2026-11-08
|
||||
updated: 2026-11-07
|
||||
verified_on: 2026-11-07
|
||||
review_after: 2026-11-09
|
||||
confidence: high
|
||||
tags: [system]
|
||||
sources: [live-cron-list, live-sqlite-cron-executions, live-sqlite-sessions, live-probe, live-executions-db, live-cron-incidents, live-deliveries-db, live-sessions-db]
|
||||
---
|
||||
|
||||
# Current State
|
||||
Updated: 2026-11-05
|
||||
Updated: 2026-11-07
|
||||
|
||||
## Recent Changes (2026-11-07, nightly audit; live probes 2026-11-07 00:3x-00:5x AWST)
|
||||
|
||||
- **A red row with no alert: the Job Radar's 11:30 slot was discarded as stale while a concurrent fire
|
||||
held the claim.** 2026-11-06 settled **172 rows across 16 jobs - 170 `completed` / 1 `failed` / 1
|
||||
`running`** (this audit). The failure is `5fc292ff0f3b`'s **11:30:32 slot**, interrupted at **13:26:38**
|
||||
and closed **13:26:44** on `Fire claim ownership lost; stale result was discarded.` - because a **`direct`
|
||||
fire of the same job (08:27:09 -> 13:26:22, 4 h 59 m 13 s)** was still holding the claim and delivered
|
||||
first. **Every previous occurrence of this string came from a restart orphaning a run; this one had no
|
||||
restart.** The failed row raised **no `cron_incidents` row** (newest ledger activity is still the
|
||||
2026-11-05T12:36:17 closures), so **nothing alerted**. The day still landed complete via the 14:30 slot
|
||||
(12 m 27 s, `MISSING_COVER_LETTER_COUNT 0`) ->
|
||||
[[incidents/2026-11-06-job-radar-concurrent-fire-claim-discard]].
|
||||
- **The stocktake's overrun is fully unwound.** `1a8e90ea2271` ran **03:00:32 -> 04:11:35 (1 h 11 m 03 s)**
|
||||
against 7 h 40 m / 5 h 41 m / 3 h 02 m on 11-03/11-04/11-05, and committed normally.
|
||||
- **Stage 1 ran clean for a second day** (`dca8482e4f76`, 07:00:44 -> 08:29:21, 1 h 28 m 37 s, `completed`,
|
||||
`suppressed`), and the **11-04 Open entry about its two lost candidate sets is cleared in substance** -
|
||||
its 11-03 ledger row closed `resolved` at 2026-11-05T12:36:17 and both runs since have been clean.
|
||||
- **Ledger clean, deliveries clean:** **unclosed incident rows 0 -> 0**; deliveries **10 rows / 10
|
||||
`delivered` / 0 `for_failure`**; fleet **29 jobs (26 enabled / 3 disabled)** - fifth consecutive audit at
|
||||
the same count, non-clean `last_status` still **1** (`066bb5191280`, `delivery_failed`, streak 0) ->
|
||||
[[systems/scheduled-tasks]].
|
||||
- **The previous audit's open question closed itself:** the Mnemosyne Health Monitor's 11-05 12:00:45 slot
|
||||
(still in flight 12 h+ with no delivery at the last audit) **completed at 2026-11-06T00:14:39 after
|
||||
12 h 13 m 54 s and delivered**; its 11-06 12:00 slot then took **7 h 47 m 56 s** and delivered too. The
|
||||
**one still-open page is unchanged:** [[incidents/2026-11-05-secrets-at-rest-in-mnemosyne-and-cron-prompts]].
|
||||
|
||||
## Recent Changes (2026-11-06, nightly audit; live probes 2026-11-06 00:1x-00:4x AWST)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user