wiki audit 2026-11-20: three-window catch-up (11-17/11-18/11-19, both prior audit slots died)

- new: incidents/2026-11-18-wiki-audit-credit-outage (audit failed 11-18 + 11-19, deliveries suppressed, incident row alerted_at=None)
- new: incidents/2026-11-18-mnemosyne-watchdog-conflict-threshold-storm (115 consecutive slots, 59 unresolved conflicts vs threshold 50)
- corrected: incidents/2026-11-14-mnemosyne-capture-stall + systems/mnemosyne — freeze ENDED 2026-11-18 19:25 AWST (105h52m, no intervention); 'capture dead' reading was conversational silence (every interactive session captured)
- systems/scheduled-tasks: new Count bullet 11-17 173(171/2) / 11-18 172(153/19) / 11-19 176(78/98); fleet 29 (26/3); non-clean last_status 1->4; unclosed incidents 5->4; deliveries 24 (23 delivered / 1 failed)
- index.md feed + incidents + systems lines; log.md dated entry; current-state.md new section; frontmatter dates -> 2026-11-20
This commit is contained in:
Tony0410
2026-11-20 01:02:44 +08:00
parent 16f2c878c0
commit 226767f3ec
8 changed files with 310 additions and 22 deletions
+4 -3
View File
@@ -3,9 +3,9 @@ title: Scheduled Tasks
type: system
status: active
created: 2026-07-22
updated: 2026-11-17
verified_on: 2026-11-17
review_after: 2026-11-18
updated: 2026-11-20
verified_on: 2026-11-20
review_after: 2026-11-21
confidence: high
tags: [cron, scheduled, automation]
sources: [live-cron-list, live-jobs-json, cron-executions-db, live-executions-db, live-deliveries-db, live-sessions-db]
@@ -14,6 +14,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db, live-executions-db
# Scheduled Tasks
## Count
- **Current as of 2026-11-20 00:5x (three-window audit — 2026-11-17, 11-18 and 11-19; the audits slotted at 11-18 00:00 and 11-19 00:00 both died, so 11-17 and 11-18 had never been audited; live `jobs.json` + `executions.db` + `deliveries.db` + `cron_incidents` + `mnemosyne.db`).** Ledger: **11-17 — 173 rows, 171 `completed` / 2 `failed`** (both `ea79a2107e36` *Mnemosyne Health Monitor*, `00:01` and `12:01`); **11-18 — 172 rows, 153 / 19** (this audit `00:00:08 → 00:01:43`, the monitor twice, and **16** *Mnemosyne Watchdog* slots from `20:01:06`); **11-19 — 176 rows, 78 / 98** — **the worst day in the retained window** (audit `00:00:16 → 00:03:02`, monitor `12:01`, **96** watchdog slots); **11-20 so far — 8 rows, 3 `completed` / 4 `failed` / 1 `running`** (this audit). Deliveries since 11-18: **24 — 23 `delivered` (9 `for_failure: 1`) / 1 `failed`** (`2026-11-19 08:34:13`, a *content* delivery: `live adapter send failed: Timed out; live adapter delivery to telegram`). Fleet **29 jobs (26 enabled / 3 disabled)** — a **twelfth consecutive window at the same count** (following the 11th, recorded on 11-17; the 11-18 and 11-19 windows went unrecorded because both audits died) — but non-clean `last_status` **1 → 4**: `cbfe434a3b5e` Wiki Audit `error`, `failure_streak` **2** (last clean run 11-17), `96eb95ce12f2` Watchdog `error`, streak **115**, `ea79a2107e36` Health Monitor `error`, streak **2**, `08eb86f02d37` Gmail Action Digest `delivery_failed`, streak **0**. **Two of the three jobs that keep the fleet safe are themselves red — and the 03:00 stocktake is not:** `1a8e90ea2271` ran `completed` on 11-17 (**11 m 35 s**), 11-18 (**23 m 07 s**) and 11-19 (**24 m 38 s**), so the estate's structural maintenance kept running while both alarm paths failed. `cron_incidents` unclosed **5 → 4**, none closed automatically: `cbfe43_fa12cca1ad4b` (*auth*, 2026-11-18 00:01:42, **`alerted_at: None`** — the audit's silent skip), `96eb95_98ade422d195` (*script*, 2026-11-18 20:01:15 → last_seen 2026-11-20 00:31:16), `ea79a2_f2be2db3bfce` (2026-11-19 12:01:58) and `ea79a2_aae4d914f3c9` (2026-11-20 00:03:36) — both monitor rows on the same `STALE MEMORY PIPELINE` metric that is now measuring a *quiet estate* rather than a broken capture (see [[systems/mnemosyne]]). Cadence and pins unchanged; this audit still runs `custom:omni-101` / `deepseek-paid-smart`, and the Mnemosyne Watchdog's model/provider fields remain unset (inherits global). Full detail: [[incidents/2026-11-18-wiki-audit-credit-outage]], [[incidents/2026-11-18-mnemosyne-watchdog-conflict-threshold-storm]].
- **Current as of 2026-11-17 00:27 (window 2026-11-16 00:00 -> 23:59; nightly audit).** Still **29
jobs** (26 enabled, 3 disabled) — 11th consecutive window unchanged. 11-16 ledger: **175 rows — 173
`completed`, 2 `failed`**; both failures are `ea79a2107e36` *Mnemosyne Health Monitor*