wiki audit 2026-10-04: CT450 v0.21.5 update incident, mnemosyne vector-recall FTS-only repair + coverage refresh, managed runtimes 10->3, doctor vec0 caveat

This commit is contained in:
Tony0410
2026-10-04 00:15:35 +08:00
parent b9f69d3f75
commit 78048bdf0b
14 changed files with 241 additions and 49 deletions
+4 -3
View File
@@ -3,8 +3,8 @@ title: Scheduled Tasks
type: system
status: active
created: 2026-07-22
updated: 2026-10-03
verified_on: 2026-10-03
updated: 2026-10-04
verified_on: 2026-10-04
confidence: high
tags: [cron, scheduled, automation]
sources: [live-cron-list, live-jobs-json, cron-executions-db]
@@ -25,6 +25,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db]
- **Resolved since the last entry:** 1 — `1b23c042e17d`'s stale `delivery_failed` flag cleared exactly as predicted. Its 2026-10-02 run delivered to topic `102` (`delivery_outcome: delivered`); it then failed on the model pin, above.
- **Shared model pin at risk:** 3 enabled jobs run `cliprox/terra` — `06a0a1c8704f`, `1b23c042e17d`, `79f9bb41ecf5`. **Two of the three have now failed on that pin.** The third (`79f9bb41ecf5`) last ran 2026-09-27 — before the credential lapsed — and is next due 2026-10-04 18:00. See [[incidents/2026-10-01-cliproxy-connection-auth-expired]].
- **Streak recount (2026-10-03):** recounted directly from `~/.hermes/cron/executions.db`. `dca8482e4f76` 6 → 7 (one more daily run since), and `1a8e90ea2271` 8 → **7** — the ×8 carried in the 2026-10-02 entry was an overcount: that job has exactly 7 recorded runs and all 7 failed.
- **Re-verified 2026-10-04 00:12 AWST — no change:** still 30 jobs (27 enabled / 3 disabled) and the same **6** failing jobs; no new failure event landed inside the audit window. `79f9bb41ecf5` (Weekly Review, the last untested `cliprox/terra` job) drops at **2026-10-04 18:00 AWST** — its result settles whether that pin is 3/3 broken. `066bb5191280` and `0e753e9720fd` still carry the bare-`telegram` target warning (`configured thread_id 34795 for telegram:1793951355 was not found; delivered without thread_id`), confirmed live in `hermes cron list`.
- Historic count progression: 26 (2026-09-19) → 30 (2026-09-25) → 30 (2026-09-30) → 30 (2026-10-01) → 30 (2026-10-02) → 30 (2026-10-03)
## Job Table
@@ -87,7 +88,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db]
### Open
- **Daily Marketing Job Radar — 403 `MODEL_NOT_IN_PLAN` (2026-09-26 → 10-02, streak 7):** Job `dca8482e4f76` is pinned to `custom:omni-101` / `gpt-5.6-terra`; the upstream (`command-code`) rejects it with `GPT-5.6 Terra available in Pro and above plans`. Anthony's primary job pipeline — needs a model or provider change; it cannot self-recover. See [[runbooks/job-radar]].
- **`cliprox/terra` provider connection auth expired — 2 of 3 jobs confirmed (streak 1 each):** Jobs `06a0a1c8704f` (2026-10-01) and `1b23c042e17d` (2026-10-02) both failed with `HTTP 401: [openai-compatible-chat-2ea20f63-…] All 1 connection(s) authentication expired — please reconnect in the dashboard` (provider `custom:Omniroute:20128`, model `cliprox/terra`). Failure notices reached their topics; no output was produced. This is a *credential/connection* problem at the proxy, not a missing model — it will not self-recover until the connection is re-authenticated. `79f9bb41ecf5` is the remaining untested job on this pin (next run 2026-10-04 18:00). Re-authenticate the OmniRoute connection before repinning anything. See [[incidents/2026-10-01-cliproxy-connection-auth-expired]].
- **`cliprox/terra` provider connection auth expired — 2 of 3 jobs confirmed (streak 1 each):** Jobs `06a0a1c8704f` (2026-10-01) and `1b23c042e17d` (2026-10-02) both failed with `HTTP 401: [openai-compatible-chat-2ea20f63-…] All 1 connection(s) authentication expired — please reconnect in the dashboard` (provider `custom:Omniroute:20128`, model `cliprox/terra`). Failure notices reached their topics; no output was produced. This is a *credential/connection* problem at the proxy, not a missing model — it will not self-recover until the connection is re-authenticated. `79f9bb41ecf5` is the remaining untested job on this pin (next run 2026-10-04 18:00 — re-check on 2026-10-05). Re-authenticate the OmniRoute connection before repinning anything. See [[incidents/2026-10-01-cliproxy-connection-auth-expired]].
- **A delivered notice is not a successful job:** both `cliprox/terra` failures recorded `delivery_outcome: delivered` because the *failure notice* is itself a deliverable. Read the execution `status` column, not the delivery outcome, when judging whether a job produced its output.
- **wiki-stocktake-maintenance — HTTP 410, model retired (streak 7):** Job `1a8e90ea2271` is pinned to `ollama-cloud / qwen3.5:397b`, which Ollama retired on 2026-09-25 (`qwen3.5:397b was retired at 2026-09-25 00:00:00 -0700 PDT`). It cannot self-recover — it needs a new model pin. **The job's script is fine:** `wiki-maintain.sh` was run by hand on 2026-10-03 and completed cleanly, regenerating [[systems/skills-index]] (344 skills / 70 categories) and refreshing the freshness audit. Only the scheduled wrapper is broken — so every non-model step of the daily stocktake needs a manual run until it is repinned.
- **Facebook Birthday scrapes — HTTP 400, model missing from catalog (streak 3 / 1):** Job `fad5a86979d4` (weekly) is pinned to `nvidia/deepseek-ai/deepseek-v4-flash`; the provider reports the model is `not available in the active live catalog for provider 'nvidia'`. **Confirmed as a shared-pin issue on 2026-10-01**, when sibling job `0e753e9720fd` (monthly) failed on the same pin at its own 09:00 run. Safest durable fix is repinning both to a routing alias (`smart-route`) instead of a raw model name.