diff --git a/current-state.md b/current-state.md index 4af48a7..b52e6e4 100644 --- a/current-state.md +++ b/current-state.md @@ -3,8 +3,8 @@ title: Current State type: concept status: active created: 2026-07-22 -updated: 2026-10-07 -verified_on: 2026-10-07 +updated: 2026-10-08 +verified_on: 2026-10-08 review_after: 2026-10-31 confidence: high tags: [system] @@ -12,7 +12,14 @@ sources: [live-config, live-cron-list, live-hermes-version] --- # Current State -Updated: 2026-10-07 +Updated: 2026-10-08 + +## Recent Changes (2026-10-08) +- **Cron fleet green for a third consecutive day — 0 failing.** Job count unchanged at 30 (27 enabled / 3 disabled) and all 30 `failure_streak` counters still read 0. Window tally (`executions.db`, 2026-10-07 00:00 → 2026-10-08 00:00 AWST): **176 `completed`, 1 `failed`, 1 `running`** (the nightly audit itself). 13 sessions started in the window, all `source = cron` — **no interactive (non-cron) sessions**; 12 deliveries, **0 delivery failures**. +- **First detect → notify → auto-resolve incident cycle since the managed-runtime event (DHL, 2026-10-07 08:00 → 11:01).** `62619aa6bcb3` (DHL Tracking, 5×/day) failed its 08:00 run inside `ensure_browser()` with `TimeoutError: timed out` — Camofox briefly unresponsive on a browser HTTP call (`dhl_tracking_monitor.py:233`). The failure notice went out at 08:00:58 and incident `62619a_f485ccfd4a0d` (`failure_type: timeout`) **auto-resolved at 11:01:13** when the next 3-hourly run completed; the 14:00, 17:00 and 20:00 runs also succeeded. No action needed — and the incident machinery behaved exactly as designed, which it had not been tested on since 2026-09-30. See [[systems/scheduled-tasks]]. +- **The tracked parcel arrived — the DHL monitor is now redundant.** The 14:00 run on 2026-10-07 reported waybill `8052838382` as **`Delivered`**. `62619aa6bcb3` still runs 5×/day for no remaining benefit; flagged as a removal candidate rather than removed. +- **The stocktake job proved its repin for a second day.** `1a8e90ea2271` completed automatically at **2026-10-07 03:00** — 346 skills / 70 categories (unchanged), `raw/stocktake/freshness-20261007.txt` (62 stale, 21 fresh, 7 no-date of 90 pages), committed to Gitea as `8c63ef2`. Two script-level findings came out of the run: (1) the **Quartz rebuild step is a no-op** — no `quartz.config.ts` is present and `npx` is unavailable, so the published site is never refreshed; (2) `skills-index` **under-counts by one** — 347 `SKILL.md` files exist on disk while the page reports 346, because the generator de-dups the nested `productivity/obsidian-git-sync/obsidian-git-sync/` duplicate away. Both are now recorded as open issues in [[systems/scheduled-tasks]]. +- **Nothing else changed in the window.** The three stale pre-repin `last_status: error` jobs (`1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd`) remain — due Fri 08:00 / Mon 03:00 / 1st 09:00. The `custom:omni` stale-key references and the three bare-`telegram` targets are unchanged. No new incident page was required, and no page was removed or archived. ## Recent Changes (2026-10-07) - **Cron fleet green for a second consecutive day — 0 failing.** Job count unchanged at 30 (27 enabled / 3 disabled) and all 30 `failure_streak` counters read 0. Window tally (`executions.db`, 2026-10-06 00:00 → 2026-10-07 00:00 AWST): **176 `completed`, 0 `failed`, 1 `running`** (the nightly audit itself) — 177 rows. 14 sessions started in the window, all `source = cron`; **no interactive (non-cron) sessions**. **No new incident was opened.** The repin wrap-up closures at 00:10–00:12 on 2026-10-06 fall inside this window but were already recorded by the 2026-10-06 audit. diff --git a/index.md b/index.md index f97855c..9695b19 100644 --- a/index.md +++ b/index.md @@ -1,6 +1,6 @@ # Anthony's Systems Wiki > Operational knowledge for Anthony's agents, infrastructure, -> model routing and automation. Last updated: 2026-10-07 +> model routing and automation. Last updated: 2026-10-08 ## Start here - [[current-state]] — Current systems status @@ -28,8 +28,8 @@ ### Integrations - [[systems/messaging-integrations]] — Telegram, Discord, email, Slack, etc. -- [[systems/scheduled-tasks]] — 30 cron jobs (27 enabled, 3 disabled; **0 failing as of 2026-10-07** — 7 dead-model-pin jobs repinned 2026-10-05, 4 now verified by live run) -- [[systems/skills-index]] — Installed skills inventory (**346 skills**; auto-regenerated daily by the 03:00 stocktake job — verified 2026-10-06) +- [[systems/scheduled-tasks]] — 30 cron jobs (27 enabled, 3 disabled; **0 failing as of 2026-10-08** — 7 dead-model-pin jobs repinned 2026-10-05, 4 now verified by live run; one transient Camofox timeout 2026-10-07 auto-resolved) +- [[systems/skills-index]] — Installed skills inventory (**346 skills**; auto-regenerated daily by the 03:00 stocktake job — verified 2026-10-07. Note: 347 `SKILL.md` files exist on disk — the generator de-dups one nested duplicate, see [[systems/scheduled-tasks]]) ## Incidents - [[incidents/2026-10-05-dead-model-pin-repin]] — Seven cron jobs on dead model pins (`cliprox/terra`, `qwen3.5:397b`, `deepseek-v4-flash`, `gpt-5.6-terra`) repinned to routing aliases on `custom:omni-101`; failing set 7 → 0 (resolved 2026-10-05) diff --git a/log.md b/log.md index bdfd633..fba534a 100644 --- a/log.md +++ b/log.md @@ -375,3 +375,23 @@ Reviewed window: 2026-10-06 00:00 → 2026-10-07 00:00 AWST. **Vault mirror:** `systems/scheduled-tasks.md`, `systems/skills-index.md`, `index.md`, `current-state.md` and `log.md` mirrored into `/home/hermes/wiki/agent-estate-wiki/` for the iPad/Ignis view. No secrets written; credential-shaped values referenced by connection ID only. + +## [2026-10-08] update | Nightly wiki audit — first auto-resolved incident since 2026-09-30, two stocktake script findings + +Reviewed window: 2026-10-07 00:00 → 2026-10-08 00:00 AWST. + +**Updated** +- `systems/scheduled-tasks.md` — new **Re-verified 2026-10-08 AWST** bullet (30 jobs, 27 enabled / 3 disabled, **0 failing**; 176 `completed` / 1 `failed` / 1 `running`; 13 sessions, all cron; 12 deliveries, 0 delivery failures). Historic count progression extended. DHL job row now records the 08:00 Camofox timeout and the parcel's arrival. Stocktake row corrected — its Quartz step is inert, and the second consecutive automatic run is recorded. New **Resolved 2026-10-07** Known Issues section, plus two new **Open** items. The "Verified live, not assumed" bullet now covers the stocktake's consecutive runs. +- `index.md` — last-updated → 2026-10-08; cron-fleet line re-dated; skills-index line re-dated to 2026-10-07 and annotated with the 347-on-disk caveat. +- `current-state.md` — new Recent Changes (2026-10-08) block; frontmatter dates bumped. + +**Findings worth the ink** +- **The incident machinery had its first real workout since 2026-09-30 — and it worked.** `62619aa6bcb3` (DHL Tracking) failed its 08:00 run inside `ensure_browser()` with `TimeoutError: timed out` on a browser HTTP call, i.e. a momentarily unresponsive Camofox. Incident `62619a_f485ccfd4a0d` (`failure_type: timeout`) was opened 08:00:52, alerted 08:00:58 and **auto-resolved 11:01:13** when the 11:00 run completed; the 14:00, 17:00 and 20:00 runs also succeeded. Transient and self-recovering — no action needed. +- **The tracked parcel has arrived.** The 14:00 run reported waybill `8052838382` as `Delivered`, so the DHL monitor is now redundant and is flagged as a removal candidate rather than removed. +- **Stocktake ran automatically for the second consecutive day** (`1a8e90ea2271`, 2026-10-07 03:00, commit `8c63ef2`; 346 skills / 70 categories; freshness 62 stale / 21 fresh / 7 no-date of 90 pages) — the repin is verified across consecutive runs, not just once. +- **Two real defects surfaced from the stocktake's own log.** (1) Step 3 of `wiki-maintain.sh` — the Quartz rebuild — **is skipped on every run**: there is no `quartz.config.ts` and no `npx`, so `public/` is never refreshed and the published site silently ages. (2) `skills-index` **under-reports by one**: 347 `SKILL.md` files exist on disk but the page says 346, because the generator de-duplicates on skill basename within a category and so swallows the nested duplicate `productivity/obsidian-git-sync/obsidian-git-sync/SKILL.md` (the other duplicated basename, `hermes-api-vault`, survives because its two copies sit in different categories). Both are now Open items in [[systems/scheduled-tasks]]. +- **Unchanged, still carrying a stale `last_status: error`:** `1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd` — due Fri 08:00 / Mon 03:00 / 1st 09:00. The three bare-`telegram` targets and the `custom:omni` stale-key references are also unchanged. + +**Vault mirror:** `systems/scheduled-tasks.md`, `index.md`, `current-state.md` and `log.md` mirrored into `/home/hermes/wiki/agent-estate-wiki/` for the iPad/Ignis view. + +No secrets written; credential-shaped values referenced by job ID and connection ID only. diff --git a/systems/scheduled-tasks.md b/systems/scheduled-tasks.md index 06ed379..7af5506 100644 --- a/systems/scheduled-tasks.md +++ b/systems/scheduled-tasks.md @@ -3,8 +3,8 @@ title: Scheduled Tasks type: system status: active created: 2026-07-22 -updated: 2026-10-07 -verified_on: 2026-10-07 +updated: 2026-10-08 +verified_on: 2026-10-08 confidence: high tags: [cron, scheduled, automation] sources: [live-cron-list, live-jobs-json, cron-executions-db] @@ -18,12 +18,13 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db] - Historical (2026-10-01 → 2026-10-05), now all remediated: `dca8482e4f76` 403 `MODEL_NOT_IN_PLAN` ×7; `1a8e90ea2271` HTTP 410 ×11 (retired `qwen3.5:397b`); `fad5a86979d4` HTTP 400 ×3; `0e753e9720fd` HTTP 400 ×1; `06a0a1c8704f`, `1b23c042e17d` and `79f9bb41ecf5` HTTP 401 (the `cliprox/terra` connection auth expiry, settled 3/3 on 2026-10-04). - **Failing (delivery only):** 1 — `066bb5191280`: the report itself generated; the send failed against the DM thread (bare `telegram` target, 2026-10-01). Unchanged this window. - **Repin applied 2026-10-05:** `free-stack` ×6 (`06a0a1c8704f`, `1b23c042e17d`, `79f9bb41ecf5`, `1a8e90ea2271`, `fad5a86979d4`, `0e753e9720fd`) and `smart-route` ×1 (`dca8482e4f76`), all on `custom:omni-101`. Routing aliases were used deliberately: the fault was raw-pin brittleness, so re-pinning to another raw model name would have re-created it. `free-stack` is in service again after appearing on the broken-combo list as recently as 2026-08-14 — it was re-verified before being handed six jobs. -- **Verified live, not assumed:** `79f9bb41ecf5` succeeded on a manual run 2026-10-05 23:39 and `06a0a1c8704f` at 23:45 (both incidents auto-resolved); `dca8482e4f76` completed 23:56 and delivered to `telegram:-1004321904721:1030` (incident resolved 2026-10-06 00:05:07); and `1a8e90ea2271` completed automatically on **2026-10-06 03:00**. Four of the seven are therefore verified by live run. The remaining three (`1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd`) are repinned but not yet re-run — they are due on their normal schedules. +- **Verified live, not assumed:** `79f9bb41ecf5` succeeded on a manual run 2026-10-05 23:39 and `06a0a1c8704f` at 23:45 (both incidents auto-resolved); `dca8482e4f76` completed 23:56 and delivered to `telegram:-1004321904721:1030` (incident resolved 2026-10-06 00:05:07); and `1a8e90ea2271` completed automatically on **2026-10-06 03:00**. Four of the seven are therefore verified by live run. The remaining three (`1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd`) are repinned but not yet re-run — they are due on their normal schedules. `1a8e90ea2271` has since run **twice more on schedule** (2026-10-06 03:00 → `33f4e95`; 2026-10-07 03:00 → `8c63ef2`), so the stocktake repin is verified across consecutive automatic runs. - **Streak counting note:** the `N failures in a row` figure in `hermes cron list` comes from the `failure_streak` counter in `jobs.json`. `executions.db` retains only rows from **2026-09-30 onward**, so DB-based recounts undercount older streaks. Live counters as of 2026-10-06: **all 0**. - **Re-verified 2026-10-06 AWST:** job count unchanged at 30 (27 enabled / 3 disabled). `066bb5191280` and `0e753e9720fd` still carry the bare-`telegram` target warning (`configured thread_id 34795 for telegram:1793951355 was not found`). - **Re-verified 2026-10-07 AWST:** job count unchanged at 30 (27 enabled / 3 disabled); **0 failing** — all 30 `failure_streak` counters read 0. Audit window 2026-10-06 00:00 → 2026-10-07 00:00: 176 `completed`, 0 `failed`, 1 `running` (the audit itself); 14 sessions, all `cron` — **no interactive (non-cron) sessions**. The stocktake job `1a8e90ea2271` ran on schedule at 03:00 and regenerated [[systems/skills-index]] (344 → 346 skills) plus `raw/stocktake/freshness-20261006.txt`, committing to Gitea as `33f4e95` — the first fully automatic stocktake since its repin. Only three jobs now carry a stale pre-repin `last_status: error` (`1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd`). - **A stale notice, not a live outage, drove this review.** The interactive session on 2026-10-05 23:16 began because Anthony replied to a `85edf6772bca` failure notice that had been sitting in topic `346` since the **2026-09-29 07:00** run. That job completed on every run from 2026-09-30 17:53 onward and its three incidents were already `resolved` on 2026-09-30 17:55:13. The notice looked current only because its wording matches the live 2026-09-28 managed-runtime incident. Check `executions.started_at` (and the `cron_incidents` state) before acting on a notice. -- Historic count progression: 26 (2026-09-19) → 30 (2026-09-25) → 30 (2026-09-30) → 30 (2026-10-01) → 30 (2026-10-02) → 30 (2026-10-03) → 30 (2026-10-05) → 30, **0 failing** (2026-10-06) → 30, **0 failing** (2026-10-07) +- **Re-verified 2026-10-08 AWST (window 2026-10-07 00:00 → 2026-10-08 00:00):** job count unchanged at 30 (27 enabled / 3 disabled); **0 failing** — every `failure_streak` reads 0. Window tally (`executions.db`): **176 `completed`, 1 `failed`, 1 `running`** (the audit itself). The single failure is `62619aa6bcb3` (DHL Tracking, 08:00) — a Camofox HTTP read timeout (`ensure_browser()` → `TimeoutError: timed out`), notified 08:00:58 and **auto-resolved 11:01:13** when the next 3-hourly run succeeded (incident `62619a_f485ccfd4a0d`). 13 sessions started in the window, all `source = cron` — **no interactive sessions**; 12 deliveries, **0 delivery failures**. The same three stale pre-repin `last_status: error` jobs remain (`1b23c042e17d`, `fad5a86979d4`, `0e753e9720fd`); the stocktake job `1a8e90ea2271` ran on schedule at 03:00 for the second consecutive day. +- Historic count progression: 26 (2026-09-19) → 30 (2026-09-25) → 30 (2026-09-30) → 30 (2026-10-01) → 30 (2026-10-02) → 30 (2026-10-03) → 30 (2026-10-05) → 30, **0 failing** (2026-10-06) → 30, **0 failing** (2026-10-07) → 30, **0 failing** (2026-10-08) ## Job Table @@ -33,7 +34,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db] | AI Newsletter Curation for Patrick (Weekly) | `06a0a1c8704f` | `0 10 * * 4` | telegram:-1004321904721:346 | — | — | ✅ **Repinned 2026-10-05** → `custom:omni-101` / `free-stack` (was `custom:Omniroute:20128` / `cliprox/terra`, HTTP 401). Verified by a live manual run at 23:45 — the incident auto-resolved. Live `last_status: ok`, streak 0 | | Daily Birthday Checker | `a432898b55a4` | `30 7 * * *` | telegram:1793951355 | — | `birthday_check.py` | | | Daily Marketing Job Radar | `dca8482e4f76` | `0 7 * * 1-5` | telegram:-1004321904721:1030 | job-search-automation | — | ✅ **Repinned 2026-10-05** → `custom:omni-101` / `smart-route` (was `command-code` / `gpt-5.6-terra`, 403 `MODEL_NOT_IN_PLAN` ×7). A manual run completed at 23:56 and delivered to topic 1030 — first successful run since 2026-09-25. Live `last_status: ok`, streak 0 | -| DHL Tracking - Parcel 8052838382 | `62619aa6bcb3` | `0 8,11,14,17,20 * * *` | telegram:1793951355 | — | `dhl_tracking_monitor.py` | no-agent; added 2026-09-24 | +| DHL Tracking - Parcel 8052838382 | `62619aa6bcb3` | `0 8,11,14,17,20 * * *` | telegram:1793951355 | — | `dhl_tracking_monitor.py` | no-agent; added 2026-09-24. ⚠ The 2026-10-07 08:00 run failed on a Camofox read timeout (`ensure_browser()` → `TimeoutError: timed out`) — incident `62619a_f485ccfd4a0d`, auto-resolved 11:01:13 when the 11:00 run succeeded. The 14:00 run reported waybill 8052838382 **Delivered** — the parcel has arrived, so the monitor is now redundant | | Enhanced Weekend Adventure Planner | `1b23c042e17d` | `0 8 * * 5` | telegram:-1004321904721:102 | — | — | 2026-09-30 delivery-target fix **confirmed working** (delivered 2026-10-02, flag cleared). ✅ **Repinned 2026-10-05** → `custom:omni-101` / `free-stack` (was the failed `cliprox/terra` pin, HTTP 401 ×1). Not yet re-run — carries a stale `last_status: error` until Fri 08:00; streak 0 | | Evening Stand-Down Briefing | `e530c89f2be0` | `45 18 * * 1-5` | telegram:-1004321904721 | — | `evening_standdown.py` | no-agent | | Facebook Birthday Monthly Scrape | `0e753e9720fd` | `0 9 1 * *` | telegram | — | — | ✅ **Repinned 2026-10-05** → `custom:omni-101` / `free-stack` (was the catalog-missing `nvidia/deepseek-ai/deepseek-v4-flash` pin, HTTP 400 ×1). Streak 0; stale `last_status: error` until the 1st. ⚠ Bare `telegram` still targets the home DM — retarget to a group topic | @@ -57,7 +58,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db] | Weekly Review | `79f9bb41ecf5` | `0 18 * * 0` | origin | weekly-review | — | ⚠ Failed 2026-10-04 18:00 on the `cliprox/terra` pin (HTTP 401), which confirmed that pin 3/3 broken. ✅ **Repinned 2026-10-05 23:39** → `custom:omni-101` / `free-stack` and **verified by a live manual run**; the incident auto-resolved. Now `last_status: ok`, streak 0 | | weekly-save-restricted-bot-restart | `a688ed7b7ab0` | `0 3 * * 0` | local | — | `restart-save-restricted-bot.sh` | no-agent; script restored 2026-09-30 | | Wiki Audit — Nightly | `cbfe434a3b5e` | `59 23 * * *` | local | task-approach | — | Reviews past 24h for wiki changes | -| wiki-stocktake-maintenance | `1a8e90ea2271` | `0 3 * * *` | local | — | `wiki-maintain.sh` | ✅ **Repinned 2026-10-05** → `custom:omni-101` / `free-stack` (was the retired `qwen3.5:397b`, HTTP 410 ×11). ✅ **Verified by live run 2026-10-06 03:00** — first success since 2026-09-25; regenerated skills-index (344 → 346 skills) and the freshness audit, committed to Gitea as `33f4e95`. Live `last_status: ok`, streak 0. Regenerates skills-index, freshness audit, rebuilds Quartz, commits to Gitea — **the script itself was never broken** | +| wiki-stocktake-maintenance | `1a8e90ea2271` | `0 3 * * *` | local | — | `wiki-maintain.sh` | ✅ **Repinned 2026-10-05** → `custom:omni-101` / `free-stack` (was the retired `qwen3.5:397b`, HTTP 410 ×11). ✅ **Verified by live run 2026-10-06 03:00** — first success since 2026-09-25; regenerated skills-index (344 → 346 skills) and the freshness audit, committed to Gitea as `33f4e95`. Live `last_status: ok`, streak 0. Regenerates skills-index, freshness audit and commits to Gitea. ⚠ **The Quartz rebuild step is a no-op in practice** — no `quartz.config.ts` exists and `npx` is unavailable, so the static site in `public/` is never refreshed (see Known Issues). Ran automatically again 2026-10-07 03:00 (346 skills, 62 stale / 21 fresh of 90 pages, commit `8c63ef2`) — a **second consecutive automatic success**, verified. The script itself was never broken | | 🦝 Hermes Config Backup to Gitea (Daily) | `e27da6c34bbb` | `0 11 * * *` | origin | — | `hermes-config-backup.sh` | no-agent | ## Removed Jobs @@ -76,6 +77,10 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db] ## Known Issues +### Resolved 2026-10-07 +- **DHL Tracking `62619aa6bcb3` — single Camofox timeout, self-recovered (2026-10-07 08:00 → 11:01):** the 08:00 run died in `ensure_browser()` with `TimeoutError: timed out` on a browser HTTP call (`dhl_tracking_monitor.py:233`), i.e. Camofox was momentarily unresponsive. The failure notice was sent at 08:00:58 and the incident (`62619a_f485ccfd4a0d`, `failure_type: timeout`) **auto-resolved at 11:01:13** when the 11:00 run completed; the 14:00, 17:00 and 20:00 runs also succeeded. **This is the first detect → notify → auto-resolve incident cycle since the 2026-09-30 managed-runtime event** — the incident machinery itself worked as designed. No action needed; a repeat would justify a Camofox health probe inside the script. +- **The tracked parcel arrived.** The 14:00 run on 2026-10-07 reported waybill **8052838382 `Delivered`** (Kensington → Perth). The monitor has therefore achieved its purpose and runs 5×/day for no remaining benefit — a candidate for removal, flagged rather than removed (the job is Anthony's to retire). + ### Resolved 2026-10-05 - **Dead model pins across 7 jobs — repinned to routing aliases (resolved 2026-10-05 23:16 → 2026-10-06 00:05):** Seven jobs were each pinned to a raw model name that had died upstream — `cliprox/terra` (HTTP 401, connection auth expired, 3 jobs), `ollama-cloud/qwen3.5:397b` (HTTP 410, retired 2026-09-25), `nvidia/deepseek-ai/deepseek-v4-flash` (HTTP 400, dropped from the live catalog, 2 jobs) and `command-code/gpt-5.6-terra` (HTTP 403 `MODEL_NOT_IN_PLAN`) — across 2026-10-01 → 2026-10-05. All seven were repinned to provider `custom:omni-101` using **routing aliases** — `free-stack` ×6, `smart-route` ×1 — deliberately, because the fault was raw-pin brittleness and re-pinning to another raw name would have re-created it. `failure_streak` is now 0 on all 30 jobs, down from 7 failing. Three were verified by live run: `79f9bb41ecf5` (23:39), `06a0a1c8704f` (23:45) and `dca8482e4f76` (23:56, delivered to topic 1030 — its first success since 2026-09-25). The remaining four are repinned but not yet re-run; the stale `last_status: error` field clears on each one's next scheduled run. See [[incidents/2026-10-05-dead-model-pin-repin]]. - The `cliprox/terra` incident is **closed for cron purposes**: no enabled job references that pin any more. The underlying proxy connection was never re-authenticated — the jobs were moved off it instead. See [[incidents/2026-10-01-cliproxy-connection-auth-expired]]. @@ -91,6 +96,8 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db] - **Stale `delivery_failed` flag on `1b23c042e17d` cleared (confirmed 2026-10-02):** the 2026-09-30 retarget held — the job's next run delivered to topic `102`. ### Open +- **Stocktake's Quartz rebuild step is dead code (NEW 2026-10-07):** `wiki-maintain.sh` documents a third step — rebuild the Quartz site — but the 2026-10-07 03:00 run logged it as skipped: no `quartz.config.ts` is present and `npx` is unavailable, so `public/` has not been refreshed. The skills-index and freshness artifacts are regenerated and committed correctly; only the published-HTML step is inert. Either restore the Quartz toolchain or drop the step from the script and the job description. +- **`skills-index` under-counts by one skill (NEW 2026-10-07):** live `find ~/.hermes/skills -name SKILL.md -type f` returns **347**, but the generated page says **346** (70 categories). The generator de-dups on skill basename within a category, so the nested duplicate `productivity/obsidian-git-sync/obsidian-git-sync/SKILL.md` collapses into its parent and is silently dropped. The other duplicated basename (`hermes-api-vault`, at the root and under `devops/`) survives because the two land in different categories. Remove the nested duplicate directory, or key the generator on path rather than basename. - **A delivered notice is not a successful job:** `cliprox/terra` failures recorded `delivery_outcome: delivered` because the *failure notice* is itself a deliverable. Read the execution `status` column, not the delivery outcome, when judging whether a job produced its output. - **Stale provider key `custom:omni` (1 enabled + 1 disabled):** the provider was removed from `custom_providers` (live set on 2026-10-01: `litellm`, `guacano`, `Omniroute:20128`, `command-code`, `omni-101`). Re-verified 2026-10-06: `e530c89f2be0` (enabled) and `d40085da631c` (disabled) still name it; both run `smart-route`, and the default `custom` provider resolves to the same base URL (`http://100.78.63.20:20128`), so no failure has been observed — but the references should be normalised to `custom:omni-101`. The 2026-10-05 repin moved seven jobs to an explicit `custom:omni-101`, leaving these two as the only remaining `custom:omni` references. - **Bare `telegram` delivery targets (3 jobs) — demonstrably breaking sends:** `0e753e9720fd`, `fad5a86979d4`, `066bb5191280` deliver to the home DM rather than a group topic. As of 2026-10-01 this is no longer theoretical: `066bb5191280` (02:11) and `0e753e9720fd` (09:01) both recorded `delivery_outcome: failed` with `configured thread_id 34795 for telegram:1793951355 was not found`. Work generated correctly; the delivery did not. Retarget all three to `telegram:-1004321904721:`. See [[systems/messaging-integrations]].