--- title: Scheduled Tasks type: system status: active created: 2026-07-22 updated: 2026-10-01 verified_on: 2026-10-01 confidence: high tags: [cron, scheduled, automation] sources: [live-cron-list, live-jobs-json] --- # Scheduled Tasks ## Count - **Total jobs:** 30 — 27 enabled, 3 disabled (verified from live `~/.hermes/cron/jobs.json`, 2026-10-01) - **Failing:** 3 — `dca8482e4f76` (403 `MODEL_NOT_IN_PLAN`, streak 5), `1a8e90ea2271` (HTTP 410, model retired upstream, streak 7), `fad5a86979d4` (HTTP 400, model absent from the NVIDIA live catalog, streak 3) - **Stale status flag:** 1 — `1b23c042e17d` still reads `delivery_failed` from its pre-fix run. The delivery target was corrected on 2026-09-30 and its next run clears the flag. - Historic count progression: 26 (2026-09-19) → 30 (2026-09-25) → 30 (2026-09-30) → 30 (2026-10-01) ## Job Table | Name | ID | Schedule | Delivery | Skills | Script | Notes | |---|---|---|---|---|---|---| | AgentMail Rhino Inbox Poll | `0a7f933d06e2` | `* * * * *` | origin | terminal | `agentmail-poll.py` | **disabled** since 2026-07-27 (legacy) | | AI Newsletter Curation for Patrick (Weekly) | `06a0a1c8704f` | `0 10 * * 4` | telegram:-1004321904721:346 | — | — | | | Daily Birthday Checker | `a432898b55a4` | `30 7 * * *` | telegram:1793951355 | — | `birthday_check.py` | | | Daily Marketing Job Radar | `dca8482e4f76` | `0 7 * * 1-5` | telegram:-1004321904721:1030 | job-search-automation | — | ⚠ 403 `MODEL_NOT_IN_PLAN` ×5 — see Known Issues | | DHL Tracking - Parcel 8052838382 | `62619aa6bcb3` | `0 8,11,14,17,20 * * *` | telegram:1793951355 | — | `dhl_tracking_monitor.py` | no-agent; added 2026-09-24 | | Enhanced Weekend Adventure Planner | `1b23c042e17d` | `0 8 * * 5` | telegram:-1004321904721:102 | — | — | delivery target corrected 2026-09-30 (was bare `telegram` → home DM); `delivery_failed` flag is stale | | Evening Stand-Down Briefing | `e530c89f2be0` | `45 18 * * 1-5` | telegram:-1004321904721 | — | `evening_standdown.py` | no-agent | | Facebook Birthday Monthly Scrape | `0e753e9720fd` | `0 9 1 * *` | telegram | — | — | bare `telegram` → home DM; should be retargeted to a group topic | | Facebook Birthday Weekly Registry Sync | `fad5a86979d4` | `0 3 * * 1` | telegram | — | `birthday_check.py` | ⚠ HTTP 400 ×3 — `nvidia/deepseek-ai/deepseek-v4-flash` not in the active NVIDIA catalog; bare `telegram` target should move to a group topic | | FreshRSS Midday News Briefing | `ecc1fc13655a` | `0 12 * * *` | telegram:-1004321904721:346 | — | `freshrss_intelligence_v2.py` | | | GitHub Trending Radar — Weekly Roundup | `c0a9fa314542` | `0 9 * * 1` | telegram:1793951355 | — | — | ⚠ Telegram bot token in `github-trending-radar/config.json` revoked (`getMe` → 401); digest builds, push fails | | Gmail Action Digest | `08eb86f02d37` | `30 8 * * *` | telegram:-1004321904721:notifications | — | `gmail_action_digest_v3.py` | | | Important-mail monitor | `d40085da631c` | `every 30m` | telegram:1793951355 | — | `_cron_silent_monitor.py` | **disabled** since 2026-08-29 (Gmail read-state incident) | | Job Tracker Dashboard Refresh | `2fd50c87b0d0` | `0 */2 * * *` | telegram:-1004321904721:1030 | — | `dashboard_watchdog.py` | no-agent | | Mnemosyne Auto-Consolidate | `0c771463c3cd` | `every 30m` | local | — | `consolidate_mnemosyne.sh` | no-agent | | Mnemosyne DB Backup | `e5ed215fc91c` | `0 3 * * 0` | telegram:1793951355 | — | `mnemosyne-backup.sh` | no-agent | | Mnemosyne Embedding Coverage Monitor | `b63ed5817212` | `0 */6 * * *` | telegram:-1004321904721:102 | — | `check_embedding_coverage.py` | Alerts if coverage < 95% | | Mnemosyne Health Monitor | `ea79a2107e36` | `0 */12 * * *` | telegram:-1004321904721:102 | mnemosyne-operations | — | Reports only on exception | | Mnemosyne Watchdog | `96eb95ce12f2` | `*/15 * * * *` | telegram:-1004321904721:102 | — | `mnemosyne-watchdog.py` | no-agent, silent ≤10 conflicts; **also runs `hermes-runtime-deps-guard.sh`** every 15 min (added 2026-09-30) | | Mosquito Tracker | `1b31187397a5` | `0 7,15,22 * * *` | telegram:-1004321904721:273 | — | `mosquito_tracker.py` | Excludes 23:00–07:00 | | Perth New Restaurant & Cafe Openings (Weekly) | `c8d52904e1d6` | `0 18 * * 4` | telegram:-1004321904721:102 | — | — | delivery target corrected 2026-09-30; verified `delivered`, status `ok` | | Political News Digest (Mon/Thu) | `0e519874d280` | `0 8 * * 1,4` | telegram:-1004321904721:346 | — | — | | | Rhino AgentMail Inbox Watcher | `f77b7e46bd09` | `every 5m` | origin | — | `agentmail-rhino-watch.sh` | **disabled** since 2026-07-27 (legacy) | | Tech & AI Newsletter Digest (Analytical) | `85edf6772bca` | `0 7 * * *` | telegram:-1004321904721:346 | — | — | | | Uber Eats Monthly Spend Report | `066bb5191280` | `0 2 1 * *` | telegram | camofox-browser-automation, browser-backend-verification | — | bare `telegram` → home DM; carries a stale DM-thread delivery error | | Weekly Review | `79f9bb41ecf5` | `0 18 * * 0` | origin | weekly-review | — | | | weekly-save-restricted-bot-restart | `a688ed7b7ab0` | `0 3 * * 0` | local | — | `restart-save-restricted-bot.sh` | no-agent; script restored 2026-09-30 | | Wiki Audit — Nightly | `cbfe434a3b5e` | `59 23 * * *` | local | task-approach | — | Reviews past 24h for wiki changes | | wiki-stocktake-maintenance | `1a8e90ea2271` | `0 3 * * *` | local | — | `wiki-maintain.sh` | ⚠ HTTP 410 ×7 — `qwen3.5:397b` retired 2026-09-25; needs repinning. Regenerates skills-index, freshness audit, rebuilds Quartz, commits to Gitea | | 🦝 Hermes Config Backup to Gitea (Daily) | `e27da6c34bbb` | `0 11 * * *` | origin | — | `hermes-config-backup.sh` | no-agent | ## Removed Jobs | Name | Removed | Reason | |---|---|---| | Workday start reminder | 2026-08-26 | No longer present in live cron list | | CT 221 Miner Watchdog | 2026-08-26 | Job `d5516d0a5147` removed; CT221 status unknown/inspected | | Review Agent Estate Phase One | 2026-08-26 | One-time job; completed 2026-08-08, now cleaned up | ## Disabled Jobs (not removed) | Name | ID | Disabled since | Reason | |---|---|---|---| | Important-mail monitor | `d40085da631c` | 2026-08-29 | Paused during Gmail read-state incident. See [[incidents/2026-08-29-gmail-read-state]]. | | Rhino AgentMail Inbox Watcher | `f77b7e46bd09` | 2026-07-27 | Legacy AgentMail polling; superseded by newer patterns. | | AgentMail Rhino Inbox Poll | `0a7f933d06e2` | 2026-07-27 | Legacy AgentMail polling; superseded by newer patterns. | ## Known Issues ### Resolved 2026-09-30 - **Telegram thread delivery failures — root cause found and fixed:** jobs `fad5a86979d4`, `1b23c042e17d`, `c8d52904e1d6` reported `configured thread_id 34795 for telegram:1793951355 was not found` / `Message thread not found`. The target was wrong, not the topic: a **bare `deliver: "telegram"` resolves to the home DM (`1793951355`)**, where a group forum topic ID cannot exist — so the send hard-failed and the message was lost (an earlier "benign, delivered without thread" reading was wrong). `1b23c042e17d` and `c8d52904e1d6` were retargeted to `telegram:-1004321904721:102` and verified end-to-end. See [[systems/messaging-integrations]]. - **Managed-runtime dependency failure (2026-09-28 → 30):** Jobs failed with `Restart-safe cron worker dispatch failed … external worker exited before ownership acknowledgement (exit 1)`. Real cause was `ModuleNotFoundError: No module named 'dotenv'` / `'mnemosyne'` in uv-managed runtime venvs. Mnemosyne Watchdog failed 176 consecutive runs. Fixed by `hermes-runtime-deps-guard.sh` v2, run every 15 min by job `96eb95ce12f2`. All managed venvs verified. See [[incidents/2026-09-28-hermes-managed-runtime-deps]]. - **`weekly-save-restricted-bot-restart` missing script (2026-08-30):** Previously failed with "Script not found: `restart-save-restricted-bot.sh`". The script existed at `/home/hermes/scripts/` but cron only resolves inside the profile folder; copied in 2026-09-30 19:17. Job now `ok`, streak 0. - **Residual pre-fix dispatch errors cleared:** `cbfe434a3b5e` returned to `ok` on its next run as predicted. The `1a8e90ea2271` error of the same era also cleared — its current failure is a *different*, upstream problem (below). ### Open - **Daily Marketing Job Radar — 403 `MODEL_NOT_IN_PLAN` (2026-09-26 → 30, streak 5):** Job `dca8482e4f76` is pinned to `custom:omni-101` / `gpt-5.6-terra`; the upstream (`command-code`) rejects it with `GPT-5.6 Terra available in Pro and above plans`. Anthony's primary job pipeline — needs a model or provider change; it cannot self-recover. See [[runbooks/job-radar]]. - **wiki-stocktake-maintenance — HTTP 410, model retired (streak 7):** Job `1a8e90ea2271` is pinned to `ollama-cloud / qwen3.5:397b`, which Ollama retired on 2026-09-25 (`qwen3.5:397b was retired at 2026-09-25 00:00:00 -0700 PDT`). It cannot self-recover — it needs a new model pin. Until then the daily 03:00 stocktake (skills-index regeneration, freshness audit, Quartz rebuild, Gitea commit) does not run. - **Facebook Birthday Weekly Registry Sync — HTTP 400, model missing from catalog (streak 3):** Job `fad5a86979d4` is pinned to `nvidia/deepseek-ai/deepseek-v4-flash`; the provider reports the model is `not available in the active live catalog for provider 'nvidia'`. The sibling job `0e753e9720fd` uses the same pin, so this is an upstream catalog/availability issue rather than a per-job config error. Safest durable fix is repinning to a routing alias (`smart-route`) instead of a raw model name. - **GitHub Trending Radar — revoked Telegram bot token:** Job `c0a9fa314542` builds its digest successfully, then fails the Telegram push (`getMe` → 401 Unauthorized). The token lives in `/home/hermes/workspace/github-trending-radar/config.json` and needs replacing from @BotFather. Digest content is recoverable from the script's `--dry-run` output in the interim. - **Stale provider key `custom:omni` (2 jobs):** the provider was removed from `custom_providers` (the live 2026-10-01 set is `litellm`, `guacano`, `Omniroute:20128`, `command-code`, `omni-101`). Jobs `e530c89f2be0` and `d40085da631c` still name it; both run `smart-route`, and the default `custom` provider resolves to the same base URL (`http://100.78.63.20:20128`), so no failure has been observed — but the references should be normalised to `custom:omni-101`. - **Bare `telegram` delivery targets (3 jobs):** `0e753e9720fd`, `fad5a86979d4`, `066bb5191280` deliver to the home DM rather than a group topic. Not a failure by itself, but it is the same class of misconfiguration that caused the 2026-09-30 thread-delivery incident, and it leaves stale `thread_id` errors on their records. Retarget to `telegram:-1004321904721:`. ### Historical - **Mnemosyne Watchdog incident (2026-08-19):** Job `96eb95ce12f2` cycled for ~2 hours reporting no consolidation for 36–41.5+ hours. See [[systems/mnemosyne]]. - **Session storage failures (2026-08-10):** Four jobs failed with TimeoutError / "session storage could not be written" at 83% disk. See [[current-state]]. - **Model repinning (2026-08-08):** 13 jobs repinned from dead/stale provider combos to `smart-route` or `gpt-5.6-terra`. - **Wiki Audit model drift (2026-08-07):** Global inference config change caused `RuntimeError: Skipped to prevent unintended spend` on an unpinned job. ## Related - [[systems/mnemosyne]] - [[runbooks/job-radar]] - [[systems/hermes-agent]] - [[incidents/2026-09-28-hermes-managed-runtime-deps]]