wiki: nightly audit 2026-11-04 window — new incident (Job Radar + Newsletter Digest both killed by upstream stream failure; Stage 2 rebuilt handoff and delivered 5 roles + 1 letter); CT299 second freeze documented; CL_MODEL 402 caveat in job-radar runbook; scheduled-tasks/current-state/index/log updated
This commit is contained in:
+15
-4
@@ -3,13 +3,13 @@ title: Daily Marketing Job Radar Operations and Recovery
|
||||
type: runbook
|
||||
status: active
|
||||
created: 2026-07-27
|
||||
updated: 2026-10-30
|
||||
updated: 2026-11-05
|
||||
review_after: 2026-11-13
|
||||
verified_on: 2026-10-30
|
||||
verified_on: 2026-11-05
|
||||
version_applies_to: Hermes Agent cron dca8482e4f76 (stage 1) + 5fc292ff0f3b (stage 2)
|
||||
confidence: high
|
||||
tags: [automation, cron, jobs, notion, dashboard, runbook]
|
||||
sources: [live-cron-output, live-notion-api, live-dashboard-api, hermes-docs]
|
||||
tags: [automation, cron, jobs, notion, dashboard, runbook, cover-letter, model-route]
|
||||
sources: [live-cron-output, live-notion-api, live-dashboard-api, hermes-docs, live-script-inspection, live-endpoint-probe]
|
||||
---
|
||||
|
||||
# Daily Marketing Job Radar Operations and Recovery
|
||||
@@ -39,6 +39,17 @@ Cron jobs inherit the profile-level `fallback_providers` chain; Hermes does not
|
||||
|
||||
## Cover letters are non-negotiable
|
||||
|
||||
> **Model route caveat - verified 2026-11-05.** `regenerate_cover_letter.py` (line 82) reads
|
||||
> `os.environ.get("CL_MODEL", "deepseek-paid-smart")`. On the 2026-11-04 run that default returned **HTTP 402
|
||||
> (billing)**, and the run pinned **`gemini/gemini-3.5-flash`** via `CL_MODEL` to get the strong-match letter out.
|
||||
> `grep` finds **no `CL_MODEL` in `~/.hermes/.env`** - the override existed only inside that process, so **the
|
||||
> next run inherits the default again**. If a run logs 402/503 while generating letters, re-pin `CL_MODEL` for the
|
||||
> process and confirm the letters rendered - do **not** ship a run without its cover letters. Note the same model
|
||||
> name still serves other cron jobs successfully (e.g. the 23:59 nightly audit), and the endpoint itself is live:
|
||||
> `omniroute:20128/v1/models` -> **401** and `100.78.63.20:20128/v1/models` -> **401** on 2026-11-05 (key-gated,
|
||||
> not down). Treat a 402 as **route/time-specific until re-tested**, not as proof the route is dead.
|
||||
|
||||
|
||||
A successful radar run includes the cover-letter stage.
|
||||
|
||||
- Audit **every Strong Match added today**, including pages created by an interrupted earlier run.
|
||||
|
||||
Reference in New Issue
Block a user