65 lines
3.7 KiB
Markdown
65 lines
3.7 KiB
Markdown
---
|
|
title: Two digests died on `content_policy_blocked` (2026-11-20) — first content-filter failures on record
|
|
type: incident
|
|
status: active
|
|
created: 2026-11-21
|
|
updated: 2026-11-21
|
|
verified_on: 2026-11-21
|
|
review_after: 2026-11-22
|
|
confidence: high
|
|
tags: [cron, content-policy, provider-error, gmail, freshrss, digests, incident, unclassified-failure]
|
|
sources: [live-probe-cron-executions-db-2026-11-21, live-probe-cron-incidents-2026-11-21, live-probe-jobs-json-2026-11-21]
|
|
---
|
|
|
|
# Two digests died on `content_policy_blocked` (2026-11-20)
|
|
|
|
**The first content-filter failures in the retained log.** Two unrelated jobs, on two different feeds, died
|
|
with the *identical* terminal string on 2026-11-20 — hours apart, with every other job in the window clean.
|
|
|
|
## What happened
|
|
|
|
| Job | Job ID | Slot (AWST) | Execution | Duration | Terminal error |
|
|
|---|---|---|---|---|---|
|
|
| Gmail Action Digest | `08eb86f02d37` | 08:31:06 → 08:32:45 | `a490b74a509e4303acad671ee1ee80cf` | 1m39s | `RuntimeError: content_policy_blocked: Output data may contain inappropriate content.` |
|
|
| FreshRSS Midday News Briefing | `ecc1fc13655a` | 12:01:14 → 12:05:04 | `0e1c11f49313428799de604afe73595c` | 3m50s | same string |
|
|
|
|
Both jobs are pinned `provider: custom:omni-101` / `model: smart-route`. Both carry
|
|
**`failure_streak = 1`** — i.e. **isolated single failures, not chronic**: the preceding slot of each job
|
|
completed. FreshRSS Midday ran clean on 11-19 12:01.
|
|
|
|
## What is established
|
|
|
|
- The failure is raised **by the model-facing API, not by the job logic**. The string is a provider content
|
|
filter surfaced to the job as a wrapped `RuntimeError`; the digest scripts assembled their input fine.
|
|
- **Both jobs alerted, and both alerts are unclosed.** Incident rows:
|
|
- `08eb86_c0cca1a55b96` — first seen 11-20 08:32:44, `alerted_at` 08:32:46
|
|
- `ecc1fc_5a10b70830d1` — first seen 11-20 12:05:03, `alerted_at` 12:05:05
|
|
|
|
This is the **opposite** of [[incidents/2026-11-18-wiki-audit-credit-outage]], which failed silently for two
|
|
nights: here the alerting path worked end to end.
|
|
- `failure_type` is recorded as **`unknown`** for both. The incident classifier does not recognise this
|
|
class, so it can neither group nor auto-close it — a recurrence will alert, but will not be understood.
|
|
- Both left a full transcript: `~/.hermes/cron/output/08eb86f02d37/2026-11-20_08-32-44.md` and
|
|
`~/.hermes/cron/output/ecc1fc13655a/2026-11-20_12-05-03.md`.
|
|
|
|
## What is NOT established
|
|
|
|
- **Whether the trigger was the request or the response.** The message reads "Output data may contain
|
|
inappropriate content", but a filter that scans a request payload reports the same way. The Gmail digest
|
|
reads mail; the FreshRSS digest reads news — the shared factor is more likely a *token* the router's filter
|
|
dislikes than the subject matter. **The two output files above settle this**: diff what the two prompts had
|
|
in common.
|
|
- **Whether the router (`omni-101`) applies a filter its upstreams do not.** If it does, the same feeds can
|
|
fail again with nothing changed on this host. Probe: replay one of the two prompts against the router and
|
|
against an upstream provider directly, and compare.
|
|
- Whether it recurs. Two slots, two jobs, one day is a pattern worth watching — not yet a trend.
|
|
|
|
## Standing risk
|
|
|
|
Two of the estate's daily digests sit behind a provider filter that has already rejected them once, and the
|
|
classifier files the failure as `unknown` — so a recurrence will alert without being recognised. Cheapest
|
|
mitigation if it recurs: **no content-policy string is worth a lost digest** — route the two digest jobs to a
|
|
provider without the filter, or strip the highest-risk section from the prompt payload.
|
|
|
|
→ [[systems/scheduled-tasks]], [[systems/mnemosyne]]
|