🦉 wiki audit 2026-10-21: gateway unclean-exit orphaned run (new), Gitea config-backup push dead since 2026-08-08 (new), model default → nous, 503 back to monitoring, canned-substitute trigger corrected

This commit is contained in:
Tony0410
2026-10-21 00:11:44 +08:00
parent fbd4f297eb
commit b662254569
10 changed files with 386 additions and 25 deletions
+49 -4
View File
@@ -3,16 +3,61 @@ title: Current State
type: concept
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
review_after: 2026-10-27
updated: 2026-10-21
verified_on: 2026-10-21
review_after: 2026-10-28
confidence: high
tags: [system]
sources: [live-cron-list, live-sqlite-cron-executions, live-sqlite-sessions, live-probe]
---
# Current State
Updated: 2026-10-20
Updated: 2026-10-21
## Recent Changes (2026-10-21, nightly audit; live probes 2026-10-21 00:03-00:15 AWST)
- **A clean day: 2026-10-20 closed 168 `completed` / 0 `failed`** (plus 1 `unknown`, plus this audit) — the first
zero-failure day since **2026-10-12**, ending a seven-day run of failures. **The `omni-101` 503 fault did not
recur** (the last failure row carrying the resource-pressure string remains 2026-10-19), and **zero new
incident rows were opened** across the whole window; 4 closed, so **unclosed rows fall 13 → 9** (8 `alerted` /
1 `detected`). → [[incidents/2026-10-16-omniroute-503-resource-pressure]] (status back to **`monitoring`**)
- **A gateway unclean exit orphaned a cron run, and nothing reported it.** `08eb86f02d37` (Gmail Action Digest,
08:30 slot) was claimed 08:31:03 and closed as **`unknown`**: *“Scheduler restarted after this execution's owner
exited before a durable terminal state; whether side effects ran is unknown.”* The previous gateway life
(`pid 2496376`, started 2026-10-17 05:14) was recorded **UNCLEANLY** at 09:09:01 — last heartbeat 08:50:28, RSS
394 MB, 12.2 GB free, **`suspected_oom=False`**. `delivery_outcome` is NULL and **no delivery row exists for
the slot**, so the ledger offers no evidence the digest arrived; **no incident row was created**, and the job
table still shows the *previous* slot's 503. First recorded orphan of its kind; first unclean exit since
2026-09-21. → [[incidents/2026-10-20-gateway-unclean-exit-orphaned-run]]
- **⚠ A green cron row is not evidence the work happened — second instance, and this one is 2.5 months old.**
`e27da6c34bbb` (Hermes Config Backup to Gitea, daily 11:00) reads **`last_status: ok`**, `last_error: null`, and
its 2026-10-20 execution reads `completed` — while **nothing has reached Gitea since 2026-08-07 11:00:47**.
The repo is **75 commits ahead of `origin/main`**; `backup.log` ends
`could not read Username for 'https://gitea.kangaroo-eel.ts.net'` → `✗ Push to Gitea FAILED` →
`--- Backup complete: FAIL ---`. `origin` is HTTPS with no credential in the URL and no helper, so the
non-interactive push cannot authenticate: **no off-box `~/.hermes` backup since 2026-08-08**. Telegram failure
notices have been firing all along (last 2026-10-20 11:00, `message_id 8447`); the AgentMail step failed.
→ [[incidents/2026-08-08-gitea-config-backup-push-dead]], [[infrastructure/backups]]
- **The global model default moved off OmniRoute.** `model.provider: custom` → **`nous`**
(`https://inference-api.nousresearch.com/v1`), `model.default: smart-route` → **`meituan/longcat-2.5-preview:free`**
(not in place 2026-10-09 19:42; in place by 2026-10-20 11:00). **Pins are unchanged** — 20 of 28 jobs still pin
OmniRoute (`smart-route` ×11, `free-stack` ×6, `deepseek-paid-smart` ×1, `custom:omni` ×2) — but the **8 jobs
with no pin now call Nous direct instead of the router**, so the fleet runs two upstream paths for the first
time. → [[systems/current-profile]], [[systems/scheduled-tasks]]
- **A user question was answered with a canned non-answer, and the request is still outstanding.** Desktop
session `20261020_183748_6d40ea` (2026-10-20 18:37, `model: minimax/minimax-m3.1-flash:free`) asked *how to use
Nous OAuth dashboard auth for the gateway* and *whether a gateway token could be configured*; the turn ended
`reason=rebuilt_restart_limit_exceeded … response_len=0` and the only assistant message is *“Your request was
not processed. Send it again if you still want me to carry it out.”* **Nothing was configured in response to
that request** — it needs re-asking. → [[incidents/2026-10-18-degenerate-stream-canned-substitute]]
- **`dashboard.basic_auth` now carries a `secret` and `session_ttl_seconds: 43200`** (present 2026-10-21 00:0x).
The plugin's *“no 'secret' configured; generating a random per-process signing key … sessions will not survive
a restart”* warning last appeared 2026-10-20 18:45:14, so dashboard sessions should now persist across
restarts.
- **Interactive sessions returned after weeks of cron-only windows:** `talaria-ios` (10:49 → 19:27, 25 messages,
12 tool calls, *“Get Australian tax return done”*, `provider: custom:command-code`) and the `desktop` session
above. **Non-clean `last_status` 12 → 9** (7 `error` / 2 `delivery_failed`); fleet unchanged at **28 jobs / 25
enabled / 3 disabled**. → [[systems/scheduled-tasks]]
## Recent Changes (2026-10-20, nightly audit; live probes 2026-10-20 00:03-00:12 AWST)