Stocktake 2026-10-30: 0 stale by age, but 5 pages drifted from live state

Ran wiki-maintain.sh (skills-index regenerated: 302 skills / 69 categories,
unchanged; freshness report 0 stale / 109 fresh / 3 no-date of 112 pages;
Quartz rebuilt and deployed).

Freshness is not accuracy. A live-vs-page diff found real drift on five pages.
All corrections verified against live sources first.

- infrastructure/tailscale: node table rebuilt from tailscale status --json.
  50 non-funnel nodes (49 IPv4 peers + hermes), not 54. 2 IP changes, 4 peers
  expired out of the tailnet. Zero IP mismatches, zero missing/extra rows on
  re-verification. m93p records online (peer record chronically stale, third
  occurrence -- ping + SSH confirm). CT460 :443 route set (12 paths) and
  CT201 :443/:8443 tables re-probed and match.
- systems/scheduled-tasks: fleet is 29 jobs (25 enabled / 4 disabled), not
  28/25/3. Mnemosyne Health Monitor ea79a2107e36 was auto-paused 2026-10-30
  01:04:22 after its 00:01:08 slot closed "unknown" on a gateway-connection
  failure. Non-clean last_status is 4, not 3. "Failing (delivery only)" is 2.
  Seven job rows carried stale error claims for jobs since cleared to ok --
  each marked superseded with a dated correction, not rewritten (SCHEMA rule 9).
- NEW incidents/2026-10-30-mnemosyne-health-monitor-paused: the job's own
  failure notice was never delivered (deliveries.db has zero rows for
  2026-10-30), so the incident row is the only trace. Job not re-enabled and
  gateway not restarted by this run; root cause recorded as unverified.
- infrastructure/hosts: CT201 LAN is 192.168.178.77 (was .166); CT450 Hermes
  is v0.22.5 / bf0203a794a0, same build as CT460 (page said v0.22.3 /
  5b147ab036d6); cron count 29/25/4; orphaned "omni" peer has expired out of
  the tailnet entirely. Skills count of 302 left alone -- it is correct.
- systems/homepage-dashboard: live app count is 123 (was 114); grid is a
  responsive repeat(auto-fill, minmax(230px, 1fr)), not a fixed 5-column grid;
  tab bar is built from the service groups (renderTabs iterates state.groups),
  so the settings.yaml layout block and its tab:/columns: keys are inert --
  there is no "Overview" tab; saved theme is Boarding Pass / light, not dark
  amber; manager.py runs inside the homepage-manager container; and the "no
  Traefik container" evidence was wrong -- the container exists but is
  stopped (Exited (255) 2 months ago). The conclusion that Traefik does not
  occupy 443 stands. Same evidence error corrected in tailscale.md.

Also corrected two of my own first-pass errors before committing: the
homepage tab list (groups vs tabs) and the CT450 version (a stale root-level
/usr/local/bin/hermes reports v0.19.0; the live agent is the hermes-user
install at v0.22.5).

No secrets written -- provider names, key names and HTTP status codes only.
This commit is contained in:
Tony0410
2026-10-30 05:41:11 +08:00
parent 1d335ca8e6
commit 987eba72e6
10 changed files with 448 additions and 73 deletions
+62
View File
@@ -1584,3 +1584,65 @@ not pushed.
`log.md`. The wiki-maintained pages under `infrastructure/` remain a separate, known gap.
- **Removed/archived:** none. **Secrets written:** none — provider names, key *names* and HTTP status codes only, never a
key or token value.
## 2026-10-30 — Stocktake (03:00 AWST): 0 stale by age, but 5 pages had drifted from live state
- **Ran `wiki-maintain.sh`.** Regenerated `systems/skills-index.md` (**302 skills / 69 categories** — unchanged; the
generator de-dups one nested duplicate, so 302 is the correct unique-skill figure), wrote
`raw/stocktake/freshness-20261030.txt` (**0 stale / 109 fresh / 3 no-date of 112 pages**), rebuilt Quartz
(654 files) and deployed to `public/`.
- **Freshness is not accuracy.** The age scan found nothing, but a live-vs-page diff against the running estate found
real drift on five pages. All corrections below were verified against live sources before being written.
- **[[infrastructure/tailscale]] — node table rebuilt.** `tailscale status --json` now reports **50 non-funnel nodes**
(49 live IPv4 peers + `hermes`), not the 54 the page claimed. **2 IP changes** (`anthonys-mac-mini` 100.74.17.15 →
100.76.60.37; `omarchymac` 100.84.180.69 → 100.126.173.115 — the peer formerly listed as `omarchymac-1`) and **4 nodes
dropped out of the tailnet entirely** (`anthonys-imac`, `cachyos`, `omarchymac-1`, `omni`). Every offline age refreshed.
`m93p` reads `Online: false` for the **third** consecutive audit yet is plainly up (`tailscale ping` → pong, SSH → `up 12
days`), so the row records it online. Total peers **72** (49 IPv4 + 23 IPv6-only funnel). CT460's `:443` route set
matched the page exactly; CT201 serves only `/` → 8085 and `/mgr` → 8084 on both `:443` and `:8443`.
- **[[systems/scheduled-tasks]] — fleet ledger corrected.** Live `jobs.json` shows **29 jobs (25 enabled / 4 disabled)**, not
the 28/25/3 on the page: `ea79a2107e36` (Mnemosyne Health Monitor) was **auto-paused 2026-10-30 01:04:22** after its
00:01:08 slot closed **`unknown`** on a gateway-connection failure. Non-clean `last_status` is **4** (not 3), and the
standing "every `failure_streak` reads 0" claim was false (`ea79a2107e36` = 1). "Failing (delivery only)" is **2**, not 1
(`fad5a86979d4` joined `066bb5191280`). The retired DHL row was dropped from the job table (it lives in Removed Jobs).
Seven job rows carried **stale `error` claims** for jobs that have since cleared to `ok` — each was marked superseded
with a dated correction rather than rewritten (SCHEMA rule 9). The `executions.db` retention floor note was corrected to
the live **2026-10-24 09:22:36**. Window tallies: **2026-10-29 = 177 / 0** and **2026-10-28 = 169 / 2** (both +1 on the
page — the 23:59 Wiki Audit row had still been `running` at audit time).
- **New incident page: [[incidents/2026-10-30-mnemosyne-health-monitor-paused]].** The Mnemosyne Health Monitor's own
failure notice **failed to send** (`send_path_degraded; live adapter delivery to telegram:-1004321904721:102 failed`) —
`deliveries.db` holds **zero rows for 2026-10-30**, so the incident row is the only trace. The job was **not re-enabled**
and the gateway was **not restarted** by this run; root cause is **unverified** and recorded as such.
- **[[infrastructure/hosts]] — four corrections.** CT201's LAN address is `192.168.178.77`, not `192.168.178.166`; CT450's Hermes Agent is **v0.22.5** (2026.9.24),
local **`bf0203a794a0`** — the same build as CT460 — so the page's "v0.22.3 / 5b147ab036d6" was wrong on both
(a stale *root-level* `/usr/local/bin/hermes` on CT450 still reports v0.19.0 and is **not** the live agent —
the live one is the `hermes`-user install under `~/.hermes/hermes-agent`); the cron count is now 29/25/4; and
the orphaned `omni`
peer has **expired out of the tailnet entirely** (no peer named `omni`, no peer at 100.93.204.4), so the "offline 69d"
description no longer applied. The **skills count of 302 was left alone** — it is correct.
- **[[systems/homepage-dashboard]] — five corrections, and a second pass to fix my own first pass.**
"All Apps" is **114 → 123 live apps**, not 122 (the API dedupes by `(name, normalised href)` across groups;
the literal `All Apps` group in `services.yaml` holds 122 and is not what the tab bar counts). The grid is a
responsive `repeat(auto-fill, minmax(230px, 1fr))` (served `/dash.css` line 3640), not a fixed 5-column grid.
The tab list was wrong — but **my first correction was wrong too** and had to be redone: the live tab bar is
built from the service *groups* (`renderTabs()` iterates `state.groups`), **not** from the `layout:` block in
`settings.yaml`. That block and its `tab:`/`columns:` keys are **inert** — `grep -c "Overview" /dash.js` = 0 and
`grep -c "columns" /dash.js` = 0. There is **no "Overview" tab**; the live tabs are `☆ Pinned`, `All (123)` and
the **14 groups** (AI Quotas 3, Infrastructure 13, Everyday 8, Agents 20, Media 11, AI & LLM 17, Tools 20,
Storage 6, More 13, Resources 6, System Health 0, Subscriptions 1, Quick Launch 2, Uncategorised 3). The saved
theme is **`{"pair":"Boarding Pass","mode":"light"}`** — one of **59** theme pairs; its light-mode amber is
`#795000`, so the old "dark base with warm amber (#d4a04a)" was wrong on both counts. **`manager.py` runs inside
the `homepage-manager` container** (its container `Cmd` is the `pip install … && python3 /opt/homepage/manager.py`
line, confirmed via `docker inspect` and `docker top`); only **`asset_proxy.py`** is a host process
(`0.0.0.0:8085 users:(("python3",pid=4038))`). And the "no Traefik container exists" evidence was wrong — the
`runtipi-reverse-proxy` Traefik container exists but is **stopped** (`Exited (255) 2 months ago`); the *conclusion*
that Traefik does not occupy 443 stands. The same evidence error was corrected in [[infrastructure/tailscale]].
- **[[current-state]]** and **[[index]]** updated with the new incident and the corrected fleet numbers.
- **Verified, not assumed:** CT600 **running** and CT601 **absent from both hypervisors** (`pct config 601` → no such
config on `pve` or `m93p`); the CT201 `:443`/`:8443` Serve tables; CT201 listeners 5173/3000/8082 (**none** — Steel's
removal confirmed); the homepage service groups and theme; and every cron counter, tally, incident row and delivery row
(read from `jobs.json`, `executions.db` — whose `cron_incidents` table lives inside it, the standalone
`cron_incidents.db`/`incidents.db` files are empty stubs — and `deliveries.db`).
- **Not changed:** `systems/skills-index.md` count (302 is correct), CT460's Serve route table (matched live exactly), and
the steel-removal records across the wiki (confirmed still true).
- **Secrets written:** none — provider names, key *names* and HTTP status codes only, never a key or token value.