Wiki maintenance (wiki-maintain.sh, exit 0) + live-vs-page drift audit.
Script:
- systems/skills-index.md regenerated from the live skill tree: 302 skills in 69
categories; 45 archived skills under ~/.hermes/skills/.archive/ now excluded and
noted. (303 SKILL.md files exist outside .archive/, but
productivity/obsidian-git-sync/obsidian-git-sync is a nested alias duplicate,
so 302 unique is correct.)
- Freshness report raw/stocktake/freshness-20261029.txt: 2 stale / 105 fresh /
3 no-date. After the corrections below and a second script run: 0 stale /
107 fresh / 3 no-date.
- Quartz rebuilt (110 pages -> 644 files) and deployed to public/.
Drift corrections (live-verified):
- infrastructure/tailscale.md: State column rebuilt from tailscale status --json,
11 rows changed. Two were state transitions, not age drift:
* sftpgo (CT299, pve) is back online - tailscale ping pong in 0s,
systemctl is-active sftpgo -> active, tailscale ip -4 inside the guest.
* m93p is online despite a stale peer record (Online: false in the JSON) -
tailscale ping pong in 1ms, SSH over the tailnet returned up 11 days.
Second occurrence of this pattern after 2026-10-26.
The other 9 rows were offline-age drift of exactly +1 day. Node count unchanged
at 54 (53 live IPv4 peers + hermes); 76 peers total, 23 IPv6-only funnel
ingress. No missing peers, no IP mismatches. Container identity re-confirmed
from inside all 34 guests via pct exec tailscale ip -4 on both hypervisors;
CT601 still absent from both.
- incidents/2026-10-01-cliproxy-connection-auth-expired.md: the residual claim
"the connection credential itself was never re-authenticated" is falsified.
Live POST /v1/chat/completions for cliprox/terra against custom:Omniroute:20128
(CT600) returns HTTP 200 - the connection has since been re-authenticated. The
same pin on custom:omni-101 (CT101) still returns HTTP 401, so the nodes carry
separate credentials. Claim marked superseded, not deleted (SCHEMA rule 9); the
remediation was never reversed and remains correct.
- incidents/2026-10-05-dead-model-pin-repin.md: every claim re-probed. All 7 jobs
still on custom:omni-101 with routing aliases; both aliases HTTP 200;
ollama-cloud/qwen3.5:397b still HTTP 410; nvidia/deepseek-v4-flash still
HTTP 400; command-code/gpt-5.6-terra now HTTP 401 rather than the original 403
- still dead, different failure signature.
- systems/scheduled-tasks.md: the Known-Issues entry repeating the
"never re-authenticated" claim now carries the correction. Fleet figures
re-checked against live jobs.json and remain accurate (29 jobs / 26 enabled /
3 disabled, 5 non-clean last_status).
- log.md / index.md: stocktake entry appended and index last-updated refreshed.
Verified correct, no change: systems/homepage-dashboard (CT201 has 8084 and 8085
listening, no 8082, Homarr on 8102); CT201 Serve routes (/ -> 8085, /mgr -> 8084
on both :443 and :8443, Steel routes still gone); CT460 Serve routes (12 paths
on :443, plus :8443/webhook Funnel and :9443).
Noted for a human, not performed: vault mirror at /home/hermes/wiki/agent-estate-wiki
has drifted on both conventions (top-level 35 match / 3 stale / 72 missing;
wiki/ 41 match / 53 stale / 16 missing). A full mirror pass is larger than a
stocktake's mandate.
Root cause of the long-standing cloud_provider: steel residue: the 2026-07-30
rollback disabled the browser-steel plugin and set browser.engine: camofox but
left browser.cloud_provider: steel in config.yaml. The split config was
inert-but-wrong, so it survived ~3 months of audits unnoticed.
Corrects pages that still described Steel as present, running or configured:
- systems/browser-backend.md (incl. gateway-restart env-var note)
- comparisons/steel-vs-camoufox.md, comparisons/camoufox-vs-chrome.md
- decisions/rollback-steel-to-camofox.md, decisions/index.md
- infrastructure/docker-services.md, infrastructure/tailscale.md, infrastructure/hosts.md
- systems/homepage-dashboard.md, systems/scheduled-tasks.md
- concepts/browser-automation.md, runbooks/recover-docker-service.md
- current-state.md, index.md, agents/nanobot.md
- log.md (dated entry; supersedes the 00:3x audit's 'endpoint dark' wording)
Also restores the Ignis compose row in infrastructure/docker-services.md, which
the 2026-10-11 revision dropped (verified: /home/hermes/docker/ignis/docker-compose.yml
exists and the ignis container is Up on CT460).
Co-Authored-By: Hermes <hermes@local>
Window 2026-10-27 settled 172 completed / 1 failed. The failure is dca8482e4f76
(Daily Marketing Job Radar - Discovery): 07:01:08 -> 11:03:54 (4h 2m 46s), closed
failed, delivered report opens "INCOMPLETE (tool-call budget exhausted before
Notion write stage)" — discovery/verify/dedup/classify finished, Notion writes and
cover letters never ran (Notion DB held 451 entries, 0 added). Incident row
dca848_e93a538ade6b opened 11:03:53, still open, failure_type: auth (misclassified).
Response the same day: split into stage 1 dca8482e4f76 (Discovery, 0 7 * * 1-5,
deliver local) + new stage 2 5fc292ff0f3b (Write and Letters, 30 11,14 * * 1-5,
topic 1030, created 2026-10-27T17:34:15). Fleet 28 -> 29 jobs (26/3).
Also: the two incident rows the 2026-10-26 planned restart raised auto-closed
00:38:32 (unclosed 3 -> 2) and the orphan class did not recur; non-clean
last_status 4 -> 5; skills-index.md regenerated by the 10-27 03:00 slot but never
committed (302 skills / 69 categories; freshness 0 stale / 104 fresh / 4 no-date of
108) — committed here and index.md corrected; tailscale.md peer table rebuilt from
tailscale status --json (14 rows, m93p peer flag disputed and ping-verified, sftpgo
CT299 running but its peer dropped); systems/hermes-agent.md TTS corrected to
provider openai -> Mac mini speech server (phonon/alloy), systems/conduit.md gained
the voice pipeline section.
New: incidents/2026-10-27-job-radar-budget-exhausted.md
Nothing removed or archived. No secrets.
Vault at /home/hermes/wiki tracks 125 files under two conventions. This run
forward-synced all 22 touched-page copies (sha256-verified) and committed the
vault locally; it is local-only and was not pushed. Remaining drift measured by
hash sweep and recorded rather than silently mirrored:
top-level: match=25 stale=2 missing=80
wiki/ : match=34 stale=58 missing=15
wiki-maintain.sh ran clean (302 skills indexed, Quartz rebuilt, 634 files emitted,
service restarted). The freshness report listed 8 stale pages; each was diffed
against live state and corrected only where reality could be verified. A second
script run rewrote the same-day report with 0 stale / 103 fresh / 4 no-date,
so the committed report matches the tree it ships with.
- infrastructure/hosts.md: CT600 is LIVE (not retired); CT601 does not exist on
either hypervisor; CT333 verified running; both hypervisors verified; config
33->51; skills 66->302; cron 25->28 (25 enabled / 3 disabled); Xvfb :102->:99;
providers split into Omniroute:20128 (CT600) and omni-101 (CT101, default).
- agents/hermes-production.md: Tailscale IP 100.96.100.82 -> 100.118.5.51 (the
old value is the PVE hypervisor); v0.21.5 -> v0.22.3; dead 8082 claim removed;
fallback chain rewritten from live config (first entry is the dead nous
credential); all 8 auxiliary slots now no-think/freeapi/auto; cron 30->28.
- agents/hermes-chaos.md: v0.22.3 (same build as CT460); "3 commits behind" ->
0 behind origin/master; default model smart-route on custom:omni-101; cron
"11 paused" -> 25 jobs (13 enabled / 12 disabled), 9 names shared with CT460.
- infrastructure/tailscale.md: node table rebuilt from status --json; octoagent
and omarchymac-1 were missing entirely (52 -> 54 non-funnel nodes); m93p now
online; every offline age refreshed; omni row corrected to "CT601 removed
from both hypervisors"; Serve tables re-probed per host.
- systems/memory-backend.md: re-verified against live config; coverage 1899/2350
(80.8%), working 96.8%, episodic 99.4% — above the 95% reindex threshold.
- runbooks/provider-health.md: the step-3 OmniRoute snippet raised IndexError
(provider renamed omniroute -> Omniroute:20128; key moved to key_env) — fixed;
new step 3b documents the omni-101 /v1/models false-failure quirk.
- runbooks/update-hermes-safely.md: added managed-runtime dependency re-assert
step; clarified --branch vs stable channel (v0.22.5 available).
- incidents/2026-10-03-ct450-hermes-update.md + incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md:
Current-state sections recording current versions and embedding coverage.
All 9 corrected pages verified 200 at http://127.0.0.1:9120/<path>.
Vault mirror at /home/hermes/wiki synced for all touched files (22 copies,
sha256-verified) and committed locally; the vault is local-only and is not pushed.
No secrets written — IDs, URLs, key env var names, counts and timestamps only.
omniroute-1 (100.78.63.20:20128, provider custom:omni-101) TCP-unreachable 10-14 08:30 -> 10-15 18:07
(httpcore.ConnectError Errno 111 -> openai.APIConnectionError -> RuntimeError: Connection error).
36 failure rows across 13 jobs = 24 genuine model-call failures on 12 jobs + 12 Mnemosyne Watchdog
symptom rows. 10-14 160/12, 10-15 151/24 (vs 10-12 175/0, 10-13 167/1).
STRUCTURAL: all 18 enabled agent-mode jobs are pinned to custom:omni-101; the only other model-capable
pin (custom:omni, CT600) sits on a no_agent job -> no model-call failover, and zero model calls
completed in 34h. Corrects a first-pass reading of Evening Stand-Down's green rows as CT600 evidence.
New: incidents/2026-10-14-omniroute-node-connection-refused-storm.md (resolved, verified live).
Updated: systems/scheduled-tasks (frontmatter, Count, new Re-verified block), systems/omniroute,
systems/mnemosyne (monitor blast-radius note), current-state, index, log.
- Who owns which section (scope rules per agent)
- What to save vs never save (no secrets, no session chatter)
- Mandatory conventions (frontmatter, wikilinks, auto-generated pages)
- Git workflow for contributing agents
- Freshness responsibility: each agent keeps its own section current
Link CONTRIBUTING from index.md start-here block.