14 Commits
Author SHA1 Message Date
Tony0410 f603c29d17 stocktake 2026-11-20: re-verify 9-page 2026-10-29 stale cohort live; correct nanobot docker/cron drift, camofox host/port claims, dead-pin residual; freshness 0 stale/136 fresh 2026-11-20 03:11:58 +08:00
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00
Tony0410 4fd138bb18 audit 2026-11-08 (window 11-07): cron claim loss in gateway restart drain (2 runs orphaned, 2 incident rows unalerted); Hermes 0.22.5->0.23.0 (snapshot skipped), camofox plugin 1.1.0, conduit 0.134.4, hq-push 0.9.42, opencode-go key expired; skills 302->299 (4 archived); freshness 0/126/3 of 129 2026-11-08 00:41:58 +08:00
Tony0410 0b4ff25fd2 resolve: Gitea config backup push dead since 2026-08-08 — 92-commit backlog drained, script hardened to fail loudly
- Wire host-scoped Gitea credential (git credential approve); origin was HTTPS with no userinfo
- Strip 1.76GB checkpoints/ from the unpushed range via git filter-repo --partial (fast-forward preserved)
- Untrack checkpoints/ at the tip; push set ~2GB -> 264.8MB
- Rewrite backup.sh: exit non-zero on failure, stamp .last_backup_ok only after remote verification,
  decide push by remote-vs-local, guard ls-remote with || true, GitHub orphan snapshot via commit-tree,
  honest AgentMail/Telegram delivery reporting
- Verified live: exit 0, origin/main == local HEAD, 0 unpushed, Gitea + GitHub APIs confirmed
- Incident page -> status: resolved; backups.md -> RESOLVED; scheduled-tasks job row updated
- Also commits the uncommitted stocktake edits (the 03:00 stocktake cron failed with
  'lost its durable fire claim ownership' and left the tree dirty)
2026-11-07 05:28:39 +08:00
hermes 3b5daccb4d wiki: nightly audit 2026-11-05 window — network-map re-verified by live probe (9377=Camofox not Stratos, 8790/5174 gone, +9120/9222/7777/6767/8085/9900, full Tailscale Serve table, two dead routes 8769/9123); commit pending 11-04 stocktake re-verifications; log + skills-index 2026-11-05 01:22:45 +08:00
Tony0410andHermes 677bb0c959 docs(wiki): correct Steel Browser references across 15 pages — Steel fully removed, Camofox is sole backend
Root cause of the long-standing cloud_provider: steel residue: the 2026-07-30
rollback disabled the browser-steel plugin and set browser.engine: camofox but
left browser.cloud_provider: steel in config.yaml. The split config was
inert-but-wrong, so it survived ~3 months of audits unnoticed.

Corrects pages that still described Steel as present, running or configured:
- systems/browser-backend.md (incl. gateway-restart env-var note)
- comparisons/steel-vs-camoufox.md, comparisons/camoufox-vs-chrome.md
- decisions/rollback-steel-to-camofox.md, decisions/index.md
- infrastructure/docker-services.md, infrastructure/tailscale.md, infrastructure/hosts.md
- systems/homepage-dashboard.md, systems/scheduled-tasks.md
- concepts/browser-automation.md, runbooks/recover-docker-service.md
- current-state.md, index.md, agents/nanobot.md
- log.md (dated entry; supersedes the 00:3x audit's 'endpoint dark' wording)

Also restores the Ignis compose row in infrastructure/docker-services.md, which
the 2026-10-11 revision dropped (verified: /home/hermes/docker/ignis/docker-compose.yml
exists and the ignis container is Up on CT460).

Co-Authored-By: Hermes <hermes@local>
2026-10-29 02:02:49 +08:00
Tony0410 4126093644 stocktake(2026-10-26): correct 8 stale pages against live state — 0 stale remaining
wiki-maintain.sh ran clean (302 skills indexed, Quartz rebuilt, 634 files emitted,
service restarted). The freshness report listed 8 stale pages; each was diffed
against live state and corrected only where reality could be verified. A second
script run rewrote the same-day report with 0 stale / 103 fresh / 4 no-date,
so the committed report matches the tree it ships with.

- infrastructure/hosts.md: CT600 is LIVE (not retired); CT601 does not exist on
  either hypervisor; CT333 verified running; both hypervisors verified; config
  33->51; skills 66->302; cron 25->28 (25 enabled / 3 disabled); Xvfb :102->:99;
  providers split into Omniroute:20128 (CT600) and omni-101 (CT101, default).
- agents/hermes-production.md: Tailscale IP 100.96.100.82 -> 100.118.5.51 (the
  old value is the PVE hypervisor); v0.21.5 -> v0.22.3; dead 8082 claim removed;
  fallback chain rewritten from live config (first entry is the dead nous
  credential); all 8 auxiliary slots now no-think/freeapi/auto; cron 30->28.
- agents/hermes-chaos.md: v0.22.3 (same build as CT460); "3 commits behind" ->
  0 behind origin/master; default model smart-route on custom:omni-101; cron
  "11 paused" -> 25 jobs (13 enabled / 12 disabled), 9 names shared with CT460.
- infrastructure/tailscale.md: node table rebuilt from status --json; octoagent
  and omarchymac-1 were missing entirely (52 -> 54 non-funnel nodes); m93p now
  online; every offline age refreshed; omni row corrected to "CT601 removed
  from both hypervisors"; Serve tables re-probed per host.
- systems/memory-backend.md: re-verified against live config; coverage 1899/2350
  (80.8%), working 96.8%, episodic 99.4% — above the 95% reindex threshold.
- runbooks/provider-health.md: the step-3 OmniRoute snippet raised IndexError
  (provider renamed omniroute -> Omniroute:20128; key moved to key_env) — fixed;
  new step 3b documents the omni-101 /v1/models false-failure quirk.
- runbooks/update-hermes-safely.md: added managed-runtime dependency re-assert
  step; clarified --branch vs stable channel (v0.22.5 available).
- incidents/2026-10-03-ct450-hermes-update.md + incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md:
  Current-state sections recording current versions and embedding coverage.

All 9 corrected pages verified 200 at http://127.0.0.1:9120/<path>.
Vault mirror at /home/hermes/wiki synced for all touched files (22 copies,
sha256-verified) and committed locally; the vault is local-only and is not pushed.
No secrets written — IDs, URLs, key env var names, counts and timestamps only.
2026-10-26 04:27:49 +08:00
Tony0410 fbd4f297eb wiki: publish pending edits (2026-10-20) 2026-10-20 18:38:11 +08:00
Tony0410 93c55efc94 stocktake 2026-10-19: skills-index 346/70; freshness 47 stale/43 fresh/7 no-date; open-claw VM 403 verified absent from PVE 2026-10-19 03:03:15 +08:00
hermes d981e30128 stocktake(2026-10-11): skills-index 346 skills/70 cats; freshness 62 stale/24 fresh/7 no-date; 5 live-drift corrections
Drift found and corrected against live sources (not guessed):
- OmniRoute: CT600 omniroute is RUNNING and serving :20128 (combo API live);
  CT601 omni is STOPPED with onboot=0 (peer offline 54d). Wiki had these inverted.
  Hermes custom:omni resolves to omniroute -> CT600, not CT601.
- Third OmniRoute node discovered: omniroute-1 at 100.78.63.20:20128, live (401),
  registered as custom provider omni-101 (5384 models, default smart-route,
  carries skills_hub/free-stack). Not in pct list; identity unconfirmed.
- Ignis: TWO live instances. CT460 ignis (:8080, vault wiki/agent-estate-wiki) and
  CT200 ignis-ignis-1 (image ignis-server:latest, host 8092, vault /opt/ignis/vaults).
  Both return HTTP 200. Page now distinguishes them.
- Tailscale: /agent-estate targets bare 127.0.0.1:8769 (was .../8769/agent-estate-role-dashboard.html);
  undocumented tailnet-only :9443 -> 127.0.0.1:5173 added; CT201 routes corrected
  (8085 targets + /steel 5173 + /steel-api 3000, replacing the wrong :8443 -> :8082 row).
- Proxmox: rebuilt from live pct list + qm list. Added CT475 agentzero and VM110 omarchy
  (both previously missing); 282 ubuntu-server-xfce corrected to stopped; dead rows removed.
  VM403 open-claw and VM401 hermes-old absent from qm list - flagged unconfirmed, not deleted.
- Nanobot: gateway nanobot-gateway (PID 2259048) alive on 8765/8766. It is NOT systemd
  nanobot.service/gateway.service (both return inactive - wrong unit names). Port 8900
  no longer listening; "Serve API" claim flagged unverified.
- Docker: old "runtime not available" was a CT460 socket artifact (root:docker srw-rw----,
  hermes not in group docker, no passwordless sudo in cron). Documented CT200 (~50 containers),
  CT333 (9 containers), CT460 via sudo.
- Mattermost CT255 confirmed stopped; matches Tailscale peer offline 21d.

Flagged, not investigated (no root cause):
- mycompose-db-1 on CT200 is in Restarting (1) - crash-looping.

62 pages remain >21d stale and were NOT rewritten this pass (concepts, decisions,
comparisons, most runbooks) - not probed against a live source, so reported not faked.
Quartz rebuild step still inert (no quartz.config.ts) - public/ not refreshed.

Artifacts: systems/skills-index.md regenerated, raw/stocktake/freshness-20261011.txt.
2026-10-11 03:11:07 +08:00
Tony0410 78048bdf0b wiki audit 2026-10-04: CT450 v0.21.5 update incident, mnemosyne vector-recall FTS-only repair + coverage refresh, managed runtimes 10->3, doctor vec0 caveat 2026-10-04 00:15:35 +08:00
Anthony 1d42c6bc83 docs(hermes-chaos): update routing, browser/VNC, wiki governance (2026-08-17) 2026-08-17 04:11:54 +08:00
Nando 5b8ce92b8c docs: update nanobot agent page with current runtime state 2026-08-17 03:07:49 +08:00
Tony0410 c8565c62ac feat: add agent estate section with 7 pages
Add agents/ directory with comprehensive documentation for all agents:
- agents/index.md — estate map and relationships
- agents/hermes-production.md — CT460 production Command Centre
- agents/hermes-chaos.md — CT450 experimental Chaos Lab
- agents/nanobot.md — CT333 Scout (quota notifications)
- agents/qwenpaw.md — CT465 multi-agent gateway
- agents/zeroclaw.md — CT333 autonomous daemon (TOML)
- agents/open-claw.md — VM 403 autonomous gateway

Updated index.md with Agent Estate section and links.
All pages follow existing SCHEMA.md conventions with YAML frontmatter.
2026-08-15 22:21:32 +08:00