Files
agent-estate-wiki/agents/hermes-chaos.md
T
Tony0410 4126093644 stocktake(2026-10-26): correct 8 stale pages against live state — 0 stale remaining
wiki-maintain.sh ran clean (302 skills indexed, Quartz rebuilt, 634 files emitted,
service restarted). The freshness report listed 8 stale pages; each was diffed
against live state and corrected only where reality could be verified. A second
script run rewrote the same-day report with 0 stale / 103 fresh / 4 no-date,
so the committed report matches the tree it ships with.

- infrastructure/hosts.md: CT600 is LIVE (not retired); CT601 does not exist on
  either hypervisor; CT333 verified running; both hypervisors verified; config
  33->51; skills 66->302; cron 25->28 (25 enabled / 3 disabled); Xvfb :102->:99;
  providers split into Omniroute:20128 (CT600) and omni-101 (CT101, default).
- agents/hermes-production.md: Tailscale IP 100.96.100.82 -> 100.118.5.51 (the
  old value is the PVE hypervisor); v0.21.5 -> v0.22.3; dead 8082 claim removed;
  fallback chain rewritten from live config (first entry is the dead nous
  credential); all 8 auxiliary slots now no-think/freeapi/auto; cron 30->28.
- agents/hermes-chaos.md: v0.22.3 (same build as CT460); "3 commits behind" ->
  0 behind origin/master; default model smart-route on custom:omni-101; cron
  "11 paused" -> 25 jobs (13 enabled / 12 disabled), 9 names shared with CT460.
- infrastructure/tailscale.md: node table rebuilt from status --json; octoagent
  and omarchymac-1 were missing entirely (52 -> 54 non-funnel nodes); m93p now
  online; every offline age refreshed; omni row corrected to "CT601 removed
  from both hypervisors"; Serve tables re-probed per host.
- systems/memory-backend.md: re-verified against live config; coverage 1899/2350
  (80.8%), working 96.8%, episodic 99.4% — above the 95% reindex threshold.
- runbooks/provider-health.md: the step-3 OmniRoute snippet raised IndexError
  (provider renamed omniroute -> Omniroute:20128; key moved to key_env) — fixed;
  new step 3b documents the omni-101 /v1/models false-failure quirk.
- runbooks/update-hermes-safely.md: added managed-runtime dependency re-assert
  step; clarified --branch vs stable channel (v0.22.5 available).
- incidents/2026-10-03-ct450-hermes-update.md + incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md:
  Current-state sections recording current versions and embedding coverage.

All 9 corrected pages verified 200 at http://127.0.0.1:9120/<path>.
Vault mirror at /home/hermes/wiki synced for all touched files (22 copies,
sha256-verified) and committed locally; the vault is local-only and is not pushed.
No secrets written — IDs, URLs, key env var names, counts and timestamps only.
2026-10-26 04:27:49 +08:00

4.8 KiB

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Hermes Chaos system active 2026-08-15 2026-10-26 2026-10-26 high
hermes
chaos
ct450
experimental
agent
live-probe
config-inspection

Hermes Chaos

Purpose

Experimental Hermes instance for testing new configurations, skills, themes, model routing, and plugin changes before promoting them to production. Changes here must not affect CT460 production without explicit approval.

Location

  • Container: LXC CT450 (hermesagent)
  • Access: no SSH — go through the PVE host: ssh root@100.96.100.82, then pct exec 450 -- <cmd> (or pct enter 450). hermes is not on pct exec's default PATH; use su - hermes -c "/home/hermes/.local/bin/hermes …".
  • Tailscale IP: 100.98.205.112
  • Hostname: hermesagent
  • Software: hermes-ui (Vite web app for browser use, port 5174)
  • Memory: CPU-only container (no GPU) — self-hosted VRAM models are off the table

Current State

  • Status: Active (experimental)
  • Hermes version: v0.22.3 (2026.9.24), local 5b147ab036d6 — the same build as CT460 (updated 2026-10-21). 0 commits behind origin/main (verified 2026-10-26; the old "3 commits behind" claim is stale), though hermes update --check reports stable v0.22.5 available. Python 3.14.7. See incidents/2026-10-03-ct450-hermes-update.
  • Mattermost connection failing: the gateway logs mattermost connect timed out after 30s, next retry in 300s every 5 minutes. Host-side condition (Mattermost unreachable), not a CT450 regression — see systems/messaging-integrations.
  • Soul: Chaos — the unhinged raccoon 🦝 (SOUL.md beat-bank variant, updated 2026-08-01)
  • Cron jobs: 25 — 13 enabled / 12 disabled (verified live in CT450's jobs.json 2026-10-26). 9 job names are shared with CT460; 8 of those are disabled here and enabled on CT460, while Important-mail monitor is the reverse (enabled on CT450, disabled on CT460)
  • Production counterparts: CT460 has 28 jobs / 25 enabled — the two fleets no longer share a job count, and the CT450 duplicates are paused

Providers & Routing

  • Default model: smart-route via custom → http://100.78.63.20:20128 (the omni-101 node) — verified live in CT450's config.yaml 2026-10-26. ⚠ The old claim of deepseek-flash-mix on custom:omniroute (100.88.81.19) no longer matches the config
  • Fallback chain (fixed 2026-08-15 — OmniRoute fallback entries removed, they were non-functional):
    1. kilo-stepfun-step-3_7-flash-free via litellm_4000
    2. nvidia/nemotron-3-ultra:free via nous
    3. auto via custom at https://freellmapi.com/v1
  • Important: fallback_providers must stay a YAML list — hermes config set writes strings, which silently disables the whole chain (_iter_fallback_entries returns [] for strings). Edit via the hermes fallback command or direct YAML edit.
  • See systems/model-providers for full breakdown.

Browser & Desktop Access

  • Browser engine: camofox on port 9377 (config: engine: camofox). Does not expose raw CDP on 9222 — do not point cdp_url at 9222 or browser tools fail with connection refused.
  • VNC desktop: camofox-vnc.service (user unit) on Xvfb :127, RFB 5901, localhost-only no-auth. View at hermesagent.kangaroo-eel.ts.net/vnc.html.
  • CDP endpoint for typed-browser ops: bind via cua_browser_prepare/cua_browser_state on the Camofox window; legacy browser_navigate works for plain navigation.
  • See systems/browser-backend for browser architecture.

Web UIs (do not conflate)

  • WebUI: port 8787 (Python/vanilla JS)
  • hermes-ui: port 5174 (Vite/React desktop-style renderer, proxies to gateway 9119)
  • These are separate projects. Deployment of hermes-ui is done via CX-dump + tar through the PVE host.

Phase One Review (2026-08-08)

Phase One seven-day review completed and independently live-verified:

  • CT450 duplicate jobs remained paused with no post-pause execution records
  • CT460 counterparts remain enabled with successful latest runs
  • CT333 schedules remained enabled and healthy
  • Phase Two is ready for consideration but is NOT approved

Key Rules

  • Do not promote changes to production without explicit approval. Chaos is for testing.
  • Duplicate cron jobs are paused. Do not re-enable without approval.
  • Separate state. Memory, skills, and config are independent from CT460.
  • Wiki governance: write only to this section (agents/hermes-chaos.md); never hand-edit auto-generated pages like systems/skills-index.

Relationship to Production

Hermes Chaos (CT450) and Hermes Production (CT460) are separate Hermes instances with:

  • Separate configs
  • Separate skills
  • Separate cron jobs
  • Separate memory state (Chaos runs Hindsight, bank hermes-clean; production runs Mnemosyne)
  • Separate Tailscale