Files
agent-estate-wiki/runbooks/restart-hermes.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

4.8 KiB

title, type, status, created, updated, verified_on, last_tested, confidence, tags, sources
title type status created updated verified_on last_tested confidence tags sources
Restart Hermes runbook active 2026-07-22 2026-11-11 2026-11-11 2026-10-20 high
runbook
hermes
restart
raw/configs/hermes-config-sanitized.txt

Restart Hermes

Purpose

Restart the Hermes agent, gateway, dashboard, and browser helper processes cleanly.

Symptoms that match this runbook

  • Hermes stops responding in chat or WebUI
  • Tools fail mid-session but the agent is otherwise running
  • After an update or config change

Prerequisites

  • User account hermes with access to systemd --user
  • No active backups in progress
  • If using a browser session, warn the user before restarting dashboards

Procedure

1. Identify the active Hermes processes

ps aux | grep -E "hermes_cli.main (gateway|dashboard) run|hermes-webui/server.py|cua-driver serve|x11vnc.*5901" | grep -v grep

Expected active pieces on this install (live ps 2026-10-20):

  • gateway: hermes gateway run → /home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/bin/python3 … gateway run (unit hermes-gateway.service)
  • dashboard: hermes dashboard --host 0.0.0.0 --port 9119 --no-open (unit hermes-dashboard.service)
  • webui: /home/hermes/.hermes/hermes-agent/venv/bin/python /home/hermes/hermes-webui/server.py (port 8787)
  • camofox/cua-driver: /home/hermes/.local/bin/cua-driver serve
  • x11vnc: -display :99 ... 5901

The venv/ path in this runbook is wrong for the gateway and dashboard (verified 2026-10-20). They no longer run from ~/.hermes/hermes-agent/venv/; they run from the uv-managed runtime ~/.hermes/tools/python-3.14.7+20260901-linux-x64/, launched through hermes_cli/main.py with a runpy bootstrap. The hermes launcher at /home/hermes/.local/bin/hermes is a 73-byte shim. Prefer the hermes CLI and the systemd user units over hand-written python -m hermes_cli.main … commands — the units already carry the correct interpreter path and the _hermes_closure_repair() bootstrap that protects against a half-finished hermes update.

Live user units (2026-10-20): hermes-gateway.service, hermes-dashboard.service, hermes-ui.service, hermes-ui-prod.service, hermes-wiki-serve.service — all active (running).

2. Restart the gateway

hermes gateway restart

Equivalent and preferred (survives a broken CLI shim):

systemctl --user restart hermes-gateway.service

3. Restart the dashboard if needed

hermes dashboard restart

If that fails, stop and rerun:

hermes dashboard stop
hermes dashboard --host 0.0.0.0 --port 9119 --no-open

4. Restart the browser helper if needed

pkill -f "cua-driver serve"
/home/hermes/.local/bin/cua-driver serve

5. Restart the WebUI if needed

Find its PID, then restart standalone:

pkill -f "hermes-webui/server.py"
/home/hermes/.hermes/hermes-agent/venv/bin/python /home/hermes/hermes-webui/server.py

Steps re-checked live 2026-11-11. All five user units are active (running): hermes-gateway.service, hermes-dashboard.service, hermes-ui.service, hermes-ui-prod.service, hermes-wiki-serve.service. The gateway runs from the uv-managed runtime (/home/hermes/.hermes/tools/python-3.14.9+202****1021-linux-x64/bin/python3 … gateway run) through the runpy bootstrap that carries _hermes_closure_repair(); the dashboard is hermes dashboard --host 0.0.0.0 --port 9119 --no-open; the WebUI is still /home/hermes/.hermes/hermes-agent/venv/bin/python /home/hermes/hermes-webui/server.py (port 8787); cua-driver serve is running (PID 4176, cua-driver 0.28.2); x11vnc is on :99/5901. Note the runtime date-stamp in the interpreter path has advanced (…14.9+20261021… vs the …14.7+20260901… recorded on 2026-10-20) — always resolve the path live, never hardcode it.

Verification

hermes gateway status
hermes dashboard status
ss -tlnp | grep -E "8787|9119|5901|6081"
ps aux | grep -E "gateway run|dashboard|cua-driver serve|server.py" | grep -v grep

Current known-good ports after verified restart:

  • gateway: active via hermes-gateway.service
  • dashboard: 0.0.0.0:9119
  • webui: 0.0.0.0:8787
  • VNC: 5901
  • noVNC: 6081

Rollback

  • If restart fails, review recent config changes and consider restoring config.yaml.bak.*
  • Do not delete old config files without explicit confirmation
  • Re-run backup first if you plan to rollback config

Notes

  • Last tested: 2026-07-22 on debian 13, user-mode systemd
  • WebUI logs: /home/hermes/.hermes/webui/bootstrap-8787.log
  • Defunct <defunct> processes in process list are expected zombie shells from prior runs and do not require action unless they pile up