wiki-maintain.sh ran clean (302 skills indexed, Quartz rebuilt, 634 files emitted, service restarted). The freshness report listed 8 stale pages; each was diffed against live state and corrected only where reality could be verified. A second script run rewrote the same-day report with 0 stale / 103 fresh / 4 no-date, so the committed report matches the tree it ships with. - infrastructure/hosts.md: CT600 is LIVE (not retired); CT601 does not exist on either hypervisor; CT333 verified running; both hypervisors verified; config 33->51; skills 66->302; cron 25->28 (25 enabled / 3 disabled); Xvfb :102->:99; providers split into Omniroute:20128 (CT600) and omni-101 (CT101, default). - agents/hermes-production.md: Tailscale IP 100.96.100.82 -> 100.118.5.51 (the old value is the PVE hypervisor); v0.21.5 -> v0.22.3; dead 8082 claim removed; fallback chain rewritten from live config (first entry is the dead nous credential); all 8 auxiliary slots now no-think/freeapi/auto; cron 30->28. - agents/hermes-chaos.md: v0.22.3 (same build as CT460); "3 commits behind" -> 0 behind origin/master; default model smart-route on custom:omni-101; cron "11 paused" -> 25 jobs (13 enabled / 12 disabled), 9 names shared with CT460. - infrastructure/tailscale.md: node table rebuilt from status --json; octoagent and omarchymac-1 were missing entirely (52 -> 54 non-funnel nodes); m93p now online; every offline age refreshed; omni row corrected to "CT601 removed from both hypervisors"; Serve tables re-probed per host. - systems/memory-backend.md: re-verified against live config; coverage 1899/2350 (80.8%), working 96.8%, episodic 99.4% — above the 95% reindex threshold. - runbooks/provider-health.md: the step-3 OmniRoute snippet raised IndexError (provider renamed omniroute -> Omniroute:20128; key moved to key_env) — fixed; new step 3b documents the omni-101 /v1/models false-failure quirk. - runbooks/update-hermes-safely.md: added managed-runtime dependency re-assert step; clarified --branch vs stable channel (v0.22.5 available). - incidents/2026-10-03-ct450-hermes-update.md + incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md: Current-state sections recording current versions and embedding coverage. All 9 corrected pages verified 200 at http://127.0.0.1:9120/<path>. Vault mirror at /home/hermes/wiki synced for all touched files (22 copies, sha256-verified) and committed locally; the vault is local-only and is not pushed. No secrets written — IDs, URLs, key env var names, counts and timestamps only.
138 lines
5.0 KiB
Markdown
138 lines
5.0 KiB
Markdown
---
|
|
title: Update Hermes safely
|
|
type: runbook
|
|
status: active
|
|
created: 2026-07-22
|
|
updated: 2026-10-26
|
|
verified_on: 2026-10-26
|
|
last_tested: 2026-10-26
|
|
confidence: high
|
|
tags: [runbook, hermes, update]
|
|
sources: [raw/configs/hermes-config-sanitized.txt]
|
|
---
|
|
|
|
# Update Hermes safely
|
|
|
|
## Purpose
|
|
Update Hermes to a newer release without losing configuration, skills, or sessions.
|
|
|
|
## Symptoms that match this runbook
|
|
- User wants to move to a newer Hermes version
|
|
- `hermes version` shows an older build than expected
|
|
- Update-related errors after manual changes in the install directory
|
|
|
|
## Prerequisites
|
|
- Hermes install method: git-based install at `/home/hermes/.hermes/hermes-agent`
|
|
- Working Python virtualenv at `/home/hermes/.hermes/hermes-agent/venv`
|
|
- **Managed runtimes live under `~/.hermes/installs/<install>/environments/<hash>/venv`** — these are what the gateway actually imports from; a missing dep there breaks cron jobs with no obvious trace (see [[incidents/2026-09-28-hermes-managed-runtime-deps]])
|
|
- Available disk space in Hermes install and backup paths
|
|
- No active Hermes chat/webui shell issues before starting
|
|
|
|
## Procedure
|
|
|
|
### 1. Create a backup first
|
|
```bash
|
|
hermes backup --quick --label pre-update-$(date +%Y%m%d)
|
|
```
|
|
If you need a full backup instead:
|
|
```bash
|
|
hermes backup --label pre-update-$(date +%Y%m%d)
|
|
```
|
|
|
|
### 2. Check whether an update is available
|
|
```bash
|
|
hermes update --check
|
|
```
|
|
|
|
### 3. Review configuration version and backups
|
|
```bash
|
|
python3 -c "import yaml; print(yaml.safe_load(open('/home/hermes/.hermes/config.yaml')).get('_config_version'))"
|
|
ls -la ~/.hermes/config.yaml.bak*
|
|
```
|
|
|
|
### 4. Run the update with noninteractive flags
|
|
```bash
|
|
cd /home/hermes/.hermes/hermes-agent
|
|
git status --short
|
|
hermes update --yes
|
|
```
|
|
Use `--branch main` only when deliberately tracking a branch other than the configured
|
|
channel. The **stable** channel is currently the default and offers **v0.22.5**
|
|
(`hermes update --check`, verified 2026-10-26; local install is v0.22.3 / `5b147ab036d6`).
|
|
`--force-venv` rebuilds the virtualenv — avoid unless the venv itself is broken.
|
|
|
|
**After any update, re-assert the managed-runtime dependencies** (the updater can
|
|
leave a runtime without `fastembed`/`dotenv`/`mnemosyne`):
|
|
```bash
|
|
~/.hermes/scripts/hermes-runtime-deps-guard.sh
|
|
```
|
|
|
|
### 5. Verify the new version
|
|
```bash
|
|
hermes version
|
|
```
|
|
|
|
### 6. Restart the required Hermes pieces
|
|
```bash
|
|
hermes gateway status
|
|
hermes dashboard status
|
|
hermes gateway restart
|
|
hermes dashboard restart
|
|
```
|
|
|
|
### 7. Sanity check health
|
|
```bash
|
|
hermes gateway status
|
|
curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:8787/health
|
|
```
|
|
|
|
## Verification
|
|
- `hermes version` shows a newer version than before
|
|
- `hermes gateway status` is active
|
|
- WebUI `/health` returns `200`
|
|
- Providers reachable with [[provider-health]]
|
|
|
|
## Rollback
|
|
- Rollback Hermes config:
|
|
```bash
|
|
ls -lt ~/.hermes/config.yaml.bak*
|
|
cp ~/.hermes/config.yaml.bak.<timestamp> ~/.hermes/config.yaml
|
|
hermes gateway restart
|
|
hermes dashboard restart
|
|
```
|
|
- Rollback entire Hermes install requires git revert or reinstall from previous checkout
|
|
- If update warns about config migration, revert the backup and re-run manually
|
|
|
|
## Notes
|
|
- Hermes version on this install: **v0.22.3** (2026.9.24), local `5b147ab036d6`; install tree rebuilt 2026-10-21 22:47→22:57 AWST. Stable **v0.22.5** available as of 2026-10-26
|
|
- Install directory: `/home/hermes/.hermes/hermes-agent`
|
|
- Update preserves venv by default unless `--force-venv` is used
|
|
|
|
## Second instance — CT450 (`hermesagent`), updated 2026-10-03 (v0.19.0 → v0.21.5)
|
|
|
|
CT450 has **no SSH**. Reach it from the PVE host:
|
|
|
|
```bash
|
|
ssh root@100.96.100.82
|
|
pct exec 450 -- <command> # or: pct enter 450
|
|
pct exec 450 -- su - hermes -c "/home/hermes/.local/bin/hermes --version"
|
|
```
|
|
|
|
`pct exec 450 -- hermes ...` fails (`Failed to exec "hermes"`): the binary is not on `lxc-attach`'s PATH. Use the absolute path or `su - hermes -c` (the dash gives a login shell, which sets `~/.local/bin`). A `hermes` login shell also prints a `hermes-setup` banner before output.
|
|
|
|
**Pre-flight before running the update** (each of these cost one failed run on 2026-10-03):
|
|
1. `clang` installed — source-compiled Python dependencies will not build without it (`pct exec 450 -- bash -lc "which clang"`).
|
|
2. No stale `.install.lock` / `.lazy-refresh-incomplete` in the Hermes home.
|
|
3. `~/.hermes/hermes-agent` owned by `hermes:hermes` (a root-owned tree from an earlier root-run update breaks the in-place build).
|
|
4. Host Caddy/`keyring` file permissions sane (`0600` keyring blocked a step mid-update).
|
|
|
|
**After:** `pct exec 450 -- su - hermes -c "/home/hermes/.local/bin/hermes --version"`, confirm CT460 production is untouched (separate config/skills/cron/memory). Full record: [[incidents/2026-10-03-ct450-hermes-update]].
|
|
|
|
## Last tested
|
|
2026-10-26 on debian 13, git-based Hermes install — `hermes update --check` (stable v0.22.5 available), `hermes backup --help` and `hermes update --help` flags confirmed
|
|
|
|
## Related
|
|
- [[restart-hermes]]
|
|
- [[backup-wiki]]
|
|
- [[provider-health]]
|