23 KiB
title, type, status, created, updated, verified_on, confidence, tags, sources
| title | type | status | created | updated | verified_on | confidence | tags | sources | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hermes Agent | system | active | 2026-07-22 | 2026-11-08 | 2026-11-08 | high |
|
|
Hermes Agent
Purpose
Primary personal agent for Anthony. Runs automation, research, communication, knowledge management, browser control, and scheduled tasks.
Current Status
- Version: ⚠
0.23.0— updated 2026-11-07 05:05:13 → 05:43:10 AWST byhermes update(methodgit, outcomesuccess, 37 m 57 s):bf0203a794a07aac3ff764972ab9087baa4e4f4e→2bd378ed1b8d6bf2192cb26e0a55ff3091e6bd19(upstream commit dated 2026-11-06 03:57:38 AWST). ⚠ Thesnapshotstage was skipped again —disabled by updates.pre_update_backup (mode: off), so there is still no rollback point (infrastructure/backups). Restarted:hermes-gateway(pre-updatepid 2981474) andhermes-dashboard(Stopping05:43:03 →Started05:43:04,:9119). The gateway was then restarted again 11:11:44 → 11:11:55 (live processpid 3109529, seen logging 11:12:30 and 12:01:32). Side effect of the same night: two cron runs lost their fire claims in the restart drain — incidents/2026-11-07-cron-claim-loss-in-gateway-restart-drain. Previous:0.22.3— install stamp5b147ab036d6d19131bff8d20701b5001eab844a, built 2026-10-21T14:57:38Z; git-describe tokenv0.21.5+15956.g5b147ab(detached HEAD). Last update: 2026-10-21 22:47:25 → 22:57:38 AWST viahermes update(action9eed3a99e799347c691ea52a72e122a1, receiptupdate_20261021_144738_866674_82212ebd…): stagesplan→snapshot(skipped) →apply(mode git) →deps→build, all success except the skip. The pre-update tree read0.22.2/d923a5eb487a17c0a68801bba9ef0f56c81040a8— an intermediate rebuild never recorded on this page, so the previously-notedv0.21.5+6169.gab86a20(2026-10-04) is at least two builds behind. Runtimes in the plan:gateway(systemd) anddashboard(manual-serve, restarted by argv). Post-restart state: gateway pid 868087,phase: running,api_server: connected(heartbeat 2026-10-22 00:13 AWST); dashboard pid 869852 on:9119. - Install method: git
- Python: managed runtime 3.14 (legacy
venv/on 3.11.15) - Upstream status: advanced 2026-10-21 to commit
5b147ab036(detached HEAD, package version 0.22.2 → 0.22.3). ⚠ That update ran with its own safety net switched off: thesnapshotstage is recorded asskipped, reasondisabled by updates.pre_update_backup (mode: off)— the samepre_update_backup: falserecorded in infrastructure/backups, while the off-box config backup push has been dead since 2026-08-08. Two advisories were printed after the run: rebuild the search index (hermes sessions optimize-storage— "on the compact layout from an earlier optimization, but a later release changed it"), and→ Preparing pinned cua-driver (Computer Use)..., whose completion line is absent from the log — status unverified. The.restart_pending.jsonrequest (via_service: true, requested 2026-10-21 22:48:42 AWST) was delivered to telegram, discord and ntfy. Neither advisory has been acted on. - Install directory:
/home/hermes/.hermes/hermes-agent - Managed runtimes: 3 uv-managed venvs under
~/.hermes/installs/019d0d114d22b7e6/environments/<hash>/venv(was 10 — the 2026-10-03 rebuild pruned superseded environments and created two new ones at 09:59/10:00; the third survives from 2026-09-28).hermes-runtime-deps-guard.shruns every 15 min viamnemosyne-watchdog.pyand now assertsdotenv+mnemosynein the installer python and every managed venv, plusfastembed+FASTEMBED_CACHE_PATH+ the local model dir in the bootstrap-selected venv — see incidents/2026-09-28-hermes-managed-runtime-deps and incidents/2026-10-03-mnemosyne-vector-recall-fts-only. - Community resources: Hermes Registry — third-party plugins, skills, and extensions
- Binary:
/home/hermes/.local/bin/hermes - Config:
~/.hermes/config.yaml - Profile:
default(top-level config~/.hermes/config.yaml; the only named sub-profile on this host isopsat~/.hermes/profiles/ops/)
Location
- Host: Local Linux host (no containerized Hermes runtime confirmed)
- Filesystem root:
~/.hermes/ - Working directory:
/home/hermes/workspace
Providers & Routing
- Default provider:
custom→http://100.78.63.20:20128(OmniRoute) — verified 2026-09-30 - Default model:
smart-route— verified 2026-09-30 - History: as at 2026-07-22 this was
custom:litellm→litellm:4000with modelfreellmapi-auto - Provider model count (2026-10-01): 5 custom providers —
omni-1015,384,Omniroute:201285,398,litellm1,728,command-code71,guacano46 - MCP servers: 5 configured (
Not human search,asana,cua-driver,hound,mcp-portal) — verified 2026-10-01 ⚠ Auth state re-verified 2026-11-01:asanais enabled but has never been authenticated.hermes mcp listshows it ✓ enabled (https://mcp.asana.com/v2/mcp,auth: oauth), but~/.hermes/mcp-tokens/holds onlyasana.client.json(319 B, mtime 2026-10-26 01:07) — no token file — so every attempt logsMCP OAuth setup failed for 'asana': … non-interactive environment and no cached tokens found. Run \hermes mcp login asana` interactively first`. The other four servers are unaffected → incidents/2026-10-31-asana-access-failures - Plugins enabled: 11 — verified 2026-10-01
- free-stack combo (OmniRoute): Nemotron 3 Ultra 550B configured with
max_tokens: 8192,reasoning_effort: medium,chat_template_kwargs.force_nonempty_content: true(see omniroute) - See model-providers for full breakdown.
Memory
- Backend: Mnemosyne (local native)
- Memory enabled: true
- Char limit: 5000
- User profile enabled: true
- User char limit: 3000
- Flush every: 6 turns
- Auto sleep: true
- Default scope: session
- Vector type: int8
- Shared surface:
data/shared/mnemosyne.db - See memory-backend for details.
Browser
- Cloud provider: Camoufox
- Managed persistence: true
- Inactivity timeout: 120 s
- Allow private URLs: true
- cua-driver: v0.9.0 at
/home/hermes/.cua-driver/packages/releases/0.9.0-x86_64-unknown-linux-gnu/cua-driver - DISPLAY:
:102(Xvfb/X11VNC/fluxbox confirmed working per session notes) - See browser-backend for details.
Terminal
- Backend: local
- Working directory:
/home/hermes/workspace - Timeout: 180 s
- Home mode: auto
- Container CPU: 1
- Persistent shell: true
- See terminal-backend for details.
Skills
- Total installed: 344
SKILL.mdfiles across 72 top-level skill directories (live count 2026-10-03; re-confirmed by the manual stocktake the same day — the last generated skills-index also reads 344 / 70 categories) - Newest addition:
fleet-wide-failure-triage(devops) — surfaced by the 2026-10-03 regeneration - Key skills: hermes, hermes-gateway-troubleshooting, hermes-mcp-troubleshooting, hermes-api-vault, omniroute, job-search-automation, weekly-review, fresh-rss-intelligence-filter, etc.
- See skills-index for categorized list (auto-regenerated daily — but the generating job is currently blocked, so it is being refreshed by hand; see Known Issues).
Multi-Profile Multiplexer
The default gateway runs as a profile multiplexer (gateway.multiplex_profiles: true), serving both the default and ops profiles through a single gateway process with one Telegram connection.
Profiles
| Profile | Chat | Purpose |
|---|---|---|
| default | -1004321904721 (Rhino chat) |
Main agent |
| ops | -1003914987043 (Elephant ops), -1003932503629 (OPS Tember rescue) |
Operations/infra agent |
How It Works
- One gateway, one Telegram bot token — shared across profiles
profile_routesmaps chat_id → profile at the gateway levelallowed_chatson both profiles must include ALL chats that either profile should hear, because the default profile's adapter processes all inbound messages beforeprofile_routesstampssource.profile- The ops profile does NOT need
platforms.telegram.enabled— the default adapter handles it
Critical Rule: allowed_chats Is Pre-Route
The allowed_chats gate runs on the default profile's adapter before profile_routes matching. If a chat_id is only in the ops profile's allowed_chats but not the default's, messages from that chat are silently dropped before routing ever fires.
Fix applied 2026-07-28: Added both ops group IDs to the default profile's allowed_chats and group_allowed_chats (comma-separated). See multiplexer-setup.
Messaging Integrations
- Telegram: enabled,
allowed_chats: -1004321904721,-1003914987043,-1003932503629, rich messages - Discord: enabled, streaming true
- Slack: enabled
- Email: ⚠ partially connected — personal Gmail works; the
workOutlook account is unreachable (tenant allows onlyXOAuth2, the account uses password auth). See incidents/2026-08-26-work-outlook-basic-auth-disabled. - Mattermost: connected; gateway warning on 2026-08-15 was caused by a disabled system-level
hermes-gateway.serviceunit, not a live duplicate gateway. Only the user-level service is active. - ntfy: connected
- Google Chat: available (
hermes-google_chat) - QQ: available (
hermes-qqbot) - Yuanbao: available
- See messaging-integrations for details.
Scheduled Tasks
- Cron jobs: 30 total (27 enabled, 3 disabled) — verified 2026-10-03
- Failing: 6 —
dca8482e4f76(403 plan-gated),1a8e90ea2271(HTTP 410, retired model),fad5a86979d4(HTTP 400),0e753e9720fd(HTTP 400),06a0a1c8704fand1b23c042e17d(HTTP 401 — thecliprox/terraprovider connection auth expired, now confirmed on 2 of the 3 jobs sharing that pin) - Delivery targets: Telegram (primary), Discord, origin, local
- Notable: Birthday checker, job radar, newsletter digests, GitHub trending, mnemosyne health/watchdog/backup, AgentMail watcher, skills stock-take regeneration
- See scheduled-tasks for full table.
Dependencies
- LiteLLM proxy at
litellm:4000 - OmniRoute at
omniroute:20128 - cua-driver binary at path above
- Xvfb + X11VNC + fluxbox for headed browser
- Telegram bot token, Discord token, etc. (stored outside wiki)
Known Issues
- Managed-runtime venvs previously drifted on dependencies, breaking cron workers — now guarded every 15 min. See incidents/2026-09-28-hermes-managed-runtime-deps.
- Vector recall silently ran keyword-only until 2026-10-03 (repaired).
fastembedwas missing from every uv-managed runtime andFASTEMBED_CACHE_PATHwas unset, so the dense leg ofmnemosyne_recallreturned no candidates and recall degraded to FTS5 — with no error raised. Repaired 2026-10-03 and the guard extended (above). Live coverage 2026-10-04: working 726/752 (96.5%), episodic 1033/1039 (99.4%). See incidents/2026-10-03-mnemosyne-vector-recall-fts-only. - Four cron jobs cannot self-recover — three pinned to models that no longer exist upstream, plus a credential fault:
1a8e90ea2271→qwen3.5:397bretired 2026-09-25 (HTTP 410);dca8482e4f76→gpt-5.6-terraplan-gated (403MODEL_NOT_IN_PLAN);fad5a86979d4+0e753e9720fd→deepseek-v4-flashabsent from the NVIDIA live catalog (HTTP 400); and06a0a1c8704f+1b23c042e17d→ thecliprox/terraprovider connection auth expired (HTTP 401), now confirmed on 2 of the 3 jobs sharing that pin. Repinning to thesmart-routealias is the durable fix for the model faults; the cliprox pair needs the OmniRoute connection re-authenticated first. See systems/scheduled-tasks, systems/model-providers and incidents/2026-10-01-cliproxy-connection-auth-expired. - The daily skills stock-take is not running. Job
1a8e90ea2271is blocked on the retired model above, so skills-index and the freshness audit are only refreshed whenwiki-maintain.shis run by hand (last done 2026-10-03). The script itself is healthy — only the scheduled wrapper is broken. - Telegram thread delivery failures — root-caused and fixed 2026-09-30. A bare
deliver: "telegram"resolves to the home DM (1793951355), where group forum topic IDs cannot exist; the send hard-failed and the message was lost. Two jobs were retargeted totelegram:-1004321904721:102and verified. Three jobs still use baretelegramtargets. See systems/messaging-integrations. - Two wiki trees drift:
/home/hermes/workspace/agent-estate-wiki(Gitea + Quartz source, pages at root) and/home/hermes/wiki/agent-estate-wiki(Obsidian/Ignis vault, pages underwiki/). They are separate git repos with different history. See current-state. - Vault repo git objects can be unwritable:
/home/hermes/wiki/.gitstill holds root-owned object fanout dirs/objects (04,42,82,91) left by root-run maintenance commits on 2026-07-22. When a new blob or tree hashes into one of those fanouts,git commitdies withinsufficient permission for adding an object to repository database(even thoughgit add/write-treelook fine). Workaround that works: build the objects in a writable store viaGIT_OBJECT_DIRECTORY=<tmp>+GIT_ALTERNATE_OBJECT_DIRECTORIES=/home/hermes/wiki/.git/objects, thengit pack-objects /home/hermes/wiki/.git/objects/pack/pack-<name>and move the branch withgit update-ref refs/heads/master <commit>. Permanent fix is a rootchown -R hermes:hermes /home/hermes/wiki/.git. Do not run overwrite/rename probes against.git/objects: on 2026-09-30 one such probe overwrote loose tree0d23eba37eb65f5e5021f558a370358f1e394c58(breaking the 2026-07-22 commit9141a88); it was reconstructed byte-exactly from its parent treed8185c99plus the two replaced children (comparisonstree822b90a7,log.mdblob043dc1cd) and reinserted, andgit fsckis clean again. - Browser persistence not yet fully verified
Related
- model-providers
- memory-backend
- browser-backend
- terminal-backend
- skills-index
- messaging-integrations
- scheduled-tasks
- current-state
2026-10-10 - gateway runtime env, TTS, Paseo
- Gateway process shape (verified 2026-10-11 00:20): PID 230494, started 2026-10-10 21:24:44,
launched as
/home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/bin/python3 -I -c '<bootstrap>' gateway run(RSS ~371 MB).activate_dependencies()replacessys.pathwith a single committed uv environment (~/.hermes/installs/<id>/environments/795020dbf24343cd9c52df24723dca83/venv), and the user site directory is invisible to it - packages the gateway needs must be installed into that env. See incidents/2026-09-28-hermes-managed-runtime-deps. - Mnemosyne provider healthy again:
Memory provider 'mnemosyne' registered (40 tools)+activatedfrom 2026-10-10 22:01:19 onward (repeated 23:59 / 00:00 / 00:02);hermes memory statusreports available. - TTS: ElevenLabs is the configured provider; its package was missing from the live managed env
and was reinstalled 2026-10-10.
voice.auto_tts: false- auto-speak is off by design, so TTS output appears only when explicitly requested.- ⚠ Superseded twice — live config checked 2026-10-30:
tts.provideris nowollama_local(~/.hermes/scripts/ollama-tts.sh, modelphonon) andstt.provideris nowlocal(faster-whisperbase);voice.speech_server.urlis nowwss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/realtime(the bare/v1/answers 403, and the plugin dials the value verbatim). See the 2026-10-30 section below. - ⚠ Superseded — live config checked 2026-10-28 (change date not established):
tts.providerisopenaipointed at the Mac mini speech server —tts.providers.openai.base_url=https://anthonys-mac-mini.kangaroo-eel.ts.net/v1/speech,model: phonon,voice: alloy. Live voice (as opposed to one-shot TTS) runs overvoice.speech_server.url=wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/, withwake_word.enabled: true. Acommand-type providerollama_local(~/.hermes/scripts/ollama-tts.sh) remains configured as an alternative.tts.elevenlabs(eleven_flash_v2_5) andtts.gemini(gemini-2.5-flash-preview-tts, voiceCharon, persona file~/.hermes/hermes-tts-persona.txt) are still present as provider blocks.voice.auto_ttsis stillfalse. See systems/conduit for the voice path.
- ⚠ Superseded twice — live config checked 2026-10-30:
- Mattermost platform unreachable since 2026-10-10; the gateway keeps retrying on a 5-minute cadence - systems/messaging-integrations.
- Paseo, the coding-agent orchestrator on
0.0.0.0:6767, is documented at systems/paseo.
2026-10-30 00:0x AWST — the Conduit voice path: a truncated URL, a 12-hour session TTL, and two dead providers
Anthony asked why Conduit voice calls never got an answer. Three faults sat in series — each one masked the next — and all three were fixed on 2026-10-29 between 08:02 and 08:4x AWST:
| Fault | Was | Symptom | Now |
|---|---|---|---|
voice.speech_server.url |
wss://…:8443/v1/ |
dialled verbatim → HTTP 403, reported as "the speech server refused the key" | wss://…:8443/v1/realtime |
dashboard.basic_auth.session_ttl_seconds |
43200 (12 h default) |
expired phone session → WS 4401 "Dashboard sign-in required", no Hermes turn, no error on the device | 2592000 (30 d, matched to the refresh-token life) |
tts.provider / stt.provider |
openai / groq |
Mac /v1/speech 502 / 403 Cloudflare 1010 |
ollama_local / local |
- Why it stayed invisible: the Mac recorded zero connections,
conduit-calls.json → historywas empty, and each attempt logged a reason string pointing at the next fault — so one broken chain looked like three unrelated problems.last_request_atwasnull: no call had ever reached Hermes' turn logic. - The 12 h TTL hid behind the others. The dashboard had stopped issuing long-enough sessions, and the iPhone only re-authenticated when the dashboard happened to prompt it — an audit-log entry shows a re-auth at 08:00:59, minutes after a silent call.
- Two TTL keys exist.
dashboard.basic_auth.session_ttl_secondswas raised; the similarly nameddashboard.session_ttl_secondsstill reads 43200. Check both on any future 4401. - Groq was blocked, not mis-keyed — it answers 403/1010 with a bogus key too, so this was Cloudflare bot-blocking rather than a credential problem.
- Verified for real: the whole path was driven (mint HMAC dashboard session → WS ticket → let the plugin dial the Mac),
then two live calls landed at 2026-10-29 08:38:04 and 08:49:36 — the first entries ever in
conduit-calls.json → history.ollama_localproduced 6.94 s of real 64 kbps speech;localSTT returnedsuccess: True. - Backups:
config.yaml.bak.url-081525,config.yaml.bak.ttl-20261029080258,config.yaml.bak.stt-084536; the dashboard restarted 08:03:26 → incidents/2026-10-29-conduit-voice-silence · systems/conduit.
2026-10-26 00:1x AWST — gateway restart record and the dead Nous credential
- Three gateway starts on 2026-10-25:
00:32:20,18:30:04,19:27:13AWST (after2026-10-24 12:24:39). The18:29:22shutdown carriedsignal=SIGTERM under_systemd=yes,loadavg_1m=3.79— a planned stop, not a crash;19:26:16logged "Stopping gateway for restart..." (the plugin self-update path). Current gateway pid 2476414, up since 19:26:33 (≈4 h 45 m at audit time). - Consequences of the 19:27 start: the Conduit dashboard gateway restarted at 18:42:33 and so picked up the
plugin's upstream commits → systems/conduit; the Mattermost reconnect attempt counter reset to 54
→ systems/messaging-integrations; the
invalid_grantspam stopped. - ⚠ The Nous Portal credential is dead —
Nous Portal runtime credentials unavailable: Invalid refresh token. Gateway-log counts: 1 line 2026-10-22, 3 on 10-24, 20 on 10-25 (last 14:53:04,agent.context_compressor: Failed to generate context summary), 0 since the 19:27:13 start. Context compression was unavailable for most of 10-25 and contributed to the digest runaway. → incidents/2026-10-22-nous-portal-refresh-token-invalid - Restart hygiene: a gateway restart resets in-process counters (Mattermost retries, cron admission bookkeeping). Never read a counter as an outage duration.
2026-11-01 00:2x AWST — gateway reloads on 10-31 (one force-drained, one clean) and the Asana MCP auth gap
- Two gateway starts on 2026-10-31, and the first one killed a live cron run.
- 18:06:41 —
SIGUSR1 received (systemctl reload / hermes gateway restart): "Restart requested with 2 active work unit(s); deferring stop() until they finish (cap=1800s)". The countdown ran every 30 s from 1770 s to 988 s (18:20:13) against a singlemanagedunit (pid 397); a secondSIGUSR1landed at 18:11:23. The cap expired at 18:37:38 → "Skipping.clean_shutdownmarker — drain timed out with interrupted agents". New gateway 18:38:43. The 12:01:06 slot ofea79a2107e36(Mnemosyne Health Monitor) closedfailedat 18:39:15 on "lost its durable fire claim ownership" after 6 h 38 m → incidents/2026-10-31-gateway-reload-force-drain-orphaned-run. - 19:23:39 → 19:24:40 — a second restart, clean:
drain took 0.00s, timed_out=False, "Previous gateway exited cleanly — skipping session suspension". Current gateway PID 175269, active since 19:23:56 (4 h 43 m at the audit probe). Nothing failed after 19:15:21.
- 18:06:41 —
- Two Telegram adapter rebuilds, both self-recovered:
05:03:42and09:12:11—Fatal telegram adapter error (telegram_network_error): Telegram updater.stop() did not finish before the network-recovery deadline; rebuilding the adapter instead of reusing an Updater whose lifecycle lock may still be held. (likely CLOSE-WAIT socket). - Plugin update machinery was active minutes before the reload (
.update-<id>/plugincapability checks 18:05:41 and 18:19:53,plugin scan passed18:20:22) and the 04:56:54 log line advertised updates forclaude-subscription-directsdk-experimental,conduit_pushandhomeassistant. Agent version unchanged —hermes-agentHEAD is stillbf0203a7(2026-10-22). - MCP inventory unchanged at 5 servers, but
asanahas never authenticated (see the MCP line above and incidents/2026-10-31-asana-access-failures): 7 728MCP OAuth setup failed for 'asana'log lines in the retained window (≥2026-10-21), 1 156 of them on 10-31.