stocktake(2026-10-11): skills-index 346 skills/70 cats; freshness 62 stale/24 fresh/7 no-date; 5 live-drift corrections

Drift found and corrected against live sources (not guessed):
- OmniRoute: CT600 omniroute is RUNNING and serving :20128 (combo API live);
  CT601 omni is STOPPED with onboot=0 (peer offline 54d). Wiki had these inverted.
  Hermes custom:omni resolves to omniroute -> CT600, not CT601.
- Third OmniRoute node discovered: omniroute-1 at 100.78.63.20:20128, live (401),
  registered as custom provider omni-101 (5384 models, default smart-route,
  carries skills_hub/free-stack). Not in pct list; identity unconfirmed.
- Ignis: TWO live instances. CT460 ignis (:8080, vault wiki/agent-estate-wiki) and
  CT200 ignis-ignis-1 (image ignis-server:latest, host 8092, vault /opt/ignis/vaults).
  Both return HTTP 200. Page now distinguishes them.
- Tailscale: /agent-estate targets bare 127.0.0.1:8769 (was .../8769/agent-estate-role-dashboard.html);
  undocumented tailnet-only :9443 -> 127.0.0.1:5173 added; CT201 routes corrected
  (8085 targets + /steel 5173 + /steel-api 3000, replacing the wrong :8443 -> :8082 row).
- Proxmox: rebuilt from live pct list + qm list. Added CT475 agentzero and VM110 omarchy
  (both previously missing); 282 ubuntu-server-xfce corrected to stopped; dead rows removed.
  VM403 open-claw and VM401 hermes-old absent from qm list - flagged unconfirmed, not deleted.
- Nanobot: gateway nanobot-gateway (PID 2259048) alive on 8765/8766. It is NOT systemd
  nanobot.service/gateway.service (both return inactive - wrong unit names). Port 8900
  no longer listening; "Serve API" claim flagged unverified.
- Docker: old "runtime not available" was a CT460 socket artifact (root:docker srw-rw----,
  hermes not in group docker, no passwordless sudo in cron). Documented CT200 (~50 containers),
  CT333 (9 containers), CT460 via sudo.
- Mattermost CT255 confirmed stopped; matches Tailscale peer offline 21d.

Flagged, not investigated (no root cause):
- mycompose-db-1 on CT200 is in Restarting (1) - crash-looping.

62 pages remain >21d stale and were NOT rewritten this pass (concepts, decisions,
comparisons, most runbooks) - not probed against a live source, so reported not faked.
Quartz rebuild step still inert (no quartz.config.ts) - public/ not refreshed.

Artifacts: systems/skills-index.md regenerated, raw/stocktake/freshness-20261011.txt.
This commit is contained in:
hermes
2026-10-11 03:11:07 +08:00
parent 7bffeab157
commit d981e30128
11 changed files with 318 additions and 70 deletions
+22 -8
View File
@@ -3,8 +3,8 @@ title: Nanobot
type: agent
status: active
created: 2026-07-22
updated: 2026-08-17
verified_on: 2026-08-17
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [agent, nanobot, homelab]
sources: [live-probe, runtime-config]
@@ -20,14 +20,28 @@ Primary homelab AI agent running on `clawtest` (CT333). Handles agent orchestrat
- **Workspace:** `/home/nano/.nanobot/workspace`
## Current state
- **Status: ACTIVE.** Confirmed running as of 2026-08-17.
- **Gateway:** port `8765` (HTTP API + chat completions)
- **WebSocket bridge:** port `8766`
- **Serve API:** port `8900` (host `0.0.0.0`, requires `api_key` in config)
- **WhatsApp Bridge:** `whatsapp-bridge.service` active on ct333
- **Model preset:** `freellmapi-auto` (current session); baseline context window 200,000 tokens
- **Status: ACTIVE.** Verified 2026-10-11: gateway process `nanobot-gateway` (PID 2259048) running and listening.
- **Gateway:** port `8765` (HTTP API + chat completions) — `LISTEN 0.0.0.0:8765`
- **WebSocket bridge:** port `8766` — `LISTEN 0.0.0.0:8766`
- **Serve API:** port `8900` was recorded as the Serve API; **not listening** in the 2026-10-11 `ss -ltn` check. Treat as unverified/possibly removed.
- **Service unit names:** earlier notes listed `whatsapp-bridge.service` — `systemctl is-active nanobot gateway` and `is-active gateway` are NOT valid here (both returned `inactive`). Nanobot runs as a user service under the `nanobot` user (`systemd --user`, PID 166) plus the `nanobot-gateway` process; it is **not** a system-level `nanobot.service`. Check the gateway port/process, not the systemd unit.
## Confirmed running on CT333 (2026-10-11)
| Process | Owner | Port(s) | Notes |
|---|---|---|---|
| `nanobot-gateway` (PID 2259048) | — | 8765, 8766 | The gateway |
| `mnemosyne` MCP (`mnemosyne mcp --transport std…`) | nano | — | `venvs/mnemosyne-mcp` venv |
| `chroma_query_api.py` | nano | — | started Sep 23 |
| `agentmail-daemon.py` | nano | — | started Sep 23 |
| `readlater` Next dev server | nano | 3210 | started Sep 23 |
Other listeners on CT333 include 8787, 8880, 8899, 8999, 3000, 5050, 8080 and the Tailscale 443.
## Docker on CT333 (verified 2026-10-11)
`steel-browser`, `gotify`, `readlater`, `readlater-extraction-worker`, `kokoro-tts`, `edge-tts`, `onecli-postgres-1`, `tender_brattain`, plus healthy `community-spark` and `quirky_darwin`.
## Cron jobs
Unverified against live Nanobot scheduler on 2026-10-11. Previously recorded:
- `auto-upgrade-freellmapi-safeguard` — daily at 03:30 AWST
- `morning-quota-report` — daily at 07:30 AWST
- `daily-quota-730am` — daily at 07:30 AWST
+11 -7
View File
@@ -3,11 +3,11 @@ title: QwenPaw
type: system
status: active
created: 2026-08-15
updated: 2026-08-15
verified_on: 2026-08-15
confidence: medium
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [qwenpaw, ct465, agent, paseo]
sources: [skill-inspection, config-reference]
sources: [live-probe, config-reference]
---
# QwenPaw
@@ -24,10 +24,12 @@ Multi-agent gateway platform. Runs on CT465 with its own workspace, agent profil
## Current State
- **Status:** Active
- **Status:** Active — `paseo.service` active on CT465, verified 2026-10-11
- **Default chat model:** OmniRoute Smart Route (`auto/smart`)
- **Config format:** JSON (`agent.json` per workspace)
- **Daemon:** `paseo.service`
- **Also listening on CT465:** PVE exporter (`100.96.150.69:9221`), a Python service on `:9876`, and local SMTP `:25`
- **CT400/CT465 identity:** confirmed CT465 = `qwenpaw`, running
## Operating Layers
@@ -58,9 +60,11 @@ qwenpaw daemon status
When adding an OpenAI-compatible provider from Proxmox:
- Verify target by VMID and hostname before writing config
- CT600/`omniroute` is the OmniRoute service at `100.88.81.19:20128`
- CT601/`omni` is a separate guest at `100.93.204.4`
- CT600/`omniroute` is the **live** OmniRoute service at `100.88.81.19:20128` — verified serving 2026-10-11
- CT601/`omni` is a separate guest at `100.93.204.4` — **stopped**, do not use
- A third instance `omniroute-1` at `100.78.63.20:20128` is also live; Hermes registers it as provider `omni-101`
- Query `/v1/models` using the exact API key stored for QwenPaw, not a management key
- See [[systems/omniroute]] for the full live node table
## Related
+15
View File
@@ -14,6 +14,21 @@ sources: [live-config, live-cron-list, live-hermes-version]
# Current State
Updated: 2026-10-11
## Recent Changes (2026-10-11, 03:00 stocktake; live probes 03:00-03:30 AWST)
- **OmniRoute CT600/CT601 roles are INVERTED.** CT600 `omniroute` (100.88.81.19) is **running** and serving `:20128` (combo API returns live combos); CT601 `omni` (100.93.204.4) is **stopped** with `onboot=0` (Tailscale peer offline 54d). Hermes resolves `custom:omni` → `omniroute:20128` → **CT600**. → [[systems/omniroute]]
- **A third OmniRoute node exists:** `omniroute-1` at `100.78.63.20:20128`, live (401 = auth-required), registered in Hermes config as provider `omni-101` (5384 models, default `smart-route`). It carries `skills_hub` (model `free-stack`) — the same provider family implicated in the 2026-10-09 timeout storm. Not present in `pct list`; identity unconfirmed. → [[systems/omniroute]]
- **Two Ignis instances exist, on different hosts.** CT460 `ignis` (port 8080, vault `/home/hermes/wiki/agent-estate-wiki`, serving 200) and CT200 `ignis-ignis-1` (image `ignis-server:latest`, host port 8092, vault `/opt/ignis/vaults`, serving 200). Both healthy; they are separate deployments. → [[systems/ignis]]
- **Tailscale route drift fixed:** `/agent-estate` targets bare `http://127.0.0.1:8769` (page had `.../agent-estate-role-dashboard.html`); a tailnet-only `:9443` → `127.0.0.1:5173` Serve was undocumented. CT201's live routes are 8085-based plus `/steel` (5173) and `/steel-api` (3000) — the old `:8443` → `localhost:8082` Homepage row is wrong. → [[infrastructure/tailscale]]
- **Proxmox inventory rebuilt** from `pct list` + `qm list`: **CT475 `agentzero` and VM110 `omarchy` were missing entirely**; 282 `ubuntu-server-xfce` corrected to stopped; dead rows removed (310, 500, 104, 106, 101). **VM403 `open-claw` and VM401 `hermes-old` are absent from `qm list`** — flagged unconfirmed, not deleted. → [[infrastructure/proxmox]]
- **Nanobot verified on CT333:** `nanobot-gateway` (PID 2259048) alive on 8765/8766. It is **not** a systemd `nanobot.service`/`gateway.service` (both probes return `inactive` — wrong unit names); it runs as the `nanobot` user service plus the gateway process. Port **8900 is no longer listening** — previous "Serve API" claim flagged unverified. → [[agents/nanobot]]
- **Docker services page corrected:** the old "Docker runtime not available" line was a CT460 socket artifact (`/var/run/docker.sock` is `root:docker srw-rw----`; `hermes` user not in group `docker`; no passwordless sudo in cron). Real fleet: CT200 holds ~50 containers, CT333 holds 9. → [[infrastructure/docker-services]]
- **Mattermost CT255 confirmed stopped**, consistent with its Tailscale peer being offline 21d. Root cause of the unreachable endpoint still undetermined.
- **⚠ `mycompose-db-1` on CT200 is in `Restarting (1)`** — crash-looping. Flagged; root cause NOT investigated this pass.
- Stocktake artifacts: skills-index regenerated (**346 skills / 70 categories**), `raw/stocktake/freshness-20261011.txt` = **62 stale / 24 fresh / 7 no-date of 93 pages**.
- **Quartz rebuild still inert** (no `quartz.config.ts`) — `public/` was not refreshed. Carried open item; unchanged from prior days.
- 62 pages remain >21d stale and were **not** corrected this pass: they were not probed against a live source, so they stay flagged rather than rewritten.
## Recent Changes (2026-10-10 -> 10-11, nightly audit of the 2026-10-09 -> 2026-10-10 window; probes 2026-10-11 00:14-00:20 AWST)
_The manual 2026-10-10 stocktake entry follows beneath this one._
+30 -6
View File
@@ -3,11 +3,11 @@ title: Docker Services
type: infrastructure
status: mixed
created: 2026-07-22
updated: 2026-07-22
verified_on: 2026-07-22
confidence: medium
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [docker, homelab, infrastructure]
sources: [raw/configs/hermes-config-sanitized.txt]
sources: [raw/configs/hermes-config-sanitized.txt, live-probe]
---
# Docker Services
@@ -16,8 +16,32 @@ sources: [raw/configs/hermes-config-sanitized.txt]
Containerized services supporting Hermes and the homelab.
## Status
- Docker runtime **not available** on this host (`docker ps` returned unavailable)
- Compose files confirmed present but execution state unverified
Docker runs on **three hosts**, and CT460's socket is not readable by the agent user. This page previously claimed "Docker runtime not available" — that is only true for CT460's socket from the `hermes` user. Real state, verified 2026-10-11:
| Host | Access | Containers running |
|---|---|---|
| CT200 `docker` (`100.96.150.69`) | `pct exec 200 -- docker ps` via PVE | **~50** — main Docker host |
| CT333 `clawtest` (`100.122.189.81`) | `pct exec 333 -- docker ps` via PVE | 9 (steel-browser, gotify, readlater, kokoro-tts, edge-tts, onecli-postgres, …) |
| CT460 `hermes` | `sudo docker` — socket is `root:docker srw-rw----`, `hermes` user not in group `docker`; **no passwordless sudo in cron** | `ignis` (confirmed running; inspect requires root) |
## CT200 live inventory (verified 2026-10-11, ~50 containers)
`guacamole-guacamole-1`, `telegram-bot-api`, `ytdlp-web-player`, `homarr`, `overtchat-searxng`, `searxng`, `khoj-search-1`, `prowlarr`, `overtchat-app`, `postgres-newsku`, `guacamole-postgres-1`, `jellyfin`, `libre-closet`, `apprise-api`, `tt-rss-web-nginx-1`, `tt-rss-updater-1`, `tt-rss-app-1`, `tt-rss-db-1`, `audiobookshelf`, `qbittorrent`, `open-webui`, `readlater`, `readlater-extraction-worker`, `worldmonitor-redis`, `redis-cache`, `nextstep-db`, `nextstep-scheduler-1`, `nextstep-app`, `portainer`, `flaresolverr`, `sshwifty`, `8mblocal`, `save-restricted-bot`, `kokoro-tts`, `edge-tts`, `filebrowser`, `filebrowser1`, `worldmonitor-redis-rest`, `worldmonitor`, `overtchat-redis`, `chromadb`, **`ignis-ignis-1`**, `nc_app_talk_bot_ai`, `excalidraw-frontend`, `nc_app_context_agent`, `portkey-gateway-web-1`, `guacamole-guacd-1`, `lobe-chat`, `mycompose-saas-1`.
**Watch this one:** `mycompose-db-1` is in `Restarting (1)` — a crash-looping database. Flagged for follow-up; root cause not investigated on 2026-10-11.
**Note the duplicate Ignis:** `ignis-ignis-1` on CT200 is a separate deployment from CT460's `ignis`. See [[systems/ignis]].
## CT201 runtipi — Tailscale routes (verified 2026-10-11)
| Host | Path | Target |
|---|---|---|
| `runtipi.kangaroo-eel.ts.net:8443` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
| `runtipi.kangaroo-eel.ts.net` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
| | `/steel` | `http://127.0.0.1:5173` |
| | `/steel-api` | `http://127.0.0.1:3000` |
**Correction:** this page previously said "Port 443 occupied by Traefik on CT201 — Tailscale uses port 8443 there" and listed CT201's `:8443` → `localhost:8082` for Homepage. Live `tailscale serve status` on CT201 shows the target is `127.0.0.1:8085` on both `:8443` and the default host, plus a `/steel` (5173) and `/steel-api` (3000) pair. The `/steel-api` route does not strip the path prefix — use the container IP directly for API calls.
## Confirmed Compose Files
+32 -29
View File
@@ -3,7 +3,8 @@ title: Proxmox VE
type: infrastructure
status: active
created: 2026-07-26
updated: 2026-08-13
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [proxmox, virtualization, infrastructure, homelab]
---
@@ -20,43 +21,45 @@ Hypervisor hosting all VMs and LXC containers for the homelab.
- **Primary user:** `Anthony@pam` (full admin)
- **Additional users:** `root@pam`, `glance@pve`, `prom-exporter@pve`, `homepage@pve`
## VMs (QEMU)
| VMID | Name | Status | Purpose |
|---|---|---|---|
| 101 | debianvm | destroyed | Debian VM — destroyed; services migrated to CT200 |
| 100 | homeassistant | running | Home Assistant |
| 282 | ubuntu-server-xfce | running | Ubuntu desktop — Sunshine, Kubuntu |
| 403 | open-claw | running | OpenClaw agent |
| 401 | hermes-old | stopped | Legacy Hermes VM (deprecated) |
## Live LXC Inventory (verified 2026-10-11 via `pct list`)
## LXC Containers
| VMID | Name | Status | Purpose |
|---|---|---|---|
| 200 | docker | running | Main Docker host — 42+ containers |
| 103 | cliproxyapi | running | Cliproxy API |
| 200 | docker | running | Main Docker host — 50+ containers (CT460's docker.sock is NOT accessible to the agent user; use CT200 for fleet inspection) |
| 201 | runtipi | running | Runtipi — Homepage, ntfy, etc. |
| 205 | ollama | running | Ollama LLM inference + Guanaco proxy (v0.8.14, updated 2026-08-14) |
| 207 | webserver | running | Web server — Vercel AI, manifest router |
| 205 | ollama | running | Ollama LLM inference |
| 207 | webserver | running | Web server |
| 209 | cloudreve | **stopped** | Cloudreve (deprecated) |
| 215 | n8n | running | N8n automation |
| 221 | gitea | running | Gitea git server |
| 241 | ntfy | running | Ntfy push notifications |
|| 245 | openweb | running | Open WebUI (Jul 30 updated, memory 4096 MB) |
| 255 | mattermost | running | Mattermost chat |
| 245 | openweb | running | Open WebUI |
| 255 | mattermost | **stopped** | Mattermost — unreachable since ~2026-09-20 (Tailscale peer last seen 21d ago) |
| 260 | freshrss | running | FreshRSS feed reader |
| 270 | nextcloud | running | NextCloud |
| 299 | sftpgo | running | SFTPGo file server — security review 2026-08-09: connects as root, unpinned fingerprint, `//` root path. **Incident 2026-09-02:** tailscaled inside CT299 died; PVE pre-start hook then failed because stale `lxc-attach` PIDs pinned the cgroup. Cleared stale PIDs (3037951/3334928/3352203/3402511/4004148), `pct stop 299 --skiplock` + `pct start 299`. Restored, tailscale direct 192.168.178.92:41641. CT299 swap is over-provisioned (43Gi swap fully used at 24Gi/31Gi RAM) — consider reducing swap or trimming workload. |
| 310 | monitoring | stopped | Monitoring stack (deprecated) |
| 270 | nextcloud | running | Nextcloud (apache2 active) |
| 299 | sftpgo | running | SFTPGo file server |
| 333 | clawtest | running | ClawTest — Nanobot, Zeroclaw |
| 450 | hermesagent | running | Hermes Agent LXC |
| 460 | hermes | running | Hermes CT460 |
| 465 | qwenpaw | running | QwenPaw LLM |
| 500 | litellm | running | LiteLLM proxy |
| 555 | elementsynapse | stopped | Element Synapse (deprecated) |
| 600 | omniroute | stopped | OmniRoute model router — retired 2026-08-13, migrated to CT601 |
| 601 | omni | running | OmniRoute model router — successor to CT600, since 2026-08-13 |
| 103 | cliproxyapi | running | Cliproxy API |
| 104 | rclone | running | Rclone sync |
| 106 | glance | stopped | Glance dashboard |
| 209 | cloudreve | stopped | Cloudreve (deprecated) |
| 450 | hermesagent | running | Hermes Agent CT450 (Chaos Lab) |
| 460 | hermes | running | Hermes CT460 (production) |
| 465 | qwenpaw | running | QwenPaw + Paseo daemon (active) |
| 475 | agentzero | running | AgentZero (`100.106.54.94`) — **not previously recorded** |
| 555 | elementsynapse | **stopped** | Element Synapse (deprecated) — also retired from Tailscale (offline 81d) |
| 600 | omniroute | **running** | OmniRoute — live server on `:20128` (see [[systems/omniroute]]) |
| 601 | omni | **stopped** | Retired successor to CT600 — stopped, `onboot=0`, offline 54d |
## Live QEMU Inventory (verified 2026-10-11 via `qm list`)
| VMID | Name | Status | Notes |
|---|---|---|---|
| 100 | homeassistant | running | Home Assistant (13GB RAM) |
| 110 | omarchy | running | Omarchy (13GB RAM, 106GB disk) — **new, not previously recorded** |
| 282 | ubuntu-server-xfce | **stopped** | Ubuntu desktop |
| 310 | cachyos | **stopped** | CachyOS |
| 403 | open-claw | — | OpenClaw agent; **not present in `qm list`** — it runs elsewhere (Open-Claw VM, see [[infrastructure/hosts]]) |
| 401 | hermes-old | — | Legacy Hermes VM; **not present in `qm list`** — appears decommissioned/removed |
**Not in either list:** VM 403 (`open-claw`) and VM 401 (`hermes-old`) are referenced by the wiki but no longer appear in `qm list`. They have either been migrated off this host or deleted. Status unconfirmed — verify before relying on either entry.
## API Tokens
- `homepage@pve!homepage` — PVEAuditor role, privsep=0 (used by Homepage dashboard)
+47 -6
View File
@@ -3,8 +3,8 @@ title: Tailscale
type: infrastructure
status: active
created: 2026-07-26
updated: 2026-10-10
verified_on: 2026-10-10
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [tailscale, vpn, networking, infrastructure]
---
@@ -16,9 +16,47 @@ WireGuard-based mesh VPN connecting all homelab hosts and providing HTTPS access
## Domain
- **Tailnet:** `kangaroo-eel.ts.net`
- **Primary node:** CT460 (`hermes.kangaroo-eel.ts.net`)
- **Primary node:** CT460 (`hermes.kangaroo-eel.ts.net`, `100.118.5.51`)
- **Runtipi node:** CT201 (`runtipi.kangaroo-eel.ts.net`)
## Tailscale Node Inventory (verified 2026-10-11)
Full node list from `tailscale status`. Offline peers are listed with their last-seen age.
| Node | IP | State |
|---|---|---|
| hermes | 100.118.5.51 | **self** |
| agentzero | 100.106.54.94 | online |
| anthonys-imac | 100.112.123.124 | offline 17d |
| anthonys-ipad | 100.111.198.120 | online |
| anthonys-iphone | 100.100.116.67 | active |
| anthonys-mac-mini | 100.68.34.66 | offline 8d |
| cachyos | 100.124.208.15 | offline 37d |
| clawtest | 100.122.189.81 | online (CT333) |
| cliproxyapi | 100.83.112.94 | online (CT103) |
| cloudreve | 100.111.187.7 | offline 78d |
| conduit | 100.81.79.87 | offline 34d |
| docker | 100.96.150.69 | online (CT200) |
| docker-01 | 100.121.12.31 | online |
| elementsynapse | 100.75.250.37 | offline 81d |
| freellmapi | 100.127.208.75 | online |
| freshrss | 100.83.60.93 | online |
| freshrss-new | 100.110.180.115 | offline 54d |
| gitea | 100.82.100.128 | online (CT221) |
| hermesagent | 100.98.205.112 | online (CT450) |
| homeassistant | 100.68.139.5 | active, exit node |
| litellm | 100.79.29.64 | online |
| m93p | 100.104.51.98 | online |
| mailflow | 100.115.60.92 | online |
| mattermost | 100.98.116.14 | **idle/offline 21d** — CT255 is stopped, matches [[infrastructure/proxmox]] |
| n8n | 100.84.124.30 | online |
| nextcloud | 100.90.102.19 | online (CT270) |
| ntfy | 100.67.153.116 | active |
| ollama | 100.108.44.119 | online (CT205) |
| omarchy-macmini | 100.104.53.105 | offline 8d |
| omarchy-quattro | 100.106.27.27 | offline 40d |
> Note: Tailscale's offline list is not authoritative for container state — cross-check `pct list` on the PVE host.
## CT460 Serve Routes
| Path | Target | Notes |
|---|---|---|
@@ -32,13 +70,16 @@ WireGuard-based mesh VPN connecting all homelab hosts and providing HTTPS access
| `/desktop` | `http://127.0.0.1:6081` | CUA noVNC |
| `/vnc-camofox` | `http://127.0.0.1:6080` | Camofox VNC |
| `/synergy-prep` | `http://127.0.0.1:9123/` | Synergy prep |
| `/agent-estate` | `http://127.0.0.1:8769/agent-estate-role-dashboard.html` | Role dashboard |
| `/agent-estate` | `http://127.0.0.1:8769` | Role dashboard — target is the bare host root, **not** `/agent-estate-role-dashboard.html` (corrected 2026-10-11) |
| `/evening-grid.html` | `http://127.0.0.1:9099/evening-grid.html` | Evening grid |
## CT460 Funnel Routes
Verified against `tailscale serve status` on 2026-10-11.
## CT460 Funnel and 9443 Routes
| Port | Target | Notes |
|---|---|---|
| `:8443/webhook` | `http://127.0.0.1:8085/webhook` | Public webhook Funnel |
| `:8443/webhook` | `http://127.0.0.1:8085/webhook` | Public webhook Funnel (`# Funnel on:` confirmed) |
| `:9443` (tailnet only) `/` | `http://127.0.0.1:5173` | Vite dev server on port 5173 exposed via a **tailnet-only** Serve on port 9443 — was missing from this page, added 2026-10-11 |
## CT201 Serve Routes
| Path | Target | Notes |
+13
View File
@@ -459,3 +459,16 @@ neither ICMP nor TCP; gateway retrying every 5 minutes. Cause not determined.
`incidents/2026-09-28-hermes-managed-runtime-deps.md`, `current-state.md`, `index.md`, `log.md`.
No secrets written; job/incident IDs, paths and tailnet addresses only.
## [2026-10-11] stocktake | Daily maintenance + live drift corrections
- Ran `wiki-maintain.sh`: skills-index regenerated (346 skills / 70 categories), freshness audit written to `raw/stocktake/freshness-20261011.txt` (62 stale / 24 fresh / 7 no-date of 93 pages)
- **OmniRoute CT600/CT601 roles are INVERTED from what the wiki claimed.** Verified via `pct list`, `pct config`, `ss` and `tailscale status`: CT600 `omniroute` (100.88.81.19) is RUNNING and serving `:20128`; CT601 `omni` (100.93.204.4) is STOPPED with `onboot=0`. Hermes resolves `custom:omni` → `omniroute:20128` → CT600, not CT601. systems/omniroute.md rewritten with the live node table.
- **Third OmniRoute node found:** `omniroute-1` at 100.78.63.20:20128 is live (returns 401 = auth-required) and is registered in Hermes config as custom provider `omni-101` (5384 models, default `smart-route`, carries `skills_hub`). Not in `pct list` — identity unconfirmed.
- **Ignis duplicate found:** a second `ignis-ignis-1` container runs on CT200 (image `ignis-server:latest`, host port 8092, vault `/opt/ignis/vaults`), separate from CT460's `ignis` (port 8080, vault `/home/hermes/wiki/agent-estate-wiki`). Both confirmed serving HTTP 200. systems/ignis.md now distinguishes them.
- **Tailscale route table corrected** against live `tailscale serve status`: `/agent-estate` target is bare `http://127.0.0.1:8769` (not `.../8769/agent-estate-role-dashboard.html`); previously-missing tailnet-only `:9443` → `127.0.0.1:5173` documented. CT201 routes refreshed (8085 targets + `/steel` 5173 + `/steel-api` 3000).
- **Proxmox inventory rebuilt** from live `pct list` + `qm list`: added CT475 `agentzero` and VM110 `omarchy`; corrected 282 `ubuntu-server-xfce` to stopped; removed stale rows (310 monitoring, 500 litellm, 104 rclone, 106 glance, 101 debianvm). **VM403 `open-claw` and VM401 `hermes-old` are absent from `qm list`** — flagged unconfirmed rather than guessed.
- **Nanobot verified** on CT333: gateway `nanobot-gateway` (PID 2259048) listening on 8765/8766; NOT a systemd `nanobot.service`/`gateway.service` (both probes returned inactive — wrong unit names). Port 8900 no longer listening — flagged as unverified.
- **Docker services page corrected:** the old "Docker runtime not available" note was a CT460 socket-permission artifact (`/var/run/docker.sock` is `root:docker srw-rw----`, `hermes` user not in group `docker`, no passwordless sudo in cron). Documented the three real Docker hosts (CT200 ~50 containers, CT333 9, CT460 via sudo).
- **Flagged for follow-up, root cause not investigated:** `mycompose-db-1` on CT200 is in `Restarting (1)` — crash-looping.
- Mattermost CT255 confirmed stopped, matching its Tailscale peer offline 21d.
- Left as stale-without-fix: 62 pages still older than 21 days (concepts, decisions, comparisons, most runbooks) — these were not probed against a live source this pass, so they were reported, not rewritten.
+106
View File
@@ -0,0 +1,106 @@
# Wiki freshness audit — 2026-10-11
Threshold: >21 days since `updated`
## STALE (62)
- [ 81d] comparisons/camoufox-vs-chrome.md (updated 2026-07-22)
- [ 81d] comparisons/hermes-vs-nanobot.md (updated 2026-07-22)
- [ 81d] concepts/agent-memory.md (updated 2026-07-22)
- [ 81d] concepts/model-routing.md (updated 2026-07-22)
- [ 81d] concepts/persistent-shells.md (updated 2026-07-22)
- [ 81d] concepts/tool-calling.md (updated 2026-07-22)
- [ 81d] decisions/index.md (updated 2026-07-22)
- [ 81d] decisions/keep-openrouter-as-fallback.md (updated 2026-07-22)
- [ 81d] decisions/persistent-shell-enabled.md (updated 2026-07-22)
- [ 81d] experiments/index.md (updated 2026-07-22)
- [ 81d] infrastructure/backups.md (updated 2026-07-22)
- [ 81d] infrastructure/credentials-map.md (updated 2026-07-22)
- [ 81d] infrastructure/docker-services.md (updated 2026-07-22)
- [ 81d] infrastructure/domains-and-tunnels.md (updated 2026-07-22)
- [ 81d] infrastructure/network-map.md (updated 2026-07-22)
- [ 81d] queries/retained-research-answers.md (updated 2026-07-22)
- [ 81d] runbooks/backup-wiki.md (updated 2026-07-22)
- [ 81d] runbooks/diagnose-docker.md (updated 2026-07-22)
- [ 81d] runbooks/recover-docker-service.md (updated 2026-07-22)
- [ 81d] runbooks/restart-browser.md (updated 2026-07-22)
- [ 81d] runbooks/restart-hermes.md (updated 2026-07-22)
- [ 81d] runbooks/restore-openrouter.md (updated 2026-07-22)
- [ 81d] runbooks/test-openrouter.md (updated 2026-07-22)
- [ 81d] runbooks/troubleshoot-tool-loop.md (updated 2026-07-22)
- [ 81d] systems/cloudflare-tunnel.md (updated 2026-07-22)
- [ 81d] systems/headless-server.md (updated 2026-07-22)
- [ 81d] systems/home-assistant.md (updated 2026-07-22)
- [ 81d] systems/openrouter.md (updated 2026-07-22)
- [ 81d] systems/terminal-backend.md (updated 2026-07-22)
- [ 78d] runbooks/gateway-resource-resilience.md (updated 2026-07-25)
- [ 78d] systems/tool-search.md (updated 2026-07-25)
- [ 77d] systems/homepage-dashboard.md (updated 2026-07-26)
- [ 77d] systems/obsidian.md (updated 2026-07-26)
- [ 75d] runbooks/job-radar.md (updated 2026-07-28)
- [ 75d] runbooks/multiplexer-setup.md (updated 2026-07-28)
- [ 74d] systems/opencode-go.md (updated 2026-07-29)
- [ 73d] systems/hermes-ui.md (updated 2026-07-30)
- [ 71d] incidents/2026-08-01-mnemosyne-capture-failure.md (updated 2026-08-01)
- [ 70d] comparisons/steel-vs-camoufox.md (updated 2026-08-02)
- [ 70d] concepts/browser-automation.md (updated 2026-08-02)
- [ 70d] decisions/rollback-steel-to-camofox.md (updated 2026-08-02)
- [ 70d] reports/uber-eats/index.md (updated 2026-08-02)
- [ 70d] systems/browser-backend.md (updated 2026-08-02)
- [ 65d] systems/nanobot.md (updated 2026-08-07)
- [ 60d] comparisons/memory-backends.md (updated 2026-08-12)
- [ 60d] comparisons/model-routers.md (updated 2026-08-12)
- [ 59d] infrastructure/proxmox.md (updated 2026-08-13)
- [ 58d] systems/guanaco.md (updated 2026-08-14)
- [ 58d] systems/omniroute.md (updated 2026-08-14)
- [ 57d] agents/index.md (updated 2026-08-15)
- [ 57d] agents/open-claw.md (updated 2026-08-15)
- [ 57d] agents/qwenpaw.md (updated 2026-08-15)
- [ 57d] agents/zeroclaw.md (updated 2026-08-15)
- [ 57d] incidents/2026-08-15-skills-index-drift.md (updated 2026-08-15)
- [ 56d] CONTRIBUTING.md (updated 2026-08-16)
- [ 55d] agents/nanobot.md (updated 2026-08-17)
- [ 55d] systems/nextcloud.md (updated 2026-08-17)
- [ 54d] incidents/2026-08-18-mnemosyne-capture-pipeline.md (updated 2026-08-18)
- [ 53d] systems/ignis.md (updated 2026-08-19)
- [ 44d] reports/communities-wa-social-media-audit-2026.md (updated 2026-08-28)
- [ 43d] incidents/2026-08-29-gmail-read-state.md (updated 2026-08-29)
- [ 37d] incidents/2026-09-02-sftpgo-outage.md (updated 2026-09-04)
## FRESH (24)
- [ 7d] agents/hermes-chaos.md
- [ 7d] agents/hermes-production.md
- [ 0d] current-state.md
- [ 8d] incidents/2026-08-26-work-outlook-basic-auth-disabled.md
- [ 0d] incidents/2026-09-28-hermes-managed-runtime-deps.md
- [ 5d] incidents/2026-10-01-cliproxy-connection-auth-expired.md
- [ 7d] incidents/2026-10-03-ct450-hermes-update.md
- [ 7d] incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md
- [ 5d] incidents/2026-10-05-dead-model-pin-repin.md
- [ 2d] incidents/2026-10-09-cron-provider-timeout-storm.md
- [ 2d] incidents/2026-10-09-memory-recall-relevance.md
- [ 7d] infrastructure/hosts.md
- [ 1d] infrastructure/tailscale.md
- [ 7d] runbooks/provider-health.md
- [ 7d] runbooks/update-hermes-safely.md
- [ 9d] systems/current-profile.md
- [ 0d] systems/hermes-agent.md
- [ 7d] systems/memory-backend.md
- [ 0d] systems/messaging-integrations.md
- [ 0d] systems/mnemosyne.md
- [ 10d] systems/model-providers.md
- [ 0d] systems/paseo.md
- [ 0d] systems/scheduled-tasks.md
- [ 0d] systems/skills-index.md
## NO updated DATE (7)
- SCHEMA.md
- experiments/active/example.md
- incidents/2026-07-29-mnemosyne-relay.md
- incidents/2026-08-04-omniroute-free-deepseek-silent-stops.md
- index.md
- log.md
- reports/uber-eats/2026-07.md
TOTAL: 62 stale, 24 fresh, 7 no-date of 93 pages
+15 -3
View File
@@ -3,8 +3,8 @@ title: Ignis
type: system
status: active
created: 2026-08-19
updated: 2026-08-19
verified_on: 2026-08-19
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [ignis, obsidian, browser-native, wiki, docker]
sources: [live-probe, docker-inspect]
@@ -18,10 +18,12 @@ Browser-native Obsidian interface for the agent estate wiki. Provides full Obsid
## Details
- **Image:** `ignis-subpath:0.8.9` (custom build with subpath support)
- **Container:** `ignis` on CT460 (Hermes LXC)
- **Port:** `8080` (bound to `0.0.0.0`)
- **Port:** `8080` (bound to `0.0.0.0`) — confirmed `HTTP 200` on `http://127.0.0.1:8080/` 2026-10-11
- **Tailscale Serve:** `https://hermes.kangaroo-eel.ts.net/ignis` → `http://127.0.0.1:8080`
- **Privileged:** No (subpath build handles glibc threading without it)
- **Restart:** `unless-stopped`
- **Server process:** `node /app/apps/ignis-server/server/index.js` running as user `hermes` (PID observed 875) under `containerd-shim-runc-v2`, container `453483b8d99c…`
- **Docker socket:** `/var/run/docker.sock` is `root:docker` `srw-rw----` — the `hermes` user is not in group `docker`, so `docker ps`/`docker inspect` on CT460 require `sudo` (no passwordless sudo in this cron context). Verify Ignis via the HTTP endpoint and process list instead.
## Volume Mounts
| Host Path | Container Path | Purpose |
@@ -38,6 +40,16 @@ Browser-native Obsidian interface for the agent estate wiki. Provides full Obsid
## Docker Compose
Located at `/home/hermes/docker/ignis/docker-compose.yml` on CT460.
## ⚠ Second Ignis instance on CT200 (found 2026-10-11)
There is a **separate** `ignis-ignis-1` container running on **CT200** (the Docker host), not CT460:
- Image `ignis-server:latest`, state running
- Host port **8092** → container 8080
- Vault mount `/opt/ignis/vaults` → `/vaults` (different vault path from CT460's `/home/hermes/wiki/agent-estate-wiki`)
- Data mount `/opt/ignis/data` → `/app/data`
- Obsidian version 1.12.7, Node 22.23.1
**Do not conflate the two.** The canonical wiki-serving Ignis for the agent estate wiki is the CT460 instance exposed at `/ignis`. The CT200 instance is a different deployment on a different vault. Confirm which instance a user means before troubleshooting.
## Access
- **Tailscale (primary):** `https://hermes.kangaroo-eel.ts.net/ignis`
- **Local (on CT460):** `http://localhost:8080`
+22 -6
View File
@@ -3,8 +3,8 @@ title: OmniRoute
type: system
status: active
created: 2026-07-22
updated: 2026-08-14
verified_on: 2026-08-14
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [routing, provider, model, inference]
sources: [raw/configs/hermes-config-sanitized.txt]
@@ -16,16 +16,32 @@ sources: [raw/configs/hermes-config-sanitized.txt]
Secondary provider/router for Hermes auxiliary roles and some agent routing.
## Current Configuration
- **Hermes provider name:** `custom:omni`
- **Base URL:** `http://omni:20128/v1`
- **Host reachability:** `omni.kangaroo-eel.ts.net` resolves to `100.93.204.4`; CT601 running, direct HTTP reachable
- **Runs on:** Proxmox CT601 (`omni`) — migrated from CT600 (`omniroute`, now stopped) on 2026-08-13
- **Hermes provider name:** `custom:omni` (maps to provider `omniroute:20128`)
- **Base URL:** `http://omniroute:20128/v1`
- **Host reachability:** bare name `omniroute` resolves to `100.88.81.19`; direct HTTP on `:20128` answers `401` (auth-required, service live)
- **Runs on:** Proxmox CT600 (`omniroute`, Tailscale `100.88.81.19`, onboot=1) — verified running 2026-10-11
- **DB path:** `/opt/omniroute/storage.sqlite` (was `/var/lib/omniroute/storage.sqlite`)
- **Backup:** pre-migration config saved at `~/.hermes/config.yaml.bak.omniroute-removal`
- **Miss:** Hermes default provider path does not use OmniRoute for main chat; used for auxiliary roles instead
- **Auxiliary roles in Hermes config:** `skills_hub`, `approval`, `mcp`
- **Key prefix visible in config:** `sk-5c67fbdc0...` — full value not stored in wiki
## CT600 / CT601 / omniroute-1 — actual state (2026-10-11)
The wiki previously recorded CT600 as retired and CT601 as the live successor. **That is no longer true.** Live PVE and Tailscale checks return:
| Node | Status (live 2026-10-11) | Tailscale IP | Notes |
|---|---|---|---|
| CT600 `omniroute` | **running**, onboot=1 | `100.88.81.19` | OmniRoute listening on `0.0.0.0:20128`; `/v1/models` returns `401` (alive, auth required) |
| CT601 `omni` | **stopped**, onboot=0 | `100.93.204.4` | Tailscale peer last seen 54d ago |
| `omniroute-1` | **active** (not in PVE `pct list`) | `100.78.63.20` | Third OmniRoute node; `/v1/models` returns `401`. Registered in Hermes config as custom provider `omni-101`, which carries `skills_hub` (model `free-stack`, 5384 models, default `smart-route`) |
- Hermes config resolving to the omniroute hostname (`custom:omni` → `omniroute:20128`) therefore points at **CT600**, not CT601.
- CT600's combo API is reachable with its local key and returns combos (e.g. `go-pro`), confirming it is the serving instance.
- **Do not treat CT601 as the production OmniRoute node.** It is stopped with `onboot=0`.
- Treat `omniroute-1` (`100.78.63.20`) as a second live instance used by the `omni-101` provider. It is not visible in `pct list`, so it is likely a VM or off-PVE host; its identity was not confirmed.
## Catalog and Routing Facts
- Catalog reachable: yes, 47 combos returned at `/api/combos`
- Combo schema observed: `name`, `strategy`, `models`, `id`; some combos have `config`, `isHidden`, `uuid`
+5 -5
View File
@@ -3,8 +3,8 @@ title: Installed Skills
type: system
status: active
created: 2026-07-22
updated: 2026-10-10
verified_on: 2026-10-10
updated: 2026-10-11
verified_on: 2026-10-11
confidence: high
tags: [skills, hermes]
sources: []
@@ -20,7 +20,7 @@ This page is auto-regenerated from the live skill inventory. Do not hand-edit.
- Category = directory structure under `~/.hermes/skills/` (`category/skill/SKILL.md`).
- Skills directly under `~/.hermes/skills/` group under `uncategorized`.
## .Archive (39)
## .Archive (40)
- `agentmail-webhook-tailscale` — # AgentMail Real-Time Webhook via Tailscale Funnel
- `anthropic-spend-forensics` — # Anthropic Spend Forensics
@@ -40,6 +40,7 @@ This page is auto-regenerated from the live skill inventory. Do not hand-edit.
- `hermes-multiplex-telegram-ops` — # Hermes Multiplex Telegram Ops
- `hermes-profile-scoped-platform-troubleshooting` — # Hermes Profile-Scoped Platform Troubleshooting
- `hermes-session-context-ops` — # Hermes Session-Context Operations
- `hermes-ui-ops` — # Hermes UI Operations
- `hermes-web-backend-config` — # Hermes Web Backend Config
- `homelab-migration-planning` — # Test
- `in-repo-documentation` — # In-Repository Documentation
@@ -326,12 +327,11 @@ This page is auto-regenerated from the live skill inventory. Do not hand-edit.
- `groktocrawl-deployment` — # GroktoCrawl Self-Hosted Deployment
## Hermes (4)
## Hermes (3)
- `hermes-personality-soulmd` — # Hermes Personality & SOUL.md
- `hermes-telegram-miniapp` — # Hermes Telegram Mini App — Deploy & Operate
- `hermes-tts-configuration` — # Hermes TTS & Voice Configuration
- `hermes-ui-ops` — # Hermes UI Operations
## Hermes Api Vault (1)