Files
agent-estate-wiki/infrastructure/docker-services.md
T
Tony0410 4a6e86948d wiki: 2026-11-14 nightly audit + corrections — Phonon TTS on CT200, Mnemosyne capture stall open
- new: systems/phonon-tts.md, incidents/2026-11-14-mnemosyne-capture-stall.md
- CT200 inventory re-verified: 64 non-seeder running / 72 total (delta = the TTS trio, no removals)
- fleet ledger: 11-14 = 167 rows / 166 completed / 0 failed; first red last_status since 11-13
  (Mnemosyne Health Monitor, error, streak 1); cron_incidents 0 -> 1 unclosed (138 rows)
- corrections pass: withdrew the 'unidentified extra container' claim; closed the alert-delivery
  question (delivered 00:03:39, for_failure=1)
- updated: index.md, current-state.md, log.md, systems/scheduled-tasks.md, systems/mnemosyne.md,
  infrastructure/docker-services.md
2026-11-15 00:42:01 +08:00

14 KiB
Raw Blame History

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Docker Services infrastructure mixed 2026-07-22 2026-11-07 2026-11-07 high
docker
homelab
infrastructure
raw/configs/hermes-config-sanitized.txt
live-probe

Docker Services

Purpose

Containerized services supporting Hermes and the homelab.

Status

Docker runs on four hosts, and CT460's socket is not readable by the agent user. This page previously claimed "Docker runtime not available" — that is only true for CT460's socket from the hermes user. Real state, re-verified 2026-11-01:

Host Access Containers running
CT200 docker (100.96.150.69) pct exec 200 -- docker ps via PVE 65 running / 72 total — main Docker host (re-verified 2026-11-15), plus 0–2 transient worldmonitor-seeder-* containers driven by systemd timers (see the note below)
CT201 runtipi (100.96.244.39) pct exec 201 -- docker ps via PVE 20 running / 26 total — Runtipi stack: audiobookshelf, calibre-web, code-server, cyberchef, duplicati, filebrowser, flaresolverr, gotify, homarr, homebox, homepage-manager, it-tools, linkwarden, linkwarden-db, mealie-1, ntfy, runtipi, runtipi-db, syncthing, transmission (re-verified 2026-11-07 — set-wise diff against live docker ps: no additions, no removals)
CT333 clawtest (100.122.189.81) pct exec 333 -- docker ps via PVE 8 running / 9 total (re-verified 2026-11-07) — nostalgic_keldysh, community-spark, readlater-extraction-worker, readlater, kokoro-tts, edge-tts, gotify, onecli-postgres-1. The steel-browser container and its image were removed 2026-10-29; thirsty_payne → nostalgic_leavitt (2026-11-02) → flamboyant_mahavira (2026-11-04) → nostalgic_keldysh (2026-11-06) — all ghcr.io/github/github-mcp-server, the live one created 2026-11-05T22:15:57Z. ⚠ Corrected 2026-11-07: the enumerated list still opened with flamboyant_mahavira, which is not running — live docker ps on CT333 returns nostalgic_keldysh (created 2026-11-05 22:15:57, Up 29 hours), community-spark, readlater-extraction-worker, readlater, kokoro-tts, edge-tts, gotify, onecli-postgres-1 (8 running), plus qdrant exited (9 total)
CT1001 docker-01 (100.121.12.31, on m93p) pct exec 1001 -- docker ps via m93p 1 running / 9 total — sshwifty. Added 2026-10-31 — a fourth Docker host, previously undocumented
CT460 hermes pct exec 460 -- docker ps as root from the PVE host — socket is root:docker srw-rw----, the hermes user is not in group docker (its own docker ps → permission denied while trying to connect to the docker API) ignis — running (Up 5 days, ignis-subpath:0.8.9, 0.0.0.0:8080->8080/tcp), re-verified 2026-11-05

CT200 live inventory (re-verified 2026-11-15, 65 running / 72 total, excluding 1 transient worldmonitor-seeder-* container)

8mblocal, apprise-api, audiobookshelf, chromadb, edge-tts, emulatorjs, excalidraw-backend, excalidraw-frontend, filebrowser, filebrowser1, flaresolverr, guacamole-guacamole-1, guacamole-guacd-1, guacamole-postgres-1, homarr, ignis-ignis-1, jellyfin, khoj-database-1, khoj-sandbox-1, khoj-search-1, khoj-server-1, kokoro-tts, libre-closet, lobe-chat, mycompose-db-1, mycompose-saas-1, nc_app_context_agent, nc_app_talk_bot_ai, nextstep-app, nextstep-db, nextstep-scheduler-1, onlyoffice-docserver, open-webui, overtchat-app, overtchat-redis, overtchat-searxng, phonon-demo, ptts-openai-server, ptts-ws-server, portainer, portkey-gateway-web-1, postgres-newsku, prowlarr, qbittorrent, quickdrop, react-resume-client-1, readlater, readlater-extraction-worker, redis-cache, save-restricted-bot, searxng, secret-santa-app, sshwifty, telegram-bot-api, tgvideo-bot, tt-rss-app-1, tt-rss-db-1, tt-rss-updater-1, tt-rss-web-nginx-1, watchtower, worldmonitor, worldmonitor-redis, worldmonitor-redis-rest, ytdlp-web-player.

Re-verified 2026-11-15 00:4x AWST — 61 → 64 running (excluding the seeder), 68 → 72 total, and the whole delta is the TTS stack. Live docker ps on CT200 returns 65 running / 72 total, which is 64 non-seeder running + 1 transient worldmonitor-seeder-scheduled. A set-wise diff of the live names against this page's 2026-11-07 list found three additions and no removals, and the arithmetic closes exactly: 61 non-seeder → 64 non-seeder is the phonon-demo, ptts-openai-server and ptts-ws-server trio — all created 2026-11-14, documented on systems/phonon-tts. Earlier in this pass a generic total − prior total subtraction suggested a fourth, unnamed addition; that was a comparison artefact (the 11-07 row's 61 was an explicit non-seeder count, today's 65 was a raw running count including the seeder) and is withdrawn — the name-level diff is exact and accounts for every added container. This pass verified counts, the TTS trio, and the absence of removals; it is not a full CT200 audit.

Re-verified 2026-11-06 — the 2026-11-05 "the seeder does not exist" finding was WRONG, and the cause is a sampling artefact, not a removal. A set-wise diff against live docker ps --format '{{.Names}}' on CT200 shows no container live but absent from the list above, and no listed container has disappeared — the 61-container running set above still matches live exactly, set-wise. The worldmonitor-seeder-* containers are transient and systemd-timer driven, so they are only visible while a seed run is in flight: at the 2026-11-06 03:1x probe both were present (worldmonitor-seeder-scheduled, created 02:45:14, Up 32 minutes (unhealthy), and worldmonitor-seeder-fast, created 03:17:15, Up 22 seconds (health: starting)); at the 2026-11-05 05:1x probe neither was, because no run was in flight — hence the zero seeder matches. The drivers are systemd timers on CT200, not cron: worldmonitor-seed.timer (OnUnitInactiveSec=4h, RandomizedDelaySec=10min → worldmonitor-seed.service → docker run --rm --name worldmonitor-seeder-scheduled … scripts/run-seeders.sh) and worldmonitor-seed-fast.timer (OnUnitInactiveSec=15min, RandomizedDelaySec=1min → worldmonitor-seed-fast.service → --name worldmonitor-seeder-fast). Both services were ActiveState=activating at the probe. The corrected claim is therefore: the seeders exist and run on a 4h + 15min schedule; a point-in-time docker ps can miss them entirely, so "absent" must never be read as "removed". The corrected on record note: the earlier worldmonitor-seeder-fast → worldmonitor-seeder-scheduled rename narrative was also wrong — there is one script with two timer-driven invocations under two container names, not a rename. Live CT200 also shows worldmonitor (Up 6 days, healthy), worldmonitor-redis (Up 6 days) and worldmonitor-redis-rest (Up 6 days).

Re-verified 2026-11-05 — 61 → 61 running, seeder entry removed. A set-wise diff against live docker ps --format '{{.Names}}' on CT200 shows no container live but absent from the list above, and no listed container has disappeared. The worldmonitor-seeder-scheduled entry recorded on 2026-11-04 as "unhealthy" does not exist on any host — verified with docker ps -a --filter name=seeder across CT200, CT201, CT333 and CT460, all of which return zero matches. ⚠ SUPERSEDED 2026-11-06 — see the correction above: this was a sampling artefact. The seeders are transient systemd-timer-driven containers, not absent. There is no seeder container and no worldmonitor-seeder cron job anywhere in the estate — the 2026-11-04 "watch item" was a transient container that has since been removed. ⚠ SUPERSEDED 2026-11-06: the seeders DO exist; the driver is a systemd timer pair, not a cron job. Correction on record: the earlier worldmonitor-seeder-fast → worldmonitor-seeder-scheduled rename narrative did not reflect a durable redeploy; the seeder is simply absent now. ⚠ SUPERSEDED 2026-11-06: one script, two timer-driven container names — not a rename.

CT333 container set — nostalgic_leavitt → flamboyant_mahavira (2026-11-05) → nostalgic_keldysh (2026-11-06). Live CT333 docker ps returns 8 running containers, and the name that had replaced thirsty_payne on 2026-11-04 and then flamboyant_mahavira on 2026-11-05 has itself been replaced again: nostalgic_keldysh (image ghcr.io/github/github-mcp-server, created 2026-11-05T22:15:57Z, Up 5 hours at the 2026-11-06 probe). The count is unchanged at 8 and the other seven names are identical (community-spark, readlater-extraction-worker, readlater, kokoro-tts, edge-tts, gotify, onecli-postgres-1). A ninth container, qdrant (qdrant/qdrant), exists but is stopped (Exited (143) 4 months ago) — it is not in the running count and is not listed above. Docker generates these random names for compose-less containers, so repeated renames are normal — the reliable check is always the live set, never the recorded name.

CT460 ignis access path corrected (2026-11-04). The old row said "no passwordless sudo in cron", which implied the container list could not be read at all. It can: pct exec 460 -- docker ps from the PVE host runs as root and works (ignis|ignis-subpath:0.8.9|Up 4 days|0.0.0.0:8080->8080/tcp). What does not work is the hermes user's own docker ps inside CT460 — permission denied while trying to connect to the docker API at unix:///var/run/docker.sock — because the socket is root:docker srw-rw---- and hermes is not in group docker. Use the root-via-PVE path.

Re-verified 2026-11-01: the 2026-10-11 inventory listed 49 containers; the live count is now 61. Corrected 2026-11-01: the inventory list had three misspelled overtchat-* entries (written overchat-*) and one stale entry, worldmonitor-seeder-fast, which is not running — live docker ps -a --filter name=worldmonitor shows only worldmonitor, worldmonitor-redis and worldmonitor-redis-rest (all Up 35 hours). The list above is regenerated verbatim from live docker ps --format '{{.Names}}'. CT201's Tailscale IP was also wrong — it read 100.116.116.116; live pct exec 201 -- tailscale ip -4 returns 100.96.244.39 (which is also what [[infrastructure/hosts]] already recorded). CT333's first container was renamed — adoring_jemison → thirsty_payne. 12 containers were added since: emulatorjs, excalidraw-backend, khoj-database-1, khoj-sandbox-1, khoj-server-1, mycompose-db-1, onlyoffice-docserver, quickdrop, react-resume-client-1, secret-santa-app, tgvideo-bot, watchtower. No container listed on the 2026-10-11 page has disappeared — the inventory is a strict superset.

✅ mycompose-db-1 crash-loop is FIXED. The 2026-10-11 page flagged it as Restarting (1) — a crash-looping database. Live docker ps -a --filter name=mycompose now shows both mycompose-db-1 and mycompose-saas-1 as Up 35 hours (checked 2026-11-01). No longer a watch item.

Note the duplicate Ignis: ignis-ignis-1 on CT200 is a separate deployment from CT460's ignis. See systems/ignis.

CT201 runtipi — Tailscale routes (verified 2026-10-11; Steel routes removed 2026-10-29)

Host Path Target
runtipi.kangaroo-eel.ts.net:8443 (tailnet) / http://127.0.0.1:8085
/mgr http://127.0.0.1:8084
runtipi.kangaroo-eel.ts.net (tailnet) / http://127.0.0.1:8085
/mgr http://127.0.0.1:8084

Correction: this page previously said "Port 443 occupied by Traefik on CT201 — Tailscale uses port 8443 there" and listed CT201's :8443 → localhost:8082 for Homepage. Live tailscale serve status on CT201 shows the target is 127.0.0.1:8085 on both :8443 and the default host. That table also carried a /steel (5173) and /steel-api (3000) pair — both routes were deleted on 2026-10-29 with tailscale serve --set-path <path> off when Steel was removed, so CT201's remaining routes are only / and /mgr on both :443 and :8443. (The /steel-api route never stripped the path prefix — moot now that it is gone.)

Confirmed Compose Files

File Path Services
Hermes Agent /home/hermes/.hermes/hermes-agent/docker-compose.yml hermes, hermes-dashboard
Hermes WebUI /home/hermes/hermes-webui/docker-compose.yml webui
ReadLater /home/hermes/workspace/readlater/docker-compose.yml readlater, edge-tts, kokoro-tts
Ignis /home/hermes/docker/ignis/docker-compose.yml (CT460) ignis, ignis-data, ignis-obsidian-app

Steel Browser — removed 2026-10-29. The CT201 compose file and Dockerfile are no longer live: the project dir /root/steel-browser was deleted along with its containers, images and network. Both files are archived at /root/_archived-steel/ on CT201 (with /root/_archived-steel/steel-cache-archive-20261029.tgz), and the images remain re-pullable from ghcr.io. → comparisons/steel-vs-camoufox

Service Definitions (from compose files)

hermes-agent

  • Images: hermes-agent
  • Containers: hermes, hermes-dashboard
  • Volumes: present (paths unverified)
  • Environment: present (values unverified — secrets redacted)

hermes-webui

  • Ports: present (specific ports unverified)
  • Volumes: present

readlater

  • Container: readlater
  • Ports: present
  • Volumes: present
  • Dependencies: edge-tts (travisvn/openai-edge-tts), kokoro-tts (ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.1)

Known Issues

  • Docker runtime unavailable on inspection host — service state unknown
  • Specific ports, volumes, and env vars not extracted from compose files during this pass

Additional Services (CT201 runtipi)

  • Homepage — Dashboard at port 8082 (Tailscale: https://runtipi.kangaroo-eel.ts.net:8443/)
  • Manager — Web UI for Homepage at port 8084
  • See systems/homepage-dashboard.md for full docs