Files
agent-estate-wiki/infrastructure/docker-services.md
T
Tony0410 0b4ff25fd2 resolve: Gitea config backup push dead since 2026-08-08 — 92-commit backlog drained, script hardened to fail loudly
- Wire host-scoped Gitea credential (git credential approve); origin was HTTPS with no userinfo
- Strip 1.76GB checkpoints/ from the unpushed range via git filter-repo --partial (fast-forward preserved)
- Untrack checkpoints/ at the tip; push set ~2GB -> 264.8MB
- Rewrite backup.sh: exit non-zero on failure, stamp .last_backup_ok only after remote verification,
  decide push by remote-vs-local, guard ls-remote with || true, GitHub orphan snapshot via commit-tree,
  honest AgentMail/Telegram delivery reporting
- Verified live: exit 0, origin/main == local HEAD, 0 unpushed, Gitea + GitHub APIs confirmed
- Incident page -> status: resolved; backups.md -> RESOLVED; scheduled-tasks job row updated
- Also commits the uncommitted stocktake edits (the 03:00 stocktake cron failed with
  'lost its durable fire claim ownership' and left the tree dirty)
2026-11-07 05:28:39 +08:00

157 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Docker Services
type: infrastructure
status: mixed
created: 2026-07-22
updated: 2026-11-07
verified_on: 2026-11-07
confidence: high
tags: [docker, homelab, infrastructure]
sources: [raw/configs/hermes-config-sanitized.txt, live-probe]
---
# Docker Services
## Purpose
Containerized services supporting Hermes and the homelab.
## Status
Docker runs on **four hosts**, and CT460's socket is not readable by the agent user. This page previously claimed "Docker runtime not available" — that is only true for CT460's socket from the `hermes` user. Real state, **re-verified 2026-11-01**:
| Host | Access | Containers running |
|---|---|---|
| CT200 `docker` (`100.96.150.69`) | `pct exec 200 -- docker ps` via PVE | **61** running / 68 total — main Docker host (**re-verified 2026-11-07**), **plus 0–2 transient `worldmonitor-seeder-*` containers** driven by systemd timers (see the note below) |
| CT201 `runtipi` (`100.96.244.39`) | `pct exec 201 -- docker ps` via PVE | **20** running / 26 total — Runtipi stack: `audiobookshelf`, `calibre-web`, `code-server`, `cyberchef`, `duplicati`, `filebrowser`, `flaresolverr`, `gotify`, `homarr`, `homebox`, `homepage-manager`, `it-tools`, `linkwarden`, `linkwarden-db`, `mealie-1`, `ntfy`, `runtipi`, `runtipi-db`, `syncthing`, `transmission` (**re-verified 2026-11-07** — set-wise diff against live `docker ps`: no additions, no removals) |
| CT333 `clawtest` (`100.122.189.81`) | `pct exec 333 -- docker ps` via PVE | **8** running / 9 total (**re-verified 2026-11-07**) — `nostalgic_keldysh`, `community-spark`, `readlater-extraction-worker`, `readlater`, `kokoro-tts`, `edge-tts`, `gotify`, `onecli-postgres-1`. The `steel-browser` container and its image were **removed 2026-10-29**; `thirsty_payne` → `nostalgic_leavitt` (2026-11-02) → `flamboyant_mahavira` (2026-11-04) → **`nostalgic_keldysh`** (2026-11-06) — all `ghcr.io/github/github-mcp-server`, the live one created 2026-11-05T22:15:57Z. ⚠ **Corrected 2026-11-07:** the enumerated list still opened with `flamboyant_mahavira`, which is **not running** — live `docker ps` on CT333 returns **`nostalgic_keldysh`** (created 2026-11-05 22:15:57, `Up 29 hours`), `community-spark`, `readlater-extraction-worker`, `readlater`, `kokoro-tts`, `edge-tts`, `gotify`, `onecli-postgres-1` (8 running), plus `qdrant` **exited** (9 total) |
| CT1001 `docker-01` (`100.121.12.31`, **on `m93p`**) | `pct exec 1001 -- docker ps` via `m93p` | **1** running / 9 total — `sshwifty`. **Added 2026-10-31** — a fourth Docker host, previously undocumented |
| CT460 `hermes` | `pct exec 460 -- docker ps` **as root** from the PVE host — socket is `root:docker srw-rw----`, the `hermes` user is not in group `docker` (its own `docker ps` → `permission denied while trying to connect to the docker API`) | `ignis` — **running** (`Up 5 days`, `ignis-subpath:0.8.9`, `0.0.0.0:8080->8080/tcp`), re-verified 2026-11-05 |
## CT200 live inventory (re-verified 2026-11-07, 61 running / 68 total, excluding 0–2 transient `worldmonitor-seeder-*` containers)
`8mblocal`, `apprise-api`, `audiobookshelf`, `chromadb`, `edge-tts`, `emulatorjs`, `excalidraw-backend`, `excalidraw-frontend`, `filebrowser`, `filebrowser1`, `flaresolverr`, `guacamole-guacamole-1`, `guacamole-guacd-1`, `guacamole-postgres-1`, `homarr`, **`ignis-ignis-1`**, `jellyfin`, `khoj-database-1`, `khoj-sandbox-1`, `khoj-search-1`, `khoj-server-1`, `kokoro-tts`, `libre-closet`, `lobe-chat`, `mycompose-db-1`, `mycompose-saas-1`, `nc_app_context_agent`, `nc_app_talk_bot_ai`, `nextstep-app`, `nextstep-db`, `nextstep-scheduler-1`, `onlyoffice-docserver`, `open-webui`, `overtchat-app`, `overtchat-redis`, `overtchat-searxng`, `portainer`, `portkey-gateway-web-1`, `postgres-newsku`, `prowlarr`, `qbittorrent`, `quickdrop`, `react-resume-client-1`, `readlater`, `readlater-extraction-worker`, `redis-cache`, `save-restricted-bot`, `searxng`, `secret-santa-app`, `sshwifty`, `telegram-bot-api`, `tgvideo-bot`, `tt-rss-app-1`, `tt-rss-db-1`, `tt-rss-updater-1`, `tt-rss-web-nginx-1`, `watchtower`, `worldmonitor`, `worldmonitor-redis`, `worldmonitor-redis-rest`, `ytdlp-web-player`.
**Re-verified 2026-11-06 — the 2026-11-05 "the seeder does not exist" finding was WRONG, and the cause is a
sampling artefact, not a removal.** A set-wise diff against live `docker ps --format '{{.Names}}'` on CT200
shows **no container live but absent from the list above**, and **no listed container has disappeared** — the
61-container running set above still matches live exactly, set-wise. The `worldmonitor-seeder-*` containers are
**transient and systemd-timer driven**, so they are only visible while a seed run is in flight: at the
2026-11-06 03:1x probe **both** were present (`worldmonitor-seeder-scheduled`, created 02:45:14, `Up 32 minutes
(unhealthy)`, and `worldmonitor-seeder-fast`, created 03:17:15, `Up 22 seconds (health: starting)`); at the
2026-11-05 05:1x probe **neither** was, because no run was in flight — hence the zero `seeder` matches. The
drivers are **systemd timers on CT200, not cron**: `worldmonitor-seed.timer` (`OnUnitInactiveSec=4h`,
`RandomizedDelaySec=10min` → `worldmonitor-seed.service` → `docker run --rm --name
worldmonitor-seeder-scheduled … scripts/run-seeders.sh`) and `worldmonitor-seed-fast.timer`
(`OnUnitInactiveSec=15min`, `RandomizedDelaySec=1min` → `worldmonitor-seed-fast.service` → `--name
worldmonitor-seeder-fast`). Both services were `ActiveState=activating` at the probe. The **corrected** claim
is therefore: **the seeders exist and run on a 4h + 15min schedule; a point-in-time `docker ps` can miss them
entirely, so "absent" must never be read as "removed".** The **corrected on record** note: the earlier
`worldmonitor-seeder-fast` → `worldmonitor-seeder-scheduled` rename narrative was also wrong — there is **one
script with two timer-driven invocations** under two container names, not a rename. Live CT200 also shows
`worldmonitor` (`Up 6 days`, **healthy**), `worldmonitor-redis` (`Up 6 days`) and `worldmonitor-redis-rest`
(`Up 6 days`).
**Re-verified 2026-11-05 — 61 → 61 running, seeder entry removed.** A set-wise diff against live
`docker ps --format '{{.Names}}'` on CT200 shows **no container live but absent from the list above**, and
**no listed container has disappeared**. ~~The `worldmonitor-seeder-scheduled` entry recorded on 2026-11-04 as
"unhealthy" **does not exist on any host** — verified with `docker ps -a --filter name=seeder` across CT200,
CT201, CT333 and CT460, all of which return **zero** matches.~~ **⚠ SUPERSEDED 2026-11-06 — see the
correction above: this was a sampling artefact. The seeders are transient systemd-timer-driven containers,
not absent.** ~~There is **no seeder container and no worldmonitor-seeder cron job** anywhere in the estate —
the 2026-11-04 "watch item" was a transient container that has since been removed.~~ **⚠ SUPERSEDED
2026-11-06: the seeders DO exist; the driver is a systemd timer pair, not a cron job.** ~~**Correction on
record:** the earlier `worldmonitor-seeder-fast` → `worldmonitor-seeder-scheduled` rename narrative did not
reflect a durable redeploy; the seeder is simply absent now.~~ **⚠ SUPERSEDED 2026-11-06: one script, two
timer-driven container names — not a rename.**
**CT333 container set — `nostalgic_leavitt` → `flamboyant_mahavira` (2026-11-05) → `nostalgic_keldysh` (2026-11-06).**
Live CT333 `docker ps` returns 8 running containers, and the name that had replaced `thirsty_payne` on 2026-11-04
and then `flamboyant_mahavira` on 2026-11-05 has **itself been replaced again**:
**`nostalgic_keldysh`** (image `ghcr.io/github/github-mcp-server`, **created 2026-11-05T22:15:57Z**, `Up 5 hours`
at the 2026-11-06 probe). The count is unchanged at 8 and the other seven names are identical
(`community-spark`, `readlater-extraction-worker`, `readlater`, `kokoro-tts`, `edge-tts`, `gotify`,
`onecli-postgres-1`). A ninth container, `qdrant`
(`qdrant/qdrant`), exists but is **stopped** (`Exited (143) 4 months ago`) — it is not in the running count and
is not listed above. **Docker generates these random names for compose-less containers, so repeated renames are
normal — the reliable check is always the live set, never the recorded name.**
**CT460 `ignis` access path corrected (2026-11-04).** The old row said "no passwordless sudo in cron", which
implied the container list could not be read at all. It can: **`pct exec 460 -- docker ps` from the PVE host
runs as root and works** (`ignis|ignis-subpath:0.8.9|Up 4 days|0.0.0.0:8080->8080/tcp`). What does *not* work is
the `hermes` user's own `docker ps` inside CT460 — `permission denied while trying to connect to the docker API
at unix:///var/run/docker.sock` — because the socket is `root:docker srw-rw----` and `hermes` is not in group
`docker`. Use the root-via-PVE path.
**Re-verified 2026-11-01:** the 2026-10-11 inventory listed **49** containers; the live count is now **61**.
**Corrected 2026-11-01:** the inventory list had three **misspelled** `overtchat-*` entries (written
`overchat-*`) and one **stale** entry, `worldmonitor-seeder-fast`, which is **not running** — live
`docker ps -a --filter name=worldmonitor` shows only `worldmonitor`, `worldmonitor-redis` and
`worldmonitor-redis-rest` (all `Up 35 hours`). The list above is regenerated verbatim from live
`docker ps --format '{{.Names}}'`. **CT201's Tailscale IP was also wrong** — it read
`100.116.116.116`; live `pct exec 201 -- tailscale ip -4` returns **`100.96.244.39`** (which is
also what `[[infrastructure/hosts]]` already recorded). **CT333's first container was renamed** —
`adoring_jemison` → **`thirsty_payne`**.
**12 containers were added** since: `emulatorjs`, `excalidraw-backend`, `khoj-database-1`, `khoj-sandbox-1`,
`khoj-server-1`, `mycompose-db-1`, `onlyoffice-docserver`, `quickdrop`, `react-resume-client-1`,
`secret-santa-app`, `tgvideo-bot`, `watchtower`. **No container listed on the 2026-10-11 page has
disappeared** — the inventory is a strict superset.
**✅ `mycompose-db-1` crash-loop is FIXED.** The 2026-10-11 page flagged it as `Restarting (1)` — a
crash-looping database. Live `docker ps -a --filter name=mycompose` now shows **both** `mycompose-db-1`
and `mycompose-saas-1` as **`Up 35 hours`** (checked 2026-11-01). No longer a watch item.
**Note the duplicate Ignis:** `ignis-ignis-1` on CT200 is a separate deployment from CT460's `ignis`. See [[systems/ignis]].
## CT201 runtipi — Tailscale routes (verified 2026-10-11; Steel routes removed 2026-10-29)
| Host | Path | Target |
|---|---|---|
| `runtipi.kangaroo-eel.ts.net:8443` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
| `runtipi.kangaroo-eel.ts.net` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
**Correction:** this page previously said "Port 443 occupied by Traefik on CT201 — Tailscale uses port 8443 there" and listed CT201's `:8443` → `localhost:8082` for Homepage. Live `tailscale serve status` on CT201 shows the target is `127.0.0.1:8085` on both `:8443` and the default host. That table also carried a `/steel` (5173) and `/steel-api` (3000) pair — **both routes were deleted on 2026-10-29** with `tailscale serve --set-path <path> off` when Steel was removed, so CT201's remaining routes are only `/` and `/mgr` on **both** `:443` and `:8443`. (The `/steel-api` route never stripped the path prefix — moot now that it is gone.)
## Confirmed Compose Files
| File | Path | Services |
|---|---|---|
| Hermes Agent | `/home/hermes/.hermes/hermes-agent/docker-compose.yml` | hermes, hermes-dashboard |
| Hermes WebUI | `/home/hermes/hermes-webui/docker-compose.yml` | webui |
| ReadLater | `/home/hermes/workspace/readlater/docker-compose.yml` | readlater, edge-tts, kokoro-tts |
| Ignis | `/home/hermes/docker/ignis/docker-compose.yml` (CT460) | ignis, ignis-data, ignis-obsidian-app |
**Steel Browser — removed 2026-10-29.** The CT201 compose file and Dockerfile are no longer live: the project
dir `/root/steel-browser` was deleted along with its containers, images and network. Both files are archived at
`/root/_archived-steel/` on CT201 (with `/root/_archived-steel/steel-cache-archive-20261029.tgz`), and the
images remain re-pullable from ghcr.io. → [[comparisons/steel-vs-camoufox]]
## Service Definitions (from compose files)
### hermes-agent
- **Images:** `hermes-agent`
- **Containers:** `hermes`, `hermes-dashboard`
- **Volumes:** present (paths unverified)
- **Environment:** present (values unverified — secrets redacted)
### hermes-webui
- **Ports:** present (specific ports unverified)
- **Volumes:** present
### readlater
- **Container:** `readlater`
- **Ports:** present
- **Volumes:** present
- **Dependencies:** `edge-tts` (travisvn/openai-edge-tts), `kokoro-tts` (ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.1)
## Known Issues
- Docker runtime unavailable on inspection host — service state unknown
- Specific ports, volumes, and env vars not extracted from compose files during this pass
## Additional Services (CT201 runtipi)
- **Homepage** — Dashboard at port 8082 (Tailscale: `https://runtipi.kangaroo-eel.ts.net:8443/`)
- **Manager** — Web UI for Homepage at port 8084
- See [systems/homepage-dashboard.md](../systems/homepage-dashboard.md) for full docs
## Related
- [[docker-services]]
- [[infrastructure/hosts]]