docs(wiki): correct Steel Browser references across 15 pages — Steel fully removed, Camofox is sole backend

Root cause of the long-standing cloud_provider: steel residue: the 2026-07-30
rollback disabled the browser-steel plugin and set browser.engine: camofox but
left browser.cloud_provider: steel in config.yaml. The split config was
inert-but-wrong, so it survived ~3 months of audits unnoticed.

Corrects pages that still described Steel as present, running or configured:
- systems/browser-backend.md (incl. gateway-restart env-var note)
- comparisons/steel-vs-camoufox.md, comparisons/camoufox-vs-chrome.md
- decisions/rollback-steel-to-camofox.md, decisions/index.md
- infrastructure/docker-services.md, infrastructure/tailscale.md, infrastructure/hosts.md
- systems/homepage-dashboard.md, systems/scheduled-tasks.md
- concepts/browser-automation.md, runbooks/recover-docker-service.md
- current-state.md, index.md, agents/nanobot.md
- log.md (dated entry; supersedes the 00:3x audit's 'endpoint dark' wording)

Also restores the Ignis compose row in infrastructure/docker-services.md, which
the 2026-10-11 revision dropped (verified: /home/hermes/docker/ignis/docker-compose.yml
exists and the ignis container is Up on CT460).

Co-Authored-By: Hermes <hermes@local>
This commit is contained in:
Tony0410
2026-10-29 02:02:49 +08:00
co-authored by Hermes
parent ec0c30fb30
commit 677bb0c959
18 changed files with 757 additions and 592 deletions
+7 -5
View File
@@ -3,8 +3,8 @@ title: Nanobot
type: agent
status: active
created: 2026-07-22
updated: 2026-10-11
verified_on: 2026-10-11
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [agent, nanobot, homelab]
sources: [live-probe, runtime-config]
@@ -35,10 +35,12 @@ Primary homelab AI agent running on `clawtest` (CT333). Handles agent orchestrat
| `agentmail-daemon.py` | nano | — | started Sep 23 |
| `readlater` Next dev server | nano | 3210 | started Sep 23 |
Other listeners on CT333 include 8787, 8880, 8899, 8999, 3000, 5050, 8080 and the Tailscale 443.
Other listeners on CT333 include 8787, 8880, 8899, 8999, 5050, 8080 and the Tailscale 443. *(Port 3000 was published by the `steel-browser` container, which was removed 2026-10-29.)*
## Docker on CT333 (verified 2026-10-11)
`steel-browser`, `gotify`, `readlater`, `readlater-extraction-worker`, `kokoro-tts`, `edge-tts`, `onecli-postgres-1`, `tender_brattain`, plus healthy `community-spark` and `quirky_darwin`.
## Docker on CT333 (verified 2026-10-11; Steel container removed 2026-10-29)
`gotify`, `readlater`, `readlater-extraction-worker`, `kokoro-tts`, `edge-tts`, `onecli-postgres-1`, `tender_brattain`, plus healthy `community-spark` and `quirky_darwin`.
**Steel is gone from CT333 too.** The `steel-browser` container (image `ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30, publishing port 3000) and its image were removed on **2026-10-29**. It was already orphaned — its last CDP navigation was 2026-10-20, it crashed on 2026-10-22, it received no requests after 2026-10-21, and it had no Tailscale Serve route, no entry in `~/.zeroclaw/config.toml` (which uses `backend = "agent_browser"` on native_webdriver at `127.0.0.1:9515`), and no reference from Nanobot's browser harness. → [[comparisons/steel-vs-camoufox]]
## Cron jobs
Unverified against live Nanobot scheduler on 2026-10-11. Previously recorded:
+6 -3
View File
@@ -3,8 +3,8 @@ title: Camoufox vs Chrome
type: comparison
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
updated: 2026-10-29
verified_on: 2026-10-29
confidence: medium
tags: [comparison, browser]
sources: [raw/configs/hermes-config-sanitized.txt]
@@ -35,7 +35,10 @@ Compare the two browser-automation options for Hermes's browser tool.
`browser.camofox:` → `adopt_existing_tab: true`, `managed_persistence: true`,
`rewrite_loopback_urls: true`, `user_id: anthony`.
**Note `inactivity_timeout` is 450s, not the 120s recorded here** — that value changed.
`cloud_provider` is `steel` with `engine: camofox`; the two keys describe different layers.
~~`cloud_provider` is `steel` with `engine: camofox`; the two keys describe different layers.~~
**Superseded 2026-10-29:** `cloud_provider` now reads **`camofox`** (with `camera_provider: camofox`,
`engine: auto`, `cdp_url: ""`), and Steel is **fully removed** from the estate — the `browser-steel` plugin
is deleted and there is no Steel fallback path. → [[systems/browser-backend]]
- **Live processes (2026-10-20):** `cua-driver serve` (PID 3312393), `camoufox-ws-server.mjs`,
and a live `camoufox-bin` (FirefoxDev build) tree under `/home/hermes/.cache/camoufox/`.
Camofox's REST port **9377 is not listening** on CT460 — the Camofox browser service runs on
+23 -14
View File
@@ -13,20 +13,20 @@ sources: [live-probe, session-history]
# Steel Browser vs Camoufox
## Purpose
Compare the two browser-automation backends available to Hermes, including the Jul 29–30 experiment and the decision to revert.
Compare the two browser-automation backends Hermes has used, including the Jul 29–30 experiment and the decision to revert. **Camofox is now the only backend available — Steel was fully removed on 2026-10-29.**
## At a glance
| | Steel Browser (CT201) | Camofox (systemd --user on CT460) |
|---|---|---|
| **In use** | Disabled since 2026-07-30 | **Yes** — primary backend |
| **In use** | **Removed 2026-10-29** (primary 2026-07-29 → 07-30) | **Yes** — the only backend |
| **Engine** | Chromium 140 (Playwright) | Camoufox (Firefox-based) |
| **Connection** | CDP (`ws://` via port 9223) | REST API + CDP WebSocket, both `localhost:9377` |
| **API** | REST at `:3000` — create/destroy/query sessions | Camoufox REST + CDP WebSocket |
| **Headless** | Native headless | Xvfb + VNC (headed, anti-fingerprint) |
| **Container** | Docker on CT201 | `systemd --user` unit on the Hermes host (CT460) |
| **Container** | Docker on CT201 — **removed 2026-10-29** | `systemd --user` unit on the Hermes host (CT460) |
| **Stealth** | Standard Chromium | Anti-detection fingerprinting |
| **Session viewer** | Built-in UI at `/steel` | VNC at `/vnc-camofox` |
| **Plugin** | `browser-steel` — **removed 2026-10-29** | Native Hermes `browser.cloud_provider: camofox` |
| **Session viewer** | Built-in UI at `/steel` — **route deleted** | VNC at `/vnc-camofox` |
| **Plugin** | `browser-steel` — **deleted 2026-10-29** | Native Hermes `browser.cloud_provider: camofox` |
## The Steel Experiment (Jul 29–30)
@@ -39,7 +39,7 @@ Compare the two browser-automation backends available to Hermes, including the J
1. **Jul 29:** Steel deployed on CT201 via Docker Compose, Chromium 140. Hermes configured with `browser.cloud_provider: steel`, `browser-steel` plugin enabled.
2. **Jul 29 (later):** CDP connectivity issues surfaced — Hermes was reaching `0.0.0.0` from the plugin's internal CDP URL. Required `_normalize_cdp_url()` patch in the Steel plugin to remap to `100.96.244.39`.
3. **Jul 30:** DeepSeek V4 Flash 0731 went live and rejected tool names that didn't match `^[a-zA-Z0-9_-]+$`. The Steel plugin's tools (`steel_scrape`, `steel_session_options`) were fine, but the route through OmniRoute → Steel → CDP was unreliable with empty-stream errors.
4. **Jul 30 (evening):** Decision to revert — Camoufox re-enabled, browser-steel plugin disabled, `CAMOFOX_URL` and `CAMOFOX_API_KEY` uncommented in `.env`. Steel left running on CT201 but Hermes no longer routes through it.
4. **Jul 30 (evening):** Decision to revert — Camoufox re-enabled, browser-steel plugin disabled, `CAMOFOX_URL` and `CAMOFOX_API_KEY` uncommented in `.env`. Steel left running on CT201 but Hermes no longer routes through it. *(Superseded 2026-10-29 — Steel was removed entirely.)*
### What broke
- CDP URL normalization required a plugin patch to handle Docker's internal networking
@@ -48,7 +48,8 @@ Compare the two browser-automation backends available to Hermes, including the J
### Lesson learned
Steel is a capable backend but adds a network hop and another plugin layer. Camofox runs locally on the
Hermes host (CT460) with fewer failure modes. The `browser-steel` plugin was **removed on 2026-10-29** after the endpoint went dark, so returning to Steel means redeploying the plugin, not flipping a flag.
Hermes host (CT460) with fewer failure modes. Steel was **fully removed on 2026-10-29** — plugin deleted, and
the CT201 and CT333 deployments torn down — so returning to Steel means redeploying it, not flipping a flag.
## Detail
@@ -60,13 +61,21 @@ Hermes host (CT460) with fewer failure modes. The `browser-steel` plugin was **r
- **Auth:** Already authenticated into services (Uber Eats, LinkedIn, etc.) — no cookie files needed
- **Limitation:** Headed via Xvfb, consumes display resources
### Steel Browser (disabled but available)
- **Deployed:** 2026-07-29 on CT201, Docker, custom `steel-browser-api:fixed` image
- **Status:** **Offline** (verified 2026-10-29 00:3x) — the host answers ping in 2.5 ms, but CDP `:9223` and REST `:3000` refuse, the tunnel route `/steel` returns **502** and `steel.martinwa.org` no longer resolves
- **CDP endpoint:** `http://100.96.244.39:9223`
- **REST API:** `http://100.96.244.39:3000`
- **UI:** `https://runtipi.kangaroo-eel.ts.net/steel`
- **Plugin:** **removed 2026-10-29 00:13** — `~/.hermes/plugins/browser-steel/` is gone and `plugins.enabled` has no steel entry, so the `plugin_steel` toolset can no longer be selected
### Steel Browser (removed 2026-10-29 — no longer available)
- **Deployed:** 2026-07-29 on CT201, Docker, custom `steel-browser-api:fixed` image *(historical)*
- **Status:** **FULLY REMOVED on 2026-10-29** — not merely offline. On CT201 the containers
`steel-browser-api-1` / `steel-browser-ui-1`, the `steel-network` Docker network, the images
(`steel-browser-api:fixed`, `ghcr.io/steel-dev/steel-browser-api:latest`,
`ghcr.io/steel-dev/steel-browser-ui:latest`) and the project dir `/root/steel-browser` were all deleted
(~6 GB reclaimed). The Tailscale Serve routes `/steel` and `/steel-api` were removed, so the former CDP
`:9223`, REST `:3000` and UI `/steel` addresses are all retired — nothing listens on 3000, 5173 or 9223.
- **CT333 copy:** a second, separate Steel install (`steel-browser`, `ghcr.io/steel-dev/steel-browser:latest`)
was also removed with its image. It had been orphaned — no requests since 2026-10-21, and no reference from
Zeroclaw, Nanobot or any Serve route.
- **Plugin:** **deleted 2026-10-29** — `~/.hermes/plugins/browser-steel/` is gone and no `browser-steel` key
remains in `plugins.disabled` / `plugins.entries`, so the `plugin_steel` toolset cannot be selected.
- **Reversibility:** the compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`, plus
`/root/_archived-steel/steel-cache-archive-20261029.tgz`; the images remain re-pullable from ghcr.io.
## Related
- [[systems/browser-backend]]
+3 -3
View File
@@ -3,8 +3,8 @@ title: Browser Automation
type: concept
status: active
created: 2026-07-22
updated: 2026-10-13
verified_on: 2026-10-13
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [concept, browser, camofox, steel]
sources: [live-probe, session-history]
@@ -31,7 +31,7 @@ Steel Browser (Chromium 140 on CT201) was briefly the primary backend. It offere
- Empty-stream errors from the `opencode-go/deepseek-v4-flash` provider path
- DeepSeek 0731's stricter tool name validation affected all providers
**Lesson:** A local browser backend (same host, no network hop) has fewer failure modes than a remote Docker container. The `browser-steel` plugin is still present but disabled. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]].
**Lesson:** A local browser backend (same host, no network hop) has fewer failure modes than a remote Docker container. The `browser-steel` plugin **was deleted on 2026-10-29** and the Steel deployments on CT201 and CT333 were torn down, so there is no Steel fallback left to fall back to — Camofox is the only backend. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]].
## Related
- [[systems/browser-backend]]
+25 -11
View File
@@ -12,7 +12,7 @@ sources: [live-cron-list, live-sqlite-cron-executions, live-sqlite-sessions, liv
---
# Current State
Updated: 2026-10-28
Updated: 2026-10-29
## Recent Changes (2026-10-28, nightly audit; live probes 2026-10-28 00:2x AWST)
@@ -75,19 +75,33 @@ dirty for the rest of the day. Its incident row `1a8e90_f57ab4cafab8` sits in st
alert** — the only outward sign was the job's own `error` status. The 2026-10-29 audit committed the two
artefacts.
### Steel left the stack: Camofox restored, plugin removed, endpoint dark
### Steel left the stack: fully removed — Camofox is the sole backend
`browser.cloud_provider` and `browser.camera_provider` are back to **`camofox`** (`engine: auto`, `cdp_url: ""`)
in `config.yaml` (mtime **2026-10-29 00:15:13**), with backups taken at every step
(`config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z` 22:00:57, and
`config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z` written 00:13:24). The **`browser-steel` plugin was
removed** from `~/.hermes/plugins/` and from `plugins.enabled`, so the `plugin_steel` toolset is no longer
selectable. Steel itself is **not serving**: the CT201/runtipi host answers ping (2.5 ms) but CDP `:9223` and
REST `:3000` refuse (also on the LAN address), the tunnel route `/steel` returns **502**, and
`steel.martinwa.org` no longer resolves — superseding the earlier "still running on CT201" notes. ⚠
`STEEL_BASE_URL=http://100.96.244.39:3000` remains exported into **10 live Hermes processes** and clears only on
a gateway restart. Detail: [[systems/browser-backend]], [[decisions/rollback-steel-to-camofox]],
[[comparisons/steel-vs-camoufox]].
removed** from `~/.hermes/plugins/`, and no `browser-steel` key remains in `plugins.disabled` or
`plugins.entries`, so the `plugin_steel` toolset is no longer selectable. **Steel is now fully REMOVED, not
merely down:** on **CT201** the containers `steel-browser-api-1` / `steel-browser-ui-1`, the `steel-network`
Docker network, the images (`steel-browser-api:fixed` and both ghcr.io Steel images) and the project dir
`/root/steel-browser` were deleted (~6 GB reclaimed; disk 70G used → 64G used), and the Tailscale Serve routes
`/steel` and `/steel-api` were removed — CT201's remaining routes are only `/` → `127.0.0.1:8085` and `/mgr` →
`127.0.0.1:8084` on **both** `:443` and `:8443`, with nothing listening on 3000, 5173 or 9223. **CT333's second,
separate Steel copy** (`steel-browser`, `ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30)
was removed with its image. Hermes core contains **zero** Steel references, so there is **no Steel fallback path
in the tree** — returning to Steel means redeploying it (the compose file and Dockerfile are archived on CT201
under `/root/_archived-steel/`; the images remain re-pullable from ghcr.io). ⚠ `STEEL_BASE_URL` is no longer in
`~/.hermes/.env` — it survived only in the inherited environment of processes already running at the cleanup,
and the **gateway was restarted on 2026-10-29 01:21 AWST** to clear it (a couple of older, unrelated services
still carry it) — **no core Hermes code reads it**, so it is a runtime-only leftover, **not** a config value.
**Camofox is the sole
backend** and healthy: REST `http://100.118.5.51:9377` (and `http://localhost:9377`), `/health` → **200**,
`/tabs` → `{"running":true,"tabs":[]}`. Detail: [[systems/browser-backend]],
[[decisions/rollback-steel-to-camofox]], [[comparisons/steel-vs-camoufox]].
> ⚠ **Superseded:** an earlier 2026-10-29 00:3x probe described Steel only as "not reachable — endpoint dark,
> still running on CT201". That was already out of date — Steel is **removed**, not down.
### Fleet and tallies
@@ -480,7 +494,7 @@ memory-backend correction still holds and needs no change.
- **OmniRoute CT600/CT601 roles are INVERTED.** CT600 `omniroute` (100.88.81.19) is **running** and serving `:20128` (combo API returns live combos); CT601 `omni` (100.93.204.4) is **stopped** with `onboot=0` (Tailscale peer offline 54d). Hermes resolves `custom:omni` → `omniroute:20128` → **CT600**. → [[systems/omniroute]]
- **A third OmniRoute node exists:** `omniroute-1` at `100.78.63.20:20128`, live (401 = auth-required), registered in Hermes config as provider `omni-101` (5384 models, default `smart-route`). It carries `skills_hub` (model `free-stack`) — the same provider family implicated in the 2026-10-09 timeout storm. Not present in `pct list`; identity unconfirmed. → [[systems/omniroute]]
- **Two Ignis instances exist, on different hosts.** CT460 `ignis` (port 8080, vault `/home/hermes/wiki/agent-estate-wiki`, serving 200) and CT200 `ignis-ignis-1` (image `ignis-server:latest`, host port 8092, vault `/opt/ignis/vaults`, serving 200). Both healthy; they are separate deployments. → [[systems/ignis]]
- **Tailscale route drift fixed:** `/agent-estate` targets bare `http://127.0.0.1:8769` (page had `.../agent-estate-role-dashboard.html`); a tailnet-only `:9443` → `127.0.0.1:5173` Serve was undocumented. CT201's live routes are 8085-based plus `/steel` (5173) and `/steel-api` (3000) — the old `:8443` → `localhost:8082` Homepage row is wrong. → [[infrastructure/tailscale]]
- **Tailscale route drift fixed:** `/agent-estate` targets bare `http://127.0.0.1:8769` (page had `.../agent-estate-role-dashboard.html`); a tailnet-only `:9443` → `127.0.0.1:5173` Serve was undocumented. CT201's live routes are 8085-based plus `/steel` (5173) and `/steel-api` (3000) — the old `:8443` → `localhost:8082` Homepage row is wrong. *(The `/steel` and `/steel-api` routes were deleted on 2026-10-29 — see [[systems/browser-backend]].)* → [[infrastructure/tailscale]]
- **Proxmox inventory rebuilt** from `pct list` + `qm list`: **CT475 `agentzero` and VM110 `omarchy` were missing entirely**; 282 `ubuntu-server-xfce` corrected to stopped; dead rows removed (310, 500, 104, 106, 101). **VM403 `open-claw` and VM401 `hermes-old` are absent from `qm list`** — flagged unconfirmed, not deleted. → [[infrastructure/proxmox]]
- **Nanobot verified on CT333:** `nanobot-gateway` (PID 2259048) alive on 8765/8766. It is **not** a systemd `nanobot.service`/`gateway.service` (both probes return `inactive` — wrong unit names); it runs as the `nanobot` user service plus the gateway process. Port **8900 is no longer listening** — previous "Serve API" claim flagged unverified. → [[agents/nanobot]]
- **Docker services page corrected:** the old "Docker runtime not available" line was a CT460 socket artifact (`/var/run/docker.sock` is `root:docker srw-rw----`; `hermes` user not in group `docker`; no passwordless sudo in cron). Real fleet: CT200 holds ~50 containers, CT333 holds 9. → [[infrastructure/docker-services]]
@@ -610,7 +624,7 @@ _The manual 2026-10-10 stocktake entry follows beneath this one._
- **WebUI connected platforms:** local, api_server, discord, email, homeassistant, mattermost, ntfy, slack, telegram, webhook — Verified, WebUI session, 2026-07-22
## Recent Changes (2026-08-02)
- **Browser backend reverted:** Camoufox restored as primary on 2026-07-30. Steel Browser (CT201) experiment ended — CDP issues + DeepSeek 0731 tool name validation. `browser.cloud_provider: camofox`, `camera_provider: camofox`. Steel plugin disabled. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]].
- **Browser backend reverted:** Camoufox restored as primary on 2026-07-30. Steel Browser (CT201) experiment ended — CDP issues + DeepSeek 0731 tool name validation. `browser.cloud_provider: camofox`, `camera_provider: camofox`. Steel plugin disabled *(the 2026-08-02 state — the plugin and the Steel deployments were later removed entirely on 2026-10-29, see [[systems/browser-backend]])*. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]].
- **Mnemosyne capture fix (2026-08-01):** Automatic conversational capture was broken by stringified YAML (`sync_roles: "['user']"`). Fixed by rewriting structured Mnemosyne settings with native YAML types. See [[incidents/2026-08-01-mnemosyne-capture-failure]].
- **Mnemosyne vector coverage gap discovered:** Only 159/1,476 working memories (10.8%) have vector embeddings. 0/458 episodic memories have embeddings. Semantic search is mostly FTS5 fallback.
- **DeepSeek V4 Flash 0731:** Post-training bump, same model ID, same pricing. Terminal Bench 2.1: 82.7 (up from 61.8). Stricter tool name validation (`^[a-zA-Z0-9_-]+$`) — may cause issues with OmniRoute-transformed tool names.
+9 -6
View File
@@ -3,8 +3,8 @@ title: Decisions
type: collection
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [decisions]
---
@@ -19,14 +19,17 @@ Stable strategic and config decisions, with evidence and status.
- [[decisions/persistent-shell-enabled]]
- [[decisions/rollback-steel-to-camofox]]
> **Status sweep 2026-10-20:** all three listed decisions are still `status: active`, but two now
> carry a review note:
> **Status sweep 2026-10-20:** all three listed decisions are still `status: active`, but each now
> carries a note:
> - [[decisions/keep-openrouter-as-fallback]] — its 30-day revisit trigger is **met**; OpenRouter is
> reachable only through the OmniRoute catalog, not as a standalone provider.
> - [[decisions/persistent-shell-enabled]] — still `true`; its `review_after: 0001-01-01` zero-date
> bug is now corrected to a real date.
> - [[decisions/rollback-steel-to-camofox]] — the *engine* is Camoufox as decided, but
> `cloud_provider: steel` remains in the config while the Steel service itself is unreachable.
> - [[decisions/rollback-steel-to-camofox]] — ~~the *engine* is Camoufox as decided, but
> `cloud_provider: steel` remains in the config while the Steel service itself is unreachable.~~
> **Superseded 2026-10-29:** `cloud_provider` is `camofox` again and Steel is **fully removed** —
> plugin deleted, CT201 and CT333 deployments torn down, Serve routes deleted. Camofox is the sole
> backend.
## History
- **Verified:** Decision pages initialized during wiki setup on 2026-07-22
+46 -17
View File
@@ -28,20 +28,26 @@ Steel Browser was deployed on CT201 on 2026-07-29 as the primary browser automat
## Decision
**Revert to Camoufox as primary.** Steel plugin disabled but left running on CT201.
> **Status check 2026-10-20 — the decision is *partially* undone in configuration.**
> The live `browser:` block in `~/.hermes/config.yaml` reads
> `cloud_provider: steel` with `engine: camofox`. The **engine** is Camoufox (decision honoured),
> but the **cloud_provider** key still says `steel`, and `camera_provider: camofox`.
> ⚠ **Superseded 2026-10-29:** the "left running on CT201" half of this decision no longer holds — Steel is
> now **fully removed** from the estate. See the Update below.
> **Status check 2026-10-20 — the decision is *partially* undone in configuration.** *(superseded 2026-10-29)*
> The live `browser:` block in `~/.hermes/config.yaml` read
> `cloud_provider: steel` with `engine: camofox`. The **engine** was Camoufox (decision honoured),
> but the **cloud_provider** key said `steel`, and `camera_provider: camofox`.
> This is the same split seen in [[comparisons/camoufox-vs-chrome]].
>
> Meanwhile the Steel service itself is **not reachable**: `http://steel.martinwa.org` returns
> Meanwhile the Steel service itself was **not reachable**: `http://steel.martinwa.org` returned
> **000** (connection failed) on 2026-10-20. Combined with the degraded CT201 stack recorded in
> [[systems/homepage-dashboard]] (main container removed, asset proxy 502), it looks like the
> "left running on CT201" part of this decision is no longer true.
> [[systems/homepage-dashboard]] (main container removed, asset proxy 502), it looked like the
> "left running on CT201" part of this decision was no longer true.
>
> **Recommendation, not actioned here:** the `cloud_provider: steel` value appears to be residue
> **Recommendation, not actioned here:** the `cloud_provider: steel` value appeared to be residue
> from the reverted deployment. Confirm against the live Steel service and either restore it or
> set `cloud_provider: camofox` so the config matches the decision this page records.
>
> **Resolved 2026-10-29:** `cloud_provider` is `camofox` again and Steel has been removed entirely — no
> restoration was needed or wanted.
## Rationale
1. **Reliability:** Camoufox runs on the same host as Hermes (CT450) — no Docker networking, no CDP URL normalization patches, no container-to-host routing issues.
@@ -52,14 +58,14 @@ Steel Browser was deployed on CT201 on 2026-07-29 as the primary browser automat
## Consequences
- **Positive:** Browser automation is stable again. Uber Eats report ran successfully via Camofox on 2026-08-01.
- **Negative:** Steel's session viewer UI and full CDP access are no longer available to Hermes.
- **Neutral:** Steel still runs on CT201 — can be re-enabled by uncommenting `.env` lines and enabling the plugin.
- **Neutral (2026-07-30 — now superseded):** Steel still ran on CT201 and could be re-enabled by uncommenting `.env` lines and enabling the plugin. **As of 2026-10-29 that is no longer true** — Steel is removed, so re-enabling means redeploying it (compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`; images remain re-pullable from ghcr.io).
## Evidence
- Camoufox verified working Jul 30: `browserConnected: true`, `sessions: 1`, `tabs: 1`, active 200s
- Uber Eats report (Aug 1) successfully scraped via Camofox REST API — 11 orders, $264.51
- Steel plugin `browser-steel` still present at `/home/hermes/.hermes/plugins/browser-steel/` — disabled
- Steel plugin `browser-steel` was present at `/home/hermes/.hermes/plugins/browser-steel/` — disabled *(2026-07-30 state; the plugin has since been deleted — see the Update below)*
## Update (2026-10-29) — the plugin was removed and the endpoint went dark
## Update (2026-10-29) — Steel was fully removed, not just darkened
The 2026-07-30 rollback disabled the Steel plugin but left it on disk and left Steel running. Both of those
were closed on **2026-10-28 22:00 – 2026-10-29 00:15**: `browser.cloud_provider` / `camera_provider` were
@@ -67,12 +73,35 @@ returned to **`camofox`** (backups `config.yaml.bak-pre-cloudprovider-fix-202610
`config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z`), and **`~/.hermes/plugins/browser-steel/` was
removed**, so the `plugin_steel` toolset is no longer selectable.
Steel itself is **not serving**: CDP `:9223` and REST `:3000` on `100.96.244.39` refuse, the tunnel route
`/steel` returns **502**, and `steel.martinwa.org` does not resolve, while the host still answers ping.
`STEEL_BASE_URL=http://100.96.244.39:3000` remains exported in 10 live Hermes processes and clears on the next
gateway restart. The decision below is unchanged — Camofox is the backend — but there is now **no Steel plugin
fallback in the tree**, so returning to Steel would mean redeploying it. Detail:
[[systems/browser-backend]], [[comparisons/steel-vs-camoufox]].
**Steel is now FULLY REMOVED — it is not merely "down" or "not reachable":**
- **Hermes config:** the `browser:` block reads exactly `cloud_provider: camofox`, `camera_provider: camofox`,
`engine: auto`, `cdp_url: ""`. No `browser-steel` key remains in `plugins.disabled` or `plugins.entries`, and
`STEEL_BASE_URL` is gone from `~/.hermes/.env` (it survived only in the inherited environment of processes
already running at the cleanup; the **gateway was restarted on 2026-10-29 01:21 AWST** to clear it, while a
couple of older, unrelated services still carry it — **no core Hermes code reads it**, so it is a
runtime-only leftover, not a config value).
- **Hermes plugins:** `~/.hermes/plugins/browser-steel/` is deleted. Hermes core source contains **zero**
Steel references, so there is **no Steel fallback path in the tree** — returning to Steel means redeploying
the plugin, not flipping a flag.
- **CT201 (runtipi):** containers `steel-browser-api-1` and `steel-browser-ui-1`, the `steel-network` Docker
network, the images `steel-browser-api:fixed` / `ghcr.io/steel-dev/steel-browser-api:latest` /
`ghcr.io/steel-dev/steel-browser-ui:latest`, and the project dir `/root/steel-browser` were all removed
(~6 GB reclaimed; disk 70G used → 64G used). The Tailscale Serve routes `/steel` and `/steel-api` were
deleted with `tailscale serve --set-path <path> off`; CT201's remaining routes are only `/` → `127.0.0.1:8085`
and `/mgr` → `127.0.0.1:8084` on both `:443` and `:8443`. Nothing listens on 3000, 5173 or 9223.
- **CT333 (clawtest):** a second, separate and orphaned Steel install — the `steel-browser` container
(`ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30, publishing port 3000) and its image
were removed as well. It had no Tailscale Serve route, no entry in `~/.zeroclaw/config.toml` (which uses
`backend = "agent_browser"`), and no reference from Nanobot — its last CDP navigation was 2026-10-20 and it
received no requests after 2026-10-21.
- **Reversibility:** the compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`, plus
`/root/_archived-steel/steel-cache-archive-20261029.tgz`; the images remain re-pullable from ghcr.io.
- **Camofox is the sole backend** and healthy — REST `http://100.118.5.51:9377` (also `http://localhost:9377`),
`/health` → **200**, `/tabs` → `{"running":true,"tabs":[]}`.
The decision itself is unchanged — Camofox is the backend — but the "Steel is still there if you need it"
assumption is dead. Detail: [[systems/browser-backend]], [[comparisons/steel-vs-camoufox]].
## Related
- [[comparisons/steel-vs-camoufox]]
+12 -1
View File
@@ -1,3 +1,14 @@
---
title: Anthony's Systems Wiki
type: collection
status: active
created: 2026-07-22
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [index, infrastructure, systems]
---
# Anthony's Systems Wiki
> Operational knowledge for Anthony's agents, infrastructure,
> model routing and automation. Last updated: 2026-10-29 (nightly audit — 1 new incident page, 9 pages updated)
@@ -24,7 +35,7 @@
- [[systems/openrouter]] — OpenRouter current state and history
- [[systems/opencode-go]] — OpenCode Go subscription, model limits, pricing
- [[systems/nanobot]] — Nanobot current state and history
- [[systems/browser-backend]] — Camofox (primary, `systemd --user` on CT460, REST `:9377`); **Steel was removed from the stack on 2026-10-29** (plugin deleted, `cloud_provider` back to `camofox`, endpoint dark, `STEEL_BASE_URL` still exported in 10 live processes); memory-containment drop-in and the 2026-10-12 ceiling event documented
- [[systems/browser-backend]] — Camofox (primary, `systemd --user` on CT460, REST `:9377`) — **the only backend**; **Steel was fully removed from the stack on 2026-10-29** (plugin deleted, `cloud_provider` back to `camofox`, CT201 and CT333 deployments torn down, Serve routes deleted; `STEEL_BASE_URL` survives only in already-running process environments); memory-containment drop-in and the 2026-10-12 ceiling event documented
- [[systems/hermes-ui]] — Browser Hermes Desktop (przbadu/hermes-ui)
- [[systems/paseo]] — Paseo coding-agent daemon + web terminal workspace (CT460, :6767)
- [[systems/ignis]] — Ignis browser-native Obsidian (wiki interface)
+11 -8
View File
@@ -3,8 +3,8 @@ title: Docker Services
type: infrastructure
status: mixed
created: 2026-07-22
updated: 2026-10-11
verified_on: 2026-10-11
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [docker, homelab, infrastructure]
sources: [raw/configs/hermes-config-sanitized.txt, live-probe]
@@ -21,7 +21,7 @@ Docker runs on **three hosts**, and CT460's socket is not readable by the agent
| Host | Access | Containers running |
|---|---|---|
| CT200 `docker` (`100.96.150.69`) | `pct exec 200 -- docker ps` via PVE | **~50** — main Docker host |
| CT333 `clawtest` (`100.122.189.81`) | `pct exec 333 -- docker ps` via PVE | 9 (steel-browser, gotify, readlater, kokoro-tts, edge-tts, onecli-postgres, …) |
| CT333 `clawtest` (`100.122.189.81`) | `pct exec 333 -- docker ps` via PVE | 8 (gotify, readlater, kokoro-tts, edge-tts, onecli-postgres, …) — the `steel-browser` container and its image were **removed 2026-10-29** |
| CT460 `hermes` | `sudo docker` — socket is `root:docker srw-rw----`, `hermes` user not in group `docker`; **no passwordless sudo in cron** | `ignis` (confirmed running; inspect requires root) |
## CT200 live inventory (verified 2026-10-11, ~50 containers)
@@ -31,17 +31,15 @@ Docker runs on **three hosts**, and CT460's socket is not readable by the agent
**Note the duplicate Ignis:** `ignis-ignis-1` on CT200 is a separate deployment from CT460's `ignis`. See [[systems/ignis]].
## CT201 runtipi — Tailscale routes (verified 2026-10-11)
## CT201 runtipi — Tailscale routes (verified 2026-10-11; Steel routes removed 2026-10-29)
| Host | Path | Target |
|---|---|---|
| `runtipi.kangaroo-eel.ts.net:8443` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
| `runtipi.kangaroo-eel.ts.net` (tailnet) | `/` | `http://127.0.0.1:8085` |
| | `/mgr` | `http://127.0.0.1:8084` |
| | `/steel` | `http://127.0.0.1:5173` |
| | `/steel-api` | `http://127.0.0.1:3000` |
**Correction:** this page previously said "Port 443 occupied by Traefik on CT201 — Tailscale uses port 8443 there" and listed CT201's `:8443` → `localhost:8082` for Homepage. Live `tailscale serve status` on CT201 shows the target is `127.0.0.1:8085` on both `:8443` and the default host, plus a `/steel` (5173) and `/steel-api` (3000) pair. The `/steel-api` route does not strip the path prefix — use the container IP directly for API calls.
**Correction:** this page previously said "Port 443 occupied by Traefik on CT201 — Tailscale uses port 8443 there" and listed CT201's `:8443` → `localhost:8082` for Homepage. Live `tailscale serve status` on CT201 shows the target is `127.0.0.1:8085` on both `:8443` and the default host. That table also carried a `/steel` (5173) and `/steel-api` (3000) pair — **both routes were deleted on 2026-10-29** with `tailscale serve --set-path <path> off` when Steel was removed, so CT201's remaining routes are only `/` and `/mgr` on **both** `:443` and `:8443`. (The `/steel-api` route never stripped the path prefix — moot now that it is gone.)
## Confirmed Compose Files
@@ -50,7 +48,12 @@ Docker runs on **three hosts**, and CT460's socket is not readable by the agent
| Hermes Agent | `/home/hermes/.hermes/hermes-agent/docker-compose.yml` | hermes, hermes-dashboard |
| Hermes WebUI | `/home/hermes/hermes-webui/docker-compose.yml` | webui |
| ReadLater | `/home/hermes/workspace/readlater/docker-compose.yml` | readlater, edge-tts, kokoro-tts |
| Steel Browser | `/root/steel-browser/docker-compose.yml` (CT201) | steel-browser-api, steel-browser-ui |
| Ignis | `/home/hermes/docker/ignis/docker-compose.yml` (CT460) | ignis, ignis-data, ignis-obsidian-app |
**Steel Browser — removed 2026-10-29.** The CT201 compose file and Dockerfile are no longer live: the project
dir `/root/steel-browser` was deleted along with its containers, images and network. Both files are archived at
`/root/_archived-steel/` on CT201 (with `/root/_archived-steel/steel-cache-archive-20261029.tgz`), and the
images remain re-pullable from ghcr.io. → [[comparisons/steel-vs-camoufox]]
## Service Definitions (from compose files)
+8 -3
View File
@@ -3,8 +3,8 @@ title: Known Hosts and Paths
type: infrastructure
status: partial
created: 2026-07-22
updated: 2026-10-26
verified_on: 2026-10-26
updated: 2026-10-29
verified_on: 2026-10-29
confidence: medium
tags: [infrastructure, hosts, containers, paths]
sources: []
@@ -20,7 +20,7 @@ sources: []
| litellm | LiteLLM proxy | `http://litellm:4000/v1` | Yes (from config) |
| `custom:Omniroute:20128` | OmniRoute proxy (CT600 node `omniroute`) | `http://omniroute:20128/v1` → `100.88.81.19:20128` | Yes (live probe 2026-10-26: 6,480 models) |
| `custom:omni-101` | OmniRoute proxy (CT101 node `omniroute-1`) — **the live default provider** | `http://100.78.63.20:20128` | Yes (live probe 2026-10-26) |
| CT201 | Steel Browser, Runtipi, Homepage dashboard | `100.96.244.39` (Tailscale), `192.168.178.166` (LAN) | Yes (live probe) |
| CT201 | Runtipi, Homepage dashboard | `100.96.244.39` (Tailscale), `192.168.178.166` (LAN) | Yes (live probe) |
| CT460 | Hermes host, Camoufox, cua-driver | `hermes.kangaroo-eel.ts.net`, `100.118.5.51` (Tailscale) | Yes |
| PVE `pve` | Proxmox VE hypervisor hosting CT460 and the other `pve` guests | `100.96.100.82` (Tailscale), `pve.kangaroo-eel.ts.net` | Yes (live `pct list` 2026-10-26) |
| PVE `m93p` | Second, unclustered Proxmox VE 9.2.10 hypervisor | `100.104.51.98` (Tailscale) | Yes (live `pct list` 2026-10-26) |
@@ -69,5 +69,10 @@ sources: []
- Individual LXCs are verified only when probed directly; the two `pct list` snapshots above are the current authority for what is running
- Guests not present in those `pct list` outputs are **not** running on either hypervisor — do not assume existence from older session notes
> **Note (2026-10-29):** CT201 no longer hosts the Steel Browser — the containers, images, network, project dir
> and Tailscale Serve routes were all removed when Steel was deleted from the estate. CT333's separate,
> orphaned Steel copy was removed at the same time. → [[comparisons/steel-vs-camoufox]]
## Related
- [[docker-services]]
- [[infrastructure/tailscale]]
+9 -5
View File
@@ -3,8 +3,8 @@ title: Tailscale
type: infrastructure
status: active
created: 2026-07-26
updated: 2026-10-28
verified_on: 2026-10-28
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [tailscale, vpn, networking, infrastructure]
---
@@ -124,11 +124,15 @@ Verified live on 2026-10-26 via `pct exec 201 -- tailscale serve status` on `pve
| `runtipi.kangaroo-eel.ts.net:8443` (tailnet only) | `/mgr` | `http://127.0.0.1:8084` | `homepage-manager` container — **200** |
| `runtipi.kangaroo-eel.ts.net:443` (tailnet only) | `/` | `http://127.0.0.1:8085` | Same asset proxy — **502** (re-probed 2026-10-26) |
| `runtipi.kangaroo-eel.ts.net:443` (tailnet only) | `/mgr` | `http://127.0.0.1:8084` | `homepage-manager` — **200** |
| `runtipi.kangaroo-eel.ts.net:443` (tailnet only) | `/steel` | `http://127.0.0.1:5173` | Steel Browser UI — **200** on :443 (re-probed 2026-10-26). ⚠ **`/steel` is NOT served on :8443** — it 502s there |
| `runtipi.kangaroo-eel.ts.net:443` (tailnet only) | `/steel-api` | `http://127.0.0.1:3000` | Steel Browser API (path prefix not stripped — use direct IP for API calls) |
**Steel routes removed 2026-10-29.** This table previously carried two more rows on `:443` — `/steel` →
`http://127.0.0.1:5173` and `/steel-api` → `http://127.0.0.1:3000`. Both were deleted with
`tailscale serve --set-path <path> off` when Steel was removed from CT201, so CT201's only remaining routes
are `/` → `127.0.0.1:8085` and `/mgr` → `http://127.0.0.1:8084`, on **both** `:443` and `:8443`.
→ [[comparisons/steel-vs-camoufox]]
Two corrections to the pre-2026-10-25 table:
1. **`:8443` is served by `tailscaled` on CT201, but so is `:443`.** `ss -ltnp` on CT201 shows `tailscaled` bound to `100.96.244.39:443`, and the serve table above lists routes on **both** ports. The old note claiming "Port 443 occupied by Traefik on CT201" is **false as of 2026-10-25** — no Traefik container or process exists on CT201 (`docker ps -a` and `ps aux` both came back empty for `traefik`), and Tailscale owns 443 there. The Homebox/steel path prefix is not stripped on `/steel-api`.
1. **`:8443` is served by `tailscaled` on CT201, but so is `:443`.** `ss -ltnp` on CT201 shows `tailscaled` bound to `100.96.244.39:443`, and the serve table above lists routes on **both** ports. The old note claiming "Port 443 occupied by Traefik on CT201" is **false as of 2026-10-25** — no Traefik container or process exists on CT201 (`docker ps -a` and `ps aux` both came back empty for `traefik`), and Tailscale owns 443 there.
2. The old table routed `:8443` straight to `http://localhost:8082`. That port has **no listener**; the live target is `8085` (the `asset_proxy.py` front door), which itself proxies to the dead 8082 and returns 502. → [[systems/homepage-dashboard]]
## Atomic Restore Script
+15
View File
@@ -1515,3 +1515,18 @@ not pushed.
- **Live probes (2026-10-29 00:3x AWST):** `jobs.json` 29 jobs / 26 enabled / 3 disabled, 5 non-clean `last_status`, pins 12 `smart-route` / 6 `free-stack` / 7 unpinned / 1 `deepseek-paid-smart`; `executions.db` — 2026-10-28 settled **168 / 2 / 1 running**, 2026-10-27 now **173 / 1**, 2026-10-29 so far 6 / 1 running; Steel: ping 2.5 ms but CDP `:9223` and REST `:3000` refuse, tunnel `/steel` **502**, `steel.martinwa.org` unresolved; no Hindsight service on this box (`:8888` closed, no `hindsight` units) so [[systems/memory-backend]] and [[comparisons/memory-backends]] were left unchanged.
- **Removed/archived:** none. **Secrets written:** none.
- **Unresolved:** the radar's discovery stage still cannot reach its own handoff write (two ceilings in two days); `1a8e90_f57ab4cafab8` sits `detected` without an alert; both radar incident rows close only on a clean 07:00 weekday slot.
## [2026-10-29] update | Steel Browser fully removed from the estate — Camofox is the sole backend
- **Root cause of the long-standing `cloud_provider: steel` residue:** the 2026-07-30 rollback disabled the `browser-steel` plugin and set `browser.engine: camofox`, but left `browser.cloud_provider: steel` in `~/.hermes/config.yaml`. Every browser tool call then routed through `_instantiate_explicit_cloud_provider()` and failed with *"no registered browser plugin has that name: steel"*. Because `engine` was already correct, the split config was inert-but-wrong and survived ~3 months of audits.
- **Hermes (CT460):** `browser:` block now reads exactly `cloud_provider: camofox`, `camera_provider: camofox`, `engine: auto`, `cdp_url: ""`. The `browser-steel` plugin directory was **deleted**, and the `browser-steel` key was removed from both `plugins.disabled` and `plugins.entries`. `STEEL_BASE_URL` was dropped from `~/.hermes/.env`. **Hermes core source contains zero Steel references**, so there is no Steel fallback path in the tree — returning to Steel now means redeploying it.
- **CT201 (runtipi):** the entire Steel deployment was torn down — containers `steel-browser-api-1` / `steel-browser-ui-1`, the `steel-network` Docker network, the images `steel-browser-api:fixed` and both ghcr.io Steel images, and the project dir `/root/steel-browser` (~6 GB reclaimed; disk 70G → 64G used). The Tailscale Serve routes `/steel` (5173) and `/steel-api` (3000) were deleted with `tailscale serve --set-path <path> off`; CT201's remaining routes are only `/` → `127.0.0.1:8085` and `/mgr` → `http://127.0.0.1:8084`, on **both** `:443` and `:8443`. Nothing listens on 3000, 5173 or 9223.
- **CT333 (clawtest):** a **second, separate and orphaned** Steel install was found and removed — the `steel-browser` container (`ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30, publishing port 3000) and its image. It had no Serve route, no entry in `~/.zeroclaw/config.toml` (which uses `backend = "agent_browser"` on 127.0.0.1:9515), no reference from Nanobot's config or the CLI-anything browser harness; its last CDP navigation was 2026-10-20, it received no requests after 2026-10-21, and its page crashed 2026-10-22.
- **Estate-wide sweep:** every LXC on the PVE host was checked — no Steel containers, images, networks or listeners remain anywhere. Homarr, the homepage dashboard and the runtipi app configs had no Steel references.
- **Gateway restarted 01:21 AWST** to clear `STEEL_BASE_URL` from the inherited process environment; the new gateway tree (PID `3913841`) reads `STEEL_BASE_URL=0`, and Telegram, Discord, ntfy, email and Home Assistant all reconnected cleanly. The dashboard unit and `agentmail-webhook.service` still carry the variable in their inherited environment, but **no core code reads it**.
- **Reversibility preserved:** CT201's compose file and Dockerfile are archived under `/root/_archived-steel/`, plus `/root/_archived-steel/steel-cache-archive-20261029.tgz`; the images remain re-pullable from ghcr.io. Backups: `config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z`, `config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z`, `.env.bak-pre-steel-cleanup-20261028T220947Z`.
- **Camofox verified healthy** after every change: REST `http://100.118.5.51:9377` (and `http://localhost:9377`), `/health` → **200**, `/tabs` → `{"running":true,"tabs":[]}`, and live navigation confirmed working.
- **Pages corrected (15):** [[systems/browser-backend]] · [[decisions/rollback-steel-to-camofox]] · [[decisions/index]] · [[comparisons/steel-vs-camoufox]] · [[comparisons/camoufox-vs-chrome]] · [[infrastructure/docker-services]] · [[infrastructure/tailscale]] · [[infrastructure/hosts]] · [[systems/homepage-dashboard]] · [[concepts/browser-automation]] · [[runbooks/recover-docker-service]] · [[current-state]] · [[index]] · [[agents/nanobot]] · [[systems/scheduled-tasks]].
- **Corrects the 2026-10-29 00:3x audit entry**, which described Steel as merely "not reachable / endpoint dark". It is **removed**, not down. Per SCHEMA rule 9 the superseded claims are marked, not deleted.
- **Removed/archived:** none. **Secrets written:** none (variable names only; no key or URL values).
- **Unresolved:** the obsolete CT333 line in Nanobot's Obsidian vault (`Areas/Homelab/architecture.md`, "Steel Browser | Clawtest | http://localhost:3000") still stands — that vault is outside this wiki and was not modified. Also noted, unrelated to Steel: `Browser Use viewing unavailable: unsupported Hermes browser resolver` has logged in the gateway since 2026-10-11.
+123
View File
@@ -0,0 +1,123 @@
# Wiki freshness audit — 2026-10-29
Threshold: >21 days since `updated`
## STALE (2)
- [ 23d] incidents/2026-10-01-cliproxy-connection-auth-expired.md (updated 2026-10-06)
- [ 23d] incidents/2026-10-05-dead-model-pin-repin.md (updated 2026-10-06)
## FRESH (105)
- [ 9d] CONTRIBUTING.md
- [ 3d] agents/hermes-chaos.md
- [ 3d] agents/hermes-production.md
- [ 9d] agents/index.md
- [ 0d] agents/nanobot.md
- [ 9d] agents/open-claw.md
- [ 18d] agents/qwenpaw.md
- [ 9d] agents/zeroclaw.md
- [ 0d] comparisons/camoufox-vs-chrome.md
- [ 9d] comparisons/hermes-vs-nanobot.md
- [ 9d] comparisons/memory-backends.md
- [ 9d] comparisons/model-routers.md
- [ 0d] comparisons/steel-vs-camoufox.md
- [ 9d] concepts/agent-memory.md
- [ 0d] concepts/browser-automation.md
- [ 9d] concepts/model-routing.md
- [ 9d] concepts/persistent-shells.md
- [ 9d] concepts/tool-calling.md
- [ 0d] current-state.md
- [ 0d] decisions/index.md
- [ 9d] decisions/keep-openrouter-as-fallback.md
- [ 9d] decisions/persistent-shell-enabled.md
- [ 0d] decisions/rollback-steel-to-camofox.md
- [ 9d] experiments/index.md
- [ 5d] incidents/2026-07-29-mnemosyne-relay.md
- [ 9d] incidents/2026-08-01-mnemosyne-capture-failure.md
- [ 5d] incidents/2026-08-04-omniroute-free-deepseek-silent-stops.md
- [ 7d] incidents/2026-08-08-gitea-config-backup-push-dead.md
- [ 6d] incidents/2026-08-15-skills-index-drift.md
- [ 17d] incidents/2026-08-18-mnemosyne-capture-pipeline.md
- [ 4d] incidents/2026-08-26-work-outlook-basic-auth-disabled.md
- [ 9d] incidents/2026-08-29-gmail-read-state.md
- [ 8d] incidents/2026-09-02-sftpgo-outage.md
- [ 18d] incidents/2026-09-28-hermes-managed-runtime-deps.md
- [ 3d] incidents/2026-10-03-ct450-hermes-update.md
- [ 3d] incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md
- [ 5d] incidents/2026-10-09-cron-provider-timeout-storm.md
- [ 20d] incidents/2026-10-09-memory-recall-relevance.md
- [ 13d] incidents/2026-10-14-omniroute-node-connection-refused-storm.md
- [ 2d] incidents/2026-10-16-omniroute-503-resource-pressure.md
- [ 10d] incidents/2026-10-18-degenerate-stream-canned-substitute.md
- [ 2d] incidents/2026-10-20-gateway-unclean-exit-orphaned-run.md
- [ 7d] incidents/2026-10-21-omniroute-pre-dispatch-filter-skip.md
- [ 4d] incidents/2026-10-22-conduit-memory-start-timeout.md
- [ 3d] incidents/2026-10-22-nous-portal-refresh-token-invalid.md
- [ 3d] incidents/2026-10-24-conduit-call-request-refused.md
- [ 3d] incidents/2026-10-25-gmail-digest-runaway-and-refire.md
- [ 1d] incidents/2026-10-26-planned-restart-orphaned-cron-run.md
- [ 0d] incidents/2026-10-27-job-radar-budget-exhausted.md
- [ 0d] incidents/2026-10-28-job-radar-stage1-iteration-cap.md
- [ 0d] index.md
- [ 7d] infrastructure/backups.md
- [ 9d] infrastructure/credentials-map.md
- [ 0d] infrastructure/docker-services.md
- [ 9d] infrastructure/domains-and-tunnels.md
- [ 0d] infrastructure/hosts.md
- [ 15d] infrastructure/network-map.md
- [ 15d] infrastructure/proxmox.md
- [ 0d] infrastructure/tailscale.md
- [ 17d] queries/retained-research-answers.md
- [ 4d] reports/communities-wa-social-media-audit-2026.md
- [ 8d] reports/uber-eats/2026-07.md
- [ 8d] reports/uber-eats/2026-08.md
- [ 8d] reports/uber-eats/2026-09.md
- [ 8d] reports/uber-eats/index.md
- [ 9d] runbooks/backup-wiki.md
- [ 9d] runbooks/diagnose-docker.md
- [ 9d] runbooks/gateway-resource-resilience.md
- [ 0d] runbooks/job-radar.md
- [ 9d] runbooks/multiplexer-setup.md
- [ 3d] runbooks/provider-health.md
- [ 0d] runbooks/recover-docker-service.md
- [ 9d] runbooks/restart-browser.md
- [ 9d] runbooks/restart-hermes.md
- [ 9d] runbooks/restore-openrouter.md
- [ 9d] runbooks/test-openrouter.md
- [ 9d] runbooks/troubleshoot-tool-loop.md
- [ 3d] runbooks/update-hermes-safely.md
- [ 0d] systems/browser-backend.md
- [ 9d] systems/cloudflare-tunnel.md
- [ 1d] systems/conduit.md
- [ 3d] systems/current-profile.md
- [ 9d] systems/guanaco.md
- [ 9d] systems/headless-server.md
- [ 1d] systems/hermes-agent.md
- [ 9d] systems/hermes-ui.md
- [ 17d] systems/home-assistant.md
- [ 0d] systems/homepage-dashboard.md
- [ 18d] systems/ignis.md
- [ 3d] systems/memory-backend.md
- [ 3d] systems/messaging-integrations.md
- [ 5d] systems/mnemosyne.md
- [ 3d] systems/model-providers.md
- [ 17d] systems/nanobot.md
- [ 9d] systems/nextcloud.md
- [ 9d] systems/obsidian.md
- [ 7d] systems/omniroute.md
- [ 9d] systems/opencode-go.md
- [ 9d] systems/openrouter.md
- [ 18d] systems/paseo.md
- [ 10d] systems/push-notifications.md
- [ 0d] systems/scheduled-tasks.md
- [ 0d] systems/skills-index.md
- [ 9d] systems/terminal-backend.md
- [ 9d] systems/tool-search.md
## NO updated DATE (3)
- SCHEMA.md
- experiments/active/example.md
- log.md
TOTAL: 2 stale, 105 fresh, 3 no-date of 110 pages
+6 -4
View File
@@ -3,8 +3,8 @@ title: Recover a Docker service
type: runbook
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
updated: 2026-10-29
verified_on: 2026-10-29
tags: [runbook, docker, recovery]
sources: []
confidence: high
@@ -30,5 +30,7 @@ Container shows healthy after recovery probe reboots.
> - The **CT201 homepage dashboard** is degraded — main container removed, `asset_proxy.py` on 8085
> proxying to a dead 8082, and the Tailscale `/` route returning 502. See
> [[systems/homepage-dashboard]].
> - The **Steel browser** service is unreachable (`steel.martinwa.org` → connection failure).
> See [[decisions/rollback-steel-to-camofox]].
> - The **Steel browser** is **not a recovery case** — it was fully removed on 2026-10-29 (plugin,
> containers, images, network, project dir and Tailscale Serve routes deleted on CT201 and CT333), so
> there is no service to bring back. The browser backend is Camofox on CT460; see
> [[systems/browser-backend]] and [[decisions/rollback-steel-to-camofox]].
+43 -26
View File
@@ -20,7 +20,7 @@ Browser automation for Hermes — web search, form filling, screenshot, CDP cont
- **Provider config:** `browser.cloud_provider: camofox`, `camera_provider: camofox`
- **REST API:** **`:9377`** — live-verified 2026-10-13 (`node server.js`, `CAMOFOX_PORT=9377`; `LISTEN *:9377`). **The `:9093` figure in earlier revisions of this page is stale** — nothing was listening on 9093 on 2026-10-13.
- **VNC viewer:** NovNC `websockify` on `127.0.0.1:6080` → `127.0.0.1:5900`, published as `https://ct460.kangaroo-eel.ts.net/vnc-camofox`
- **Steel Browser (CT201):** **Not reachable as of 2026-10-29 00:3x** — host answers ping (2.5 ms) but CDP `:9223` and REST `:3000` refuse; plugin removed. See the 2026-10-29 correction below.
- **Steel Browser (CT201):** **Fully removed on 2026-10-29** — plugin deleted, CT201 deployment torn down, Serve routes deleted. **Camofox is the sole backend.** See the 2026-10-29 correction below.
## Camofox (local `systemd --user` service)
@@ -87,12 +87,11 @@ Restart (`systemctl --user restart camofox-browser.service`) is measurable rough
baseline: `Memory: 16.5M` cgroup, `peak: 1.4G`, `swap peak: 234.5M`, 16 tasks, `browserConnected: false`
until the next session starts.
## Steel Browser (CT201) — Disabled
- **Status:** Still running on CT201, Hermes not connected since 2026-07-30
- **Plugin:** `browser-steel` at `/home/hermes/.hermes/plugins/browser-steel/` — disabled
- **CDP:** `http://100.96.244.39:9223`
- **REST API:** `http://100.96.244.39:3000`
- **UI:** `https://runtipi.kangaroo-eel.ts.net/steel`
## Steel Browser (CT201) — Removed (2026-10-29)
- **Status:** **Fully removed.** Nothing Steel-related survives on CT201 or CT333 — containers, images, network, project dir and Serve routes are all gone. Camofox is the only browser backend.
- **Plugin:** `browser-steel` — **deleted** (`~/.hermes/plugins/browser-steel/` no longer exists; no `browser-steel` key in `plugins.disabled` or `plugins.entries`)
- **CDP / REST / UI:** **retired** — the former CDP `:9223`, REST `:3000` and `/steel` UI addresses no longer serve anything
- **Reversibility:** the compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`, and the images remain re-pullable from ghcr.io
- **See:** [[comparisons/steel-vs-camoufox]] for the full experiment story
## History
@@ -108,29 +107,47 @@ until the next session starts.
"Camoufox UI") does not exist; `CT450` host attribution replaced with the local `systemd --user` unit on
the Hermes host (CT460). Containment drop-in documented. Same day, the 07:01 Job Radar session was audited
end-to-end and its memory-pressure evidence added above.
- **2026-10-29 — Steel fully removed.** Config back to `cloud_provider: camofox` / `camera_provider: camofox`,
the `browser-steel` plugin deleted, `STEEL_BASE_URL` dropped from `~/.hermes/.env`, and the Steel deployments
on **CT201** (containers, images, network, project dir, Serve routes) and **CT333** (container + image)
torn down. Camofox is the sole backend.
## Correction (2026-10-29) — Steel is out of the stack, not just disabled
Verified against live state at **2026-10-29 00:3x AWST**:
Verified against live state at **2026-10-29**. The earlier 00:3x probe said only "not reachable"; the truth is
stronger — Steel is **fully removed**, not merely down:
- **Hermes is back on Camofox.** `config.yaml` (mtime **2026-10-29 00:15:13**) reads
`browser.cloud_provider: camofox`, `browser.camera_provider: camofox`, `engine: auto`, `cdp_url: ""`.
- **The `browser-steel` plugin is gone** — `~/.hermes/plugins/browser-steel/` no longer exists and
`plugins.enabled` carries no steel entry, so the `plugin_steel` toolset cannot be selected. Backup taken
immediately before the change: `config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z` (written
**2026-10-29 00:13:24**). A second backup, `config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z`, records
the 2026-10-28 22:00 attempt.
- **The Steel endpoint is not serving.** `ping 100.96.244.39` (runtipi/CT201) answers **2.5 ms**, but
`http://100.96.244.39:9223/json/version` (CDP) and `http://100.96.244.39:3000/v1/health` (REST) both return
**000** — as does the LAN address `192.168.178.77:9223`. The tunnel route
`https://runtipi.kangaroo-eel.ts.net/steel` returns **502**, and `steel.martinwa.org` no longer resolves.
Earlier pages describing Steel as "still running on CT201" are superseded by this probe.
- ⚠ **Stale environment variable still exported:** `STEEL_BASE_URL=http://100.96.244.39:3000` is present in
**10 live Hermes processes** — the gateway tree (`3371780`, `3372125`, `3372912`, `3844013`, `3880214`,
`3880663`), `agent-browser` 0.26.0 (`3843841`), two chromium crashpad handlers (`3843878`, `3843880`) and
`agentmail-webhook.py` (`402`). It now points at a dead endpoint and clears only when the gateway is
restarted. The config is explicit (`cloud_provider: camofox`), so it does not currently misroute browser
calls — but it is a trap for anything that reads the variable.
- **Hermes is back on Camofox, and Camofox is the sole backend.** The `browser:` block in
`~/.hermes/config.yaml` reads exactly `cloud_provider: camofox`, `camera_provider: camofox`, `engine: auto`,
`cdp_url: ""`.
- **The `browser-steel` plugin is deleted** — `~/.hermes/plugins/browser-steel/` no longer exists, and
`plugins.disabled` / `plugins.entries` carry no `browser-steel` key, so the `plugin_steel` toolset cannot be
selected. Hermes core source contains **zero** Steel references, so there is no Steel fallback path in the
tree. Backups: `config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z`,
`config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z`.
- **CT201 is clean.** The containers `steel-browser-api-1` and `steel-browser-ui-1`, the `steel-network` Docker
network, the images `steel-browser-api:fixed` / `ghcr.io/steel-dev/steel-browser-api:latest` /
`ghcr.io/steel-dev/steel-browser-ui:latest`, and the project dir `/root/steel-browser` were all removed
(~6 GB reclaimed; disk 70G used → 64G used). The Tailscale Serve routes `/steel` and `/steel-api` were
deleted with `tailscale serve --set-path <path> off`; CT201's remaining routes are only `/` → `127.0.0.1:8085`
and `/mgr` → `127.0.0.1:8084`, on **both** `:443` and `:8443`. No listeners remain on 3000, 5173 or 9223.
- **CT333's second, separate Steel copy is gone too.** The orphaned `steel-browser` container (image
`ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30, publishing port 3000) and its image
were removed; no Steel containers, images or listeners remain on CT333.
- **Reversibility:** the compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`, plus
`/root/_archived-steel/steel-cache-archive-20261029.tgz`; the images remain re-pullable from ghcr.io.
- **Camofox is healthy and working** — REST `http://100.118.5.51:9377` (and `http://localhost:9377`);
`/health` returns **200**, `/tabs` returns `{"running":true,"tabs":[]}`.
- **Stale environment variable — now cleared from the gateway.** `STEEL_BASE_URL` is no longer in
`~/.hermes/.env`. It survived in the environment of already-running processes (inherited at spawn), so the
**gateway was restarted on 2026-10-29 01:21 AWST** to clear it: the new gateway tree (PID `3913841`) reads
`STEEL_BASE_URL=0`. A few older, unrelated services still carry the variable in their inherited
environment (the dashboard unit and `agentmail-webhook.service`) — **no core Hermes code reads
`STEEL_BASE_URL`**, so it is inert either way, and it is a runtime-only leftover, **not** a config value.
> ⚠ **Superseded (the 2026-10-29 00:3x probe):** an earlier version of this correction described Steel as
> merely "not reachable — endpoint dark, still running on CT201". That is outdated: Steel is now **removed**,
> not down.
## Related
- [[comparisons/steel-vs-camoufox]]
+19 -8
View File
@@ -3,8 +3,8 @@ title: Homepage Dashboard
type: systems
status: degraded
created: 2026-07-26
updated: 2026-10-25
verified_on: 2026-10-25
updated: 2026-10-29
verified_on: 2026-10-29
confidence: high
tags: [dashboard, homepage, homelab, proxmox, monitoring]
---
@@ -40,8 +40,9 @@ CT201 (runtipi)
├── homebox container (7745) — up
├── asset_proxy.py (python3, port 8085) — proxies 8085 → 8082, currently 502
├── manager.py (python3) — dashboard management service, up
└── Tailscale serve (ports 443 AND 8443) — / → 8085, /mgr → 8084 on both;
/steel → 5173 and /steel-api → 3000 on :443 ONLY
└── Tailscale serve (ports 443 AND 8443) — / → 8085, /mgr → 8084 on both ports
(the former /steel and /steel-api routes were
removed 2026-10-29)
```
No container is listening on **8082** — `docker ps` on CT201 lists no `homepage` container at all.
@@ -128,15 +129,25 @@ no Traefik container or process exists on CT201 (`docker ps -a` and `ps aux` bot
(not in Docker) — restart via whatever supervises them; check `ps aux | grep -E 'manager.py|asset_proxy'`
- **Recover the upstream Homepage (8082):** the image `ghcr.io/gethomepage/homepage` is still present on
CT201 (4 months old) but no container uses it — recreate it to restore 8082 and the 8085 proxy path
- **Tailscale serve on CT201:** `/` → 8085 and `/mgr` → 8084 on **both** `:443` and `:8443`;
`/steel` → 5173 and `/steel-api` → 3000 on **`:443` only** (`/steel` on `:8443` returns 502)
- **Tailscale serve on CT201:** `/` → 8085 and `/mgr` → 8084 on **both** `:443` and `:8443`.
The former `/steel` (5173) and `/steel-api` (3000) routes were **deleted on 2026-10-29** when
Steel was removed from CT201 — there are no other paths. → [[infrastructure/tailscale]]
## Recent Changes
- **2026-10-29 (Steel removal):** the `/steel` (5173) and `/steel-api` (3000) Serve routes on CT201 were
**deleted** with `tailscale serve --set-path <path> off` when the Steel Browser deployment was removed.
CT201's remaining routes are only `/` → 8085 and `/mgr` → 8084 on both `:443` and `:8443`.
→ [[infrastructure/tailscale]], [[comparisons/steel-vs-camoufox]]
- **2026-10-25 (live probe, stocktake):** the "Port 443 occupied by Traefik on CT201" note was **false** — no Traefik
container or process exists on CT201, and `tailscaled` owns **both** `:443` and `:8443` there. Serve tables corrected:
`/` → 8085 and `/mgr` → 8084 on both ports, but `/steel` → 5173 and `/steel-api` → 3000 on **`:443` only** (`/steel`
on `:8443` returns **502**; on `:443` it returns **200**). Re-confirmed live: 8084 → **200**, 8085 → **502**,
`/` → 8085 and `/mgr` → 8084 on both ports *(the `/steel` and `/steel-api` routes listed at the time were
deleted on 2026-10-29 — see above)*. Re-confirmed live: 8084 → **200**, 8085 → **502**,
8082 → **no listener**, `homepage-manager` up. → [[infrastructure/tailscale]]
- **2026-10-20 (live probe):** `homepage` container absent, 8082 dead, 8085 → 502, Homarr on 8102 → 200.
Status changed `active` → `degraded`. VM/LXC table re-read from both PVE hosts; corrected IDs and
host assignments, dropped the non-existent OpenClaw VM 403, marked Mattermost CT255 stopped.
## Related
- [[infrastructure/tailscale]] — Tailscale Serve routes on CT201 and CT460
- [[infrastructure/docker-services]] — container inventory for CT201 and CT333
- [[comparisons/steel-vs-camoufox]] — why the `/steel` routes were removed on 2026-10-29
+1 -1
View File
@@ -14,7 +14,7 @@ sources: [live-cron-list, live-jobs-json, cron-executions-db]
# Scheduled Tasks
## Count
- **Current as of 2026-10-29 00:3x AWST (live `jobs.json` + `executions.db`) — the radar's two-stage split ran for real and the discovery stage still could not finish: stage 1 hit the agent iteration cap and wrote no handoff, so both stage-2 slots correctly no-oped.** Fleet unchanged at **29 jobs (26 enabled / 3 disabled)**; non-clean `last_status` **5 → 5**; unclosed incident rows **2 → 4**. Window **2026-10-28 settled 168 / 2**, and both failures are in-window. `dca8482e4f76` (Job Radar stage 1) ran **07:00:32 → 10:33:18** (**3 h 32 m 46 s**), closed **`failed`**, **`failure_streak: 2`**, on *"hit the agent iteration cap before the handoff could be written"*; `~/.hermes/radar-ref/handoff/` holds only `2026-10-27.json`, and stage 2 (`5fc292ff0f3b`) fired **11:30:27 → 11:31:31** and **14:30:27 → 14:31:43**, both `completed` / **`suppressed`** — the no-op path, confirmed in production → [[incidents/2026-10-28-job-radar-stage1-iteration-cap]]. The other failure is the **03:00 stocktake** (`1a8e90ea2271`): **03:00:28 → 03:37:58** (37 m 30 s), `failed`, **streak 1**, delivery `suppressed`, on `RuntimeError: [400] … duplicate field 'reasoning_content' at line 1 column 512203` — the provider could not deserialize the model's own 512 KB final response. It filed incident `1a8e90_f57ab4cafab8` in state **`detected`** and **never alerted**, and left `systems/skills-index.md` modified and `raw/stocktake/freshness-20261028.txt` untracked; the 2026-10-29 audit committed both. **Steel left the stack in-window:** `browser.cloud_provider`/`camera_provider` are back to **`camofox`** (config mtime 2026-10-29 00:15:13), the `browser-steel` plugin was **removed** (backup `config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z` written 00:13:24), and the endpoint is **not serving** — CDP `:9223`/REST `:3000` refuse, tunnel `/steel` **502**, `steel.martinwa.org` unresolved — while **`STEEL_BASE_URL=http://100.96.244.39:3000` is still exported into 10 live Hermes processes** and clears only on a gateway restart → [[systems/browser-backend]], [[decisions/rollback-steel-to-camofox]]. **2026-10-27 settled 173 / 1** (was 172 / 1 at the 2026-10-28 audit — one late run landed after that count). **Pins: 12 `smart-route` / 6 `free-stack` / 7 unpinned / 1 `deepseek-paid-smart`.**
- **Current as of 2026-10-29 00:3x AWST (live `jobs.json` + `executions.db`) — the radar's two-stage split ran for real and the discovery stage still could not finish: stage 1 hit the agent iteration cap and wrote no handoff, so both stage-2 slots correctly no-oped.** Fleet unchanged at **29 jobs (26 enabled / 3 disabled)**; non-clean `last_status` **5 → 5**; unclosed incident rows **2 → 4**. Window **2026-10-28 settled 168 / 2**, and both failures are in-window. `dca8482e4f76` (Job Radar stage 1) ran **07:00:32 → 10:33:18** (**3 h 32 m 46 s**), closed **`failed`**, **`failure_streak: 2`**, on *"hit the agent iteration cap before the handoff could be written"*; `~/.hermes/radar-ref/handoff/` holds only `2026-10-27.json`, and stage 2 (`5fc292ff0f3b`) fired **11:30:27 → 11:31:31** and **14:30:27 → 14:31:43**, both `completed` / **`suppressed`** — the no-op path, confirmed in production → [[incidents/2026-10-28-job-radar-stage1-iteration-cap]]. The other failure is the **03:00 stocktake** (`1a8e90ea2271`): **03:00:28 → 03:37:58** (37 m 30 s), `failed`, **streak 1**, delivery `suppressed`, on `RuntimeError: [400] … duplicate field 'reasoning_content' at line 1 column 512203` — the provider could not deserialize the model's own 512 KB final response. It filed incident `1a8e90_f57ab4cafab8` in state **`detected`** and **never alerted**, and left `systems/skills-index.md` modified and `raw/stocktake/freshness-20261028.txt` untracked; the 2026-10-29 audit committed both. **Steel left the stack in-window — and was then fully removed:** `browser.cloud_provider`/`camera_provider` are back to **`camofox`** (config mtime 2026-10-29 00:15:13), the `browser-steel` plugin was **deleted** (backup `config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z` written 00:13:24), and by the end of 2026-10-29 the Steel deployments on **CT201** (containers, images, network, project dir, and the Serve routes `/steel` and `/steel-api`) and **CT333** (container + image) were **removed entirely** — nothing listens on 3000, 5173 or 9223, and there is no Steel fallback path left in the tree. `STEEL_BASE_URL` is no longer in `~/.hermes/.env`; it survived only in the inherited environment of processes already running at the cleanup, and the **gateway was restarted on 2026-10-29 01:21 AWST** to clear it → [[systems/browser-backend]], [[decisions/rollback-steel-to-camofox]], [[comparisons/steel-vs-camoufox]]. **2026-10-27 settled 173 / 1** (was 172 / 1 at the 2026-10-28 audit — one late run landed after that count). **Pins: 12 `smart-route` / 6 `free-stack` / 7 unpinned / 1 `deepseek-paid-smart`.**
- **Prior window — as of 2026-10-28 00:2x AWST** (live `jobs.json` + `executions.db`) — the radar's budget overrun stopped being silent, and the radar is now two jobs.** Window 2026-10-27: **172 `completed` / 1 `failed`** (+ this audit `running`). The single failure is `dca8482e4f76` (**Daily Marketing Job Radar - Discovery**), which ran **07:01:08 → 11:03:54** (**4 h 2 m 46 s**), closed `failed`, and delivered a report that opens *"INCOMPLETE (tool-call budget exhausted before Notion write stage)"* — discovery/verify/dedup/classify finished, **Notion ingestion, cover-letter generation and the final audit did not run**. Incident row `dca848_e93a538ade6b` opened **11:03:53**, `alerted` 11:03:59, **still open** (`failure_type: auth`, which does not describe a budget fault) → [[incidents/2026-10-27-job-radar-budget-exhausted]]. **Response, same day:** the radar was split into a **two-stage pipeline** — stage 1 `dca8482e4f76` (Discovery, `0 7 * * 1-5`, `deliver: local`, failures → topic 1030) and new stage 2 **`5fc292ff0f3b`** (Daily Marketing Job Radar - Write and Letters, `30 11,14 * * 1-5`, Telegram topic 1030, created **2026-10-27T17:34:15**; two test fires the same evening, 17:35:00 → 17:36:52 and 17:38:38 → 17:43:42, both `completed`, delivery `suppressed`, `last_status: ok`) → [[runbooks/job-radar]]. **Fleet 28 → 29 jobs (26 enabled / 3 disabled).** **Unclosed incident rows 3 → 2:** the two rows the 2026-10-26 planned restart raised (`ea79a2_a4661554a3c2`, `ea79a2_c3ab2d784ba3`) **auto-closed 00:38:32**, and the radar row opened. **Non-clean `last_status` 4 → 5** (churn): `ea79a2107e36` cleared to `ok`; `dca8482e4f76` joined as `error` (streak 1) and `08eb86f02d37` re-joined as `delivery_queued` — still non-clean: `0e753e9720fd` (error), `fad5a86979d4` (delivery_failed), `066bb5191280` (delivery_failed). **Pins unchanged.** The 2026-10-26 orphan class **did not recur** — every 10-27 run except the radar's completed. *(superseded by the 2026-10-29 entry above)*
- **Prior window — as of 2026-10-27 00:1x AWST *(superseded by the 2026-10-28 entry above)* (live `jobs.json` + `executions.db`) — a *planned* restart interrupted a live job, and the fleet got a loud failure instead of a silent one.** Window 2026-10-26 settled **169 `completed` / 3 `failed`**, and **all three failures are a single fault**: `ea79a2107e36` (Mnemosyne Health Monitor, 12:01 slot) plus two derivative Mnemosyne Watchdog reds → [[incidents/2026-10-26-planned-restart-orphaned-cron-run]]. 2026-10-25 settled **172 / 2**. **Fleet unchanged: 28 jobs — 25 enabled / 3 disabled** (`d40085da631c` Important-mail monitor, `f77b7e46bd09` Rhino AgentMail Inbox Watcher, `0a7f933d06e2` AgentMail Rhino Inbox Poll). **Non-clean `last_status` 5 → 4:** `c0a9fa314542` (GitHub Trending Radar) and `08eb86f02d37` (Gmail Action Digest) both cleared to `ok`, while `ea79a2107e36` joins as **`error`, streak 2**; `0e753e9720fd` (`error`) and `fad5a86979d4` + `066bb5191280` (`delivery_failed`) unchanged. **Unclosed incident rows 2 → 3:** both `ea79a2_*` rows — 14:33:37 *"Cron execution unverified; reconcile `admissions/724a69108b…json`"* and 14:35:59 *"lost its durable fire claim ownership"* — are still `alerted`, alongside `0e753e_e7f8194a7d06` (`detected` since 2026-10-01). **One row closed:** `c0a9fa_764558b5ae83` auto-closed **09:13:00** when the radar's 09:00 slot ran clean — **every row of the 2026-10-16 503 resource-pressure incident is now closed** → [[incidents/2026-10-16-omniroute-503-resource-pressure]]. **Pins unchanged:** 11 `custom:omni-101` / `smart-route`, 6 `custom:omni-101` / `free-stack`, 2 `custom:omni` / `smart-route`, 1 `custom:omni-101` / `deepseek-paid-smart`, 8 unpinned (global default still `custom:omni-101` / `smart-route`). **One gateway start on 2026-10-26** (14:35:18, after a planned `SIGTERM` at 14:33:35; teardown 54.06 s, `.clean_shutdown` marker **skipped**) → [[systems/hermes-agent]]. ⚠ **`fad5a86979d4`'s 03:08 slot completed but delivered `delivery_failed`** — *Thread 34795 not found*, so the digest fell back to the unthreaded chat; the gateway's own shutdown notice hit the same dead thread at 14:33:37. ⚠ **The radar's 07:00 slot closed `completed` at 09:10 but delivered “RUN INCOMPLETE — tool-call budget exhausted”** — no Notion writes, no cover letters — and the same morning its cover-letter PDF pipeline moved to Chromium-over-canonical-HTML with model `deepseek-paid-smart` → [[runbooks/job-radar]]. ⚠ **Off-cron, unchanged: Nous Portal credentials still invalid** → [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]. **Sessions in window: 17** (15 `cron`, 1 `ios`, 1 `telegram`).
- **Current as of 2026-10-26 00:1x AWST** *(superseded by the 2026-10-27 entry above)* **(live `jobs.json` + `executions.db`)** — **fleet is clean, but one job misbehaved badly and the global model default moved back.** Window 2026-10-25 settled **172 `completed` / 2 `failed`**; **both failures are `08eb86f02d37` (Gmail Action Digest)**, which also fired **six times that day** → [[incidents/2026-10-25-gmail-digest-runaway-and-refire]]. 2026-10-24 settled **168 / 0** (the 2026-10-25 entry read 167 — its day was still open). **Fleet unchanged: 28 jobs — 25 enabled / 3 disabled**; all 28 IDs still on this page. **Unclosed incident rows still 2** (`0e753e_e7f8194a7d06` `detected` since 2026-10-01; `c0a9fa_764558b5ae83` `alerted` since 2026-10-19) — **nothing opened or closed in-window, so neither 10-25 failure raised an incident row.** Non-clean `last_status` **4 → 5**: `08eb86f02d37` joins as **`delivery_queued`** with an **empty** `last_error` (never resolved to `ok`), alongside `0e753e9720fd` (`error`), `fad5a86979d4` + `066bb5191280` (`delivery_failed`) and `c0a9fa314542` (`error`, streak 1). **Pins unchanged:** 11 `custom:omni-101` / `smart-route`, 6 `custom:omni-101` / `free-stack`, 2 `custom:omni` / `smart-route`, 1 `deepseek-paid-smart`, 8 unpinned. ⚠ **Global default reverted to OmniRoute on 2026-10-24 19:06** (`custom:omni-101` / `smart-route`), so the 8 unpinned jobs are back on the same upstream as the pinned ones — **the fleet is a single path again** → [[systems/current-profile]]. ⚠ **Three gateway starts on 2026-10-25** (00:32:20, 18:30:04, 19:27:13; gateway pid 2476414 up since 19:26:33) — the five extra digest fires bracket the evening pair → [[systems/hermes-agent]]. ⚠ **Nous Portal credential still invalid** (`Invalid refresh token`) — context summaries unavailable on 10-25 (20 log lines); all 8 `auxiliary` slots were moved to `no-think/freeapi/auto` on `custom:omni-101` at 19:13 → [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]. **Sessions in window: 21** (18 `cron` — 15 `smart-route`, 2 `free-stack`, 1 `deepseek-paid-smart`; 1 `desktop`; 1 `oneshot`; 1 `telegram`).
+391 -477
View File
@@ -12,10 +12,7 @@ sources: []
# Installed Skills
**Total installed: 302 skills** in `~/.hermes/skills/`.
- **45 archived skills** are parked under `~/.hermes/skills/.archive/` and are excluded from this count — the loader does not expose them.
- The loader's own catalogue (`hermes skills list`, e.g. *34 hub-installed, 52 builtin, N local — N enabled*) is the authoritative loadable figure; this page is a filesystem inventory.
**Total installed: 348 skills** in `~/.hermes/skills/`.
This page is auto-regenerated from the live skill inventory. Do not hand-edit.
@@ -23,515 +20,432 @@ This page is auto-regenerated from the live skill inventory. Do not hand-edit.
- Category = directory structure under `~/.hermes/skills/` (`category/skill/SKILL.md`).
- Skills directly under `~/.hermes/skills/` group under `uncategorized`.
## Apple (4)
- `apple-notes` — # Apple Notes
- `apple-reminders` — # Apple Reminders
- `findmy` — # Find My (Apple)
- `imessage` — # iMessage
## Autonomous Ai Agents (8)
- `antigravity-cli` — # Antigravity CLI (`agy`)
- `claude-code` — # Claude Code — Hermes Orchestration Guide
- `codex` — # Codex CLI
- `computer-use` — # Computer Use (universal, any-model, cross-platform)
- `hermes-agent` — # Hermes Agent
- `merge-reconciler` — # Merge Reconciler
- `opencode` — # OpenCode CLI
- `openhands` — # OpenHands CLI
## Chromadb (1)
- `chromadb` — # ChromaDB — Vector Database Operations for the Homelab
## Chromadb Preflight (1)
- `chromadb-preflight` — # ChromaDB Preflight — Auto-Context Integration
## Chromadb Skills Rag (1)
- `chromadb-skills-rag` — # ChromaDB Skills RAG — Full Integration Layer
## Clean Architecture (1)
- `clean-architecture` — # Clean Architecture Framework
## Clean Code (1)
- `clean-code` — # Clean Code Framework
## Cloudflare Tunnel Ops (1)
- `cloudflare-tunnel-ops` — # Cloudflare Tunnel Operations 🦝🌐
## Creative (18)
- `architecture-diagram` — # Architecture Diagram Skill
- `ascii-art` — # ASCII Art Skill
- `ascii-video` — # ASCII Video Production Pipeline
- `baoyu-infographic` — # Infographic Generator
- `claude-design` — # Claude Design for CLI/API Agents
- `comfyui` — # ComfyUI
- `creative-ideation` — # Creative Ideation
- `design-md` — # DESIGN.md Skill
- `excalidraw` — # Excalidraw Diagram Skill
- `humanizer` — # Humanizer: Remove AI Writing Patterns
- `hyperframes` — # HyperFrames
- `manim-video` — # Manim Video Production Pipeline
- `p5js` — # p5.js Production Pipeline
- `popular-web-designs` — # Popular Web Designs
- `pretext` — # Pretext Creative Demos
- `sketch` — # Sketch
- `songwriting-and-ai-music` — # Songwriting & AI Music Generation
- `touchdesigner-mcp` — # TouchDesigner Integration (twozero MCP)
## Cron (1)
- `cron-digest-formatting` — # Cron Digest Formatting (Telegram)
## Cron Model Repair (1)
- `cron-model-repair` — # Cron Model-Pin Repair
## Data Science (1)
- `jupyter-live-kernel` — # Jupyter Live Kernel (hamelnb)
## Ddia Systems (1)
- `ddia-systems` — # Designing Data-Intensive Applications Framework
## Devops (108)
- `abyss-ios-pairing` — # Pair iPhone with The Abyss (iOS)
- `agent-extension-audit` — # Agent Extension Audit
- `agent-observability` — # Agent Observability
- `agent-stack-weekly-review` — # Agent Stack Weekly Review
- `browser-automation-recovery` — # Browser Automation Recovery
- `browser-backend-verification` — # Browser Backend Verification
- `browser-tab-lifecycle-and-recovery` — # Browser Tab Lifecycle & Recovery
- `camofox-browser-automation` — # Camofox Browser Automation
- `camofox-pitfalls` — # Camofox Pitfalls
- `camoufox-remote-websocket` — # Camoufox Remote WebSocket Access
- `chat4000-ops` — # chat4000 Operations & Pairing
- `cloudflare-ops` — # Cloudflare Operations
- `codex-via-cliproxy` — # Codex CLI via Cliproxy (mitmproxy redirect)
- `crawlee-batch-scrape` — ## Installation
- `cron-job-maintenance` — ## Mnemosyne Watchdog Cron Job — Conflict Alert Thresholds (2026-07-25)
- `cross-profile-session-handoff` — # Cross-Profile Session Handoff
- `cua-computer-use` — # CUA Computer Use Automation
- `cua-driver-lifecycle` — # cua-driver Lifecycle Management
- `dashboard-server-ops` — # Dashboard Server Operations
- `devops` — (no description)
- `docker-management` — # Docker Management
- `external-auth-script-integration` — # External Auth Script Integration
- `external-skill-integration` — # External Skill Integration
- `fleet-wide-failure-triage` — # Fleet-Wide Failure Triage
- `free-web-tools` — # Free Web Search & Extract
- `freshrss-intelligence-filter` — # FreshRSS Intelligence Filter
- `gcp-vertex-ai-provider` — # GCP Vertex AI Provider
- `gmail-imap-email-monitor` — # Gmail IMAP Email Monitor (Cron)
- `google-apis` — # Google APIs Tool Suite
- `google-contacts-birthday-sync` — > **See also:**
- `hermes-agent-install` — # Hermes Agent Install & Upgrade
- `hermes-agent-ops` — ### Dashboard Auth Plugin — `basic` Must Be Enabled
- `hermes-api-vault` — # Hermes API Vault
- `hermes-config-diagnostics` — # Hermes Config Diagnostics
- `hermes-cron-operations` — # Hermes Cron Operations
- `hermes-cron-patterns` — ## Reference Files
- `hermes-gateway-config-loading` — # Hermes Gateway Config Loading Pipeline
- `hermes-gateway-ops` — # Hermes Gateway Operations & Platform Debugging
- `hermes-instance-migration` — # Hermes Instance Migration (audit + curated port)
- `hermes-instance-update` — # Hermes Instance Update Operations
- `hermes-mattermost-integration` — # Hermes Mattermost Integration
- `hermes-memory-systems` — # Hermes Agent Memory Systems
- `hermes-multi-agent` — # Hermes Multi-Agent Architecture
- `hermes-plugin-live-verification` — # Hermes Plugin Live Verification
- `hermes-plugin-ops` — ### Session Learnings (2026-07-07)
- `hermes-profile-ops` — # Hermes Profile Operations
- `hermes-provider-configuration` — # Hermes Provider Configuration
- `hermes-routing-and-domains` — # Hermes Routing & Domains
- `hermes-self-diagnosis` — # Hermes Self-Diagnosis
- `hermes-soul-forge` — # Soul Forge Plugin Operations
- `hermes-web-tools` — # Hermes Web Tools (this instance)
- `hermes-webui-ops` — # Hermes WebUI Operations
- `hindsight-docs` — # Hindsight Documentation Skill
- `hindsight-import-mnemosyne` — # Import Mnemosyne → Hindsight
- `hindsight-ops` — # Hindsight Operations & Diagnostics
- `hindsight-selfhost` — # Self-Host Hindsight (bare-metal) + Official Control Plane
- `homarr-dashboard-management` — # Homarr Dashboard Management (v1 on runtipi)
- `homarr-dashboard-ops` — # Homarr Dashboard Operations
- `homelab-docker-update` — # Homelab Docker Update
- `homelab-instance-migration` — # Homelab Instance Migration
- `homelab-service-management` — # Homelab Service Management
- `homelab-virtual-desktop` — # Homelab Virtual Desktop (for computer-use)
- `ignis-ops` — # Ignis Deployment & Operations
- `job-board-scraping-reference` — # Job Board Scraping Reference
- `kanban-orchestrator` — # Kanban Orchestrator — Decomposition Playbook
- `kanban-worker` — # Kanban Worker — Pitfalls and Examples
- `linux-systemd-desktop` — # Linux headless desktop as a systemd user service
- `litellm-openrouter-key-pooling` — # LiteLLM OpenRouter Key Pooling
- `litellm-proxy-ops` — # LiteLLM Proxy Operations
- `llm-model-diagnostics` — # LLM Model Diagnostics
- `mattermost-ops` — # Mattermost Operations 🦝📡
- `mattermost-platform-ops` — # Mattermost Platform Operations
- `mcp-agent-integration` — # MCP Agent Integration
- `memory-governance` — # Memory Governance
- `mnemosyne-operations` — # Mnemosyne Operations & Diagnostics
- `multi-agent-git-ops` — # Multi-Agent Git Operations
- `nanobot-omniroute-skills` — # Nanobot ↔ OmniRoute Skills Integration
- `nanobot-ops` — > **Session detail / full diagnostic transcript:** see `references/nanobot-webui-auth-and-404-recovery.md` (covers the v0.2.x 404 root cause, the `tokenIssueSecret`-vs-`token` password trap, port m...
- `nextcloud-ops` — # Nextcloud Ops (CT270)
- `obsidian-ops` — \n# Obsidian Vault Operations\n\nAnthony's Obsidian vault at `/home/hermes/obsidian-vault`, git-synced to Gitea via `obsidian-git` plugin (auto-sync every 5 min).\n\n## Vault Sync Scripts\n\nTwo au...
- `omniroute-combo-troubleshooting` — # OmniRoute Combo Troubleshooting
- `omniroute-context-tuning` — # OmniRoute Context Window Tuning
- `omniroute-mcp-portal-cloudflare-auth` — # Cloudflare Access Auth for mcp-portal.martinwa.org/mcp
- `omniroute-ops` — # OmniRoute Operations & Diagnostics
- `omniroute-skill-caller` — # Call an OmniRoute Skill by Name (auto-resolving wrapper)
- `omniroute-skill-execution` — # Execute Skills on OmniRoute MCP
- `openclaw-ops` — # Open-Claw Operations & Configuration
- `pve-ops` — # PVE Operations
- `radar-jobs-dashboard` — # RADAR Jobs Dashboard
- `readlater-ops` — # ReadLater Operations
- `runtipi-container-management` — # Runtipi Container Management
- `sdlc-review` — # SDLC Review Skill
- `self-hosted-app-deploy` — # Self-Hosted App Deployment
- `solutions-first-agent-conduct` — # Solutions-First Agent Conduct
- `tailscale-ops` — # Tailscale Operations 🦝🔗
- `tailscale-serve-route-management` — # Tailscale Serve Route Management
- `tailscale-serve-routes` — # Tailscale Serve Route Management
- `task-approach` — # Task Approach
- `telegram-create-forum-topic` — # Telegram Create Forum Topic Skill
- `tmux-config` — # tmux-config
- `tool-ecosystem-evaluation` — # Tool Ecosystem Evaluation
- `uber-eats-monitor` — # Uber Eats Order Monitor
- `vision-model-validation` — # Vision Model Validation via OmniRoute
- `watchers` — # Watchers
- `web-content-extraction` — # Web Content Extraction — Fallback Techniques
- `weekly-comms-tracker` — # Weekly Communications Tracker
- `wiki-maintenance` — # Wiki Maintenance
- `zeroclaw-ops` — # ZeroClaw Operations & Configuration
## Devops Homelab Architecture Live Probe (1)
- `devops-homelab-architecture-live-probe` — # Homelab Architecture Live Probe
## Devops Proxmox Operations (1)
- `devops-proxmox-operations` — # Proxmox Operations
## Devops Service Management (1)
- `devops-service-management` — # Service Operations Umbrella
## Devops Tailscale Web Routing (1)
- `devops-tailscale-web-routing` — # Tailscale Web Routing
## Devops Telegram Bot Api Self Hosted (1)
- `devops-telegram-bot-api-self-hosted` — # Self-Hosted Telegram Bot API Server
## Domain Driven Design (1)
- `domain-driven-design` — # Domain-Driven Design Framework
## Email (17)
- `agent-email-patterns` — # Agent Email Patterns
- `agentmail` — # AgentMail — Agent-Owned Email Inboxes
- `agentmail-check-email` — # Check Email
- `agentmail-cli` — # AgentMail CLI
- `agentmail-manage-inboxes` — # Manage Inboxes
- `agentmail-mcp` — # AgentMail MCP
- `agentmail-rest-fallback` — # AgentMail REST API Fallback
- `agentmail-sdk` — # AgentMail SDK
- `agentmail-send-email` — # Send Email
- `agentmail-toolkit` — # AgentMail Toolkit
- `cron-email-briefing` — # Cron Email Briefing Workflow
- `email-for-ai-agents` — # Agent Email Patterns
- `email-inbox-triage` — # Email Inbox Triage
- `gmail-readonly-fetch` — # Gmail Read-Only Fetch (Large Inbox)
- `himalaya` — # Himalaya Email CLI
- `mail-lookup` — # Ad-Hoc Mail Lookup
- `realtime-email-monitor` — # Real-time Email Monitor (IMAP IDLE)
## Email Himalaya (1)
- `email-himalaya` — # Himalaya Email CLI
## Finance (2)
- `3-statement-model` — ## Environment
- `australian-tax-return-review` — # Australian Tax Return Review
## Frontend (4)
- `design-alignment` — # Design Alignment
- `impeccable` — Designs and iterates production-grade frontend interfaces. Real working code, committed design choices, exceptional craft.
- `landing-dashboard-reference` — # Landing Page & Dashboard Design Reference
- `pixel-perfect-iteration` — # Pixel-Perfect Visual Iteration
## Github (6)
- `github-auth` — # GitHub Authentication Setup
- `github-code-review` — # GitHub Code Review
- `github-issue-to-pr` — # GitHub Issue to Pull Request
- `github-issues` — # GitHub Issues Management
- `github-pr-workflow` — # GitHub Pull Request Workflow
- `github-repo-management` — # GitHub Repository Management
## Groktocrawl Deployment (1)
- `groktocrawl-deployment` — # GroktoCrawl Self-Hosted Deployment
- `antigravity-cli` — Operate the Antigravity CLI (agy): plugins, auth, sandbox.
- `claude-code` — Delegate coding to Claude Code CLI (features, PRs).
- `codex` — Delegate coding to OpenAI Codex CLI (features, PRs).
- `computer-use` — Drive the desktop background-first; escalate on signal.
- `hermes-agent` — Use, configure, theme, extend, and orchestrate Hermes Agent.
- `merge-reconciler` — Neutral third-party resolution of agent merge conflicts.
- `opencode` — Delegate coding to OpenCode CLI (features, PR review).
- `openhands` — Delegate coding to OpenHands CLI (model-agnostic, LiteLLM).
## Hermes (3)
- `hermes-personality-soulmd` — # Hermes Personality & SOUL.md
- `hermes-telegram-miniapp` — # Hermes Telegram Mini App — Deploy & Operate
- `hermes-tts-configuration` — # Hermes TTS & Voice Configuration
- `hermes-personality-soulmd` — Edit and verify Hermes Agent's core personality/identity file (SOUL.md). Use when the user wants to change 'who Hermes i
- `hermes-telegram-miniapp` — Deploy and operate the standalone Hermes Telegram Mini App repo onto a Hermes Agent LXC/VM. Covers the full CT460 workfl
- `hermes-tts-configuration` — Set up Hermes TTS: providers, voice, persona, config edits.
## Hermes Api Vault (1)
## Devops (107)
- `hermes-api-vault` — # Hermes API Vault
- `abyss-ios-pairing` — Pair an iPhone with this Hermes instance via The Abyss iOS app — Tailscale Funnel/Serve + API key + QR. Detects host OS
- `agent-extension-audit` — >
- `agent-observability` — Observability and evaluation for Hermes agent tasks: event logging, memory attribution, repair-memory tracking, failure
- `agent-stack-weekly-review` — Weekly health review for the Hermes+Mnemosyne agent stack: read-only data collection from governance, observability, Mne
- `browser-automation-recovery` — >
- `browser-backend-verification` — Verify active browser backend and CDP connectivity.
- `browser-tab-lifecycle-and-recovery` — >-
- `camofox-browser-automation` — Operate the Camofox headless browser service for authenticated scraping, periodic data extraction, and browser automatio
- `camofox-pitfalls` — Runtime pitfalls and edible fixes for the Camofox browser REST API on Anthony's homelab. Complements camofox-browser-aut
- `camoufox-remote-websocket` — Expose Camoufox as a remote Playwright WebSocket server for cross-host browser automation. Use when another agent needs
- `chat4000-ops` — Operate, diagnose, and pair the chat4000 Hermes plugin — the native iPhone/Mac app for Hermes. Covers CLI commands, devi
- `cloudflare-ops` — Operate Cloudflare services — DNS, Tunnels, R2 Object Storage (S3-compatible), API authentication, and domain management
- `codex-via-cliproxy` — Run the OpenAI Codex CLI (codex) on a machine where chatgpt.com/backend-api is geo-blocked (HTTP 451) or the stored OAut
- `crawlee-batch-scrape` — Lightweight batch URL scraping with Crawlee. Extracts markdown/text from multiple URLs in parallel. Low resource footpri
- `cron-job-maintenance` — Debug, fix, and harden Hermes cron jobs — particularly multi-step autonomous jobs where models skip or incompletely exec
- `cross-profile-session-handoff` — Continue work from a dead or stuck Hermes specialist profile in the default/supervisor profile. Covers session dump insp
- `cua-computer-use` — Set up and operate cua-driver for GUI automation on headless Linux servers. Covers virtual display setup (Xtigervnc/Xvfb
- `cua-driver-lifecycle` — Manage cua-driver processes on Hermes VMs — diagnose runaway instances, implement idle-timeout wrappers, and configure H
- `dashboard-server-ops` — Operate, extend, and troubleshoot self-hosted Python HTTP API servers serving SPAs. Covers http.server-based dashboards,
- `docker-management` — Manage Docker containers, images, volumes, and Compose.
- `external-auth-script-integration` — Fix broken script authentication by redirecting to local helper scripts, preserving module APIs during rewrites, and ver
- `external-skill-integration` — Bring external skills, standalone CLI tools, and MCP server binaries from GitHub repos into Hermes — for Hermes skills u
- `fleet-wide-failure-triage` — Triage when many components fail at once from one cause.
- `free-web-tools` — >-
- `freshrss-intelligence-filter` — Intelligence-filter upgrade for FreshRSS midday cron: clusters articles by topic, flags urgent items, adds 'why this mat
- `gcp-vertex-ai-provider` — Configure Google Cloud Vertex AI as an inference provider in the homelab – save service account credentials, test connec
- `gmail-imap-email-monitor` — Cron-friendly Gmail IMAP pattern for Hermes Agent: find new messages by UID range, parse headers robustly, detect urgenc
- `google-apis` — Google Gmail, Calendar, Drive, Docs, Sheets tools.
- `google-contacts-birthday-sync` — Sync Facebook birthdays to contacts services (Google Contacts, iCloud). Cross-references Facebook birthday data with Goo
- `hermes-agent-install` — Install, reinstall, upgrade, and diagnose Hermes Agent installations. Covers the official installer (scripts/install.sh)
- `hermes-agent-ops` — Diagnose and fix problems with Hermes Agent's OWN runtime — the web UI (port 8787), the memory-ui dashboard plugin ('Hin
- `hermes-api-vault` — Manage Hermes plugin/agent API credentials stored in a Notion vault — query the vault, map entries to ~/.hermes/.env var
- `hermes-config-diagnostics` — >-
- `hermes-cron-operations` — Operate, diagnose, and fix Hermes Agent scheduled cron jobs — list jobs, inspect configs, debug failures, manage scripts
- `hermes-cron-patterns` — Reliable cron job patterns for Hermes Agent. Covers prompt hardening, step enforcement, verification loops, subagent del
- `hermes-gateway-config-loading` — >-
- `hermes-gateway-ops` — Operate, diagnose, and troubleshoot the Hermes Agent messaging gateway — platform connections, config management, servic
- `hermes-instance-migration` — Audit one Hermes Agent instance's skills/state and port a curated subset (skills, scripts, data, cookies) to another ins
- `hermes-instance-update` — Update and verify Hermes Agent instances across the estate.
- `hermes-mattermost-integration` — Configure/troubleshoot Mattermost adapter for Hermes.
- `hermes-memory-systems` — Evaluate, configure, migrate, and troubleshoot Hermes Agent memory providers. Covers all 9 official providers (Hindsight
- `hermes-multi-agent` — Design, configure, and troubleshoot multi-agent Hermes setups using profiles, delegate_task, Kanban, and Telegram bot se
- `hermes-plugin-live-verification` — Verify Hermes plugin enablement, hook registration, live event emission, dual-write consistency, and fallback evidence p
- `hermes-plugin-ops` — Install, configure, optimize, and troubleshoot Hermes Agent plugins — covers plugin installation via CLI, provider/routi
- `hermes-profile-ops` — Audit, diagnose, and optimize Hermes agent profiles. Covers systematic inspection of config.yaml, profile.yaml, SOUL.md,
- `hermes-provider-configuration` — Configure Hermes Agent inference providers and custom OpenAI-compatible endpoints. Use when the user asks to add/switch
- `hermes-routing-and-domains` — Route requests correctly across Hermes CT460's multi-service architecture. Covers Tailscale Serve, Cloudflare Tunnel, do
- `hermes-self-diagnosis` — >-
- `hermes-soul-forge` — Operate the Soul Forge plugin — manage community souls, import profile SOUL.md files from repos, verify profile sync, an
- `hermes-web-tools` — Use the web search + content-extraction capability already installed on this Hermes instance. Covers the built-in web_ex
- `hermes-webui-ops` — Operate, restart, and verify the standalone Hermes WebUI on port 8787 (NOT the official control UI — that's on 9119). Co
- `hindsight-docs` — Complete Hindsight documentation for AI agents. Use this to learn about Hindsight architecture, APIs, configuration, and
- `hindsight-import-mnemosyne` — Import memories from a Mnemosyne SQLite DB into self-hosted Hindsight (:8888). 6,658 chunks (episodic+gists+working+fact
- `hindsight-ops` — Hindsight memory system — self-hosted LIVE on :8888 (systemd hindsight.service; OmniRoute free-IA LLM + Google/Gemini em
- `hindsight-selfhost` — Self-host Hindsight bare-metal (no Docker) + official Control Plane dashboard. hindsight-all-slim + embedded pg0 + hybri
- `homarr-dashboard-management` — Manage Homarr v1 dashboard on runtipi — multi-board setup, permissions, schema validation, PVE integration, and master b
- `homarr-dashboard-ops` — >-
- `homelab-docker-update` — Audit, update, and maintain Docker containers running inside homelab LXCs/VMs — across runtipi, bare Docker hosts, and a
- `homelab-instance-migration` — Audit and migrate a Hermes Agent instance from a source (old/bloated) box to a target (lean) box. Covers skill auditing,
- `homelab-service-management` — Deploy, update, and troubleshoot services running as Docker containers or bare-metal on Anthony's homelab Tailscale node
- `homelab-virtual-desktop` — Build, persist, expose, and debug a viewable virtual desktop (TigerVNC + websockify + Tailscale serve + systemd) on the
- `ignis-ops` — Deploy and operate Ignis browser-native Obsidian on CT460.
- `job-board-scraping-reference` — Platform-specific scraping behaviors for Australian job boards (SEEK, Indeed, LinkedIn) including access patterns, salar
- `kanban-orchestrator` — Decomposition playbook + anti-temptation rules for an orchestrator profile routing work through Kanban. The "don't do th
- `kanban-worker` — Pitfalls, examples, and edge cases for Hermes Kanban workers. The lifecycle itself is auto-injected into every worker's
- `linux-systemd-desktop` — |
- `litellm-openrouter-key-pooling` — Add additional OpenRouter API keys to a LiteLLM proxy setup as a pool, without replacing the existing key. Use when the
- `litellm-proxy-ops` — Manage a running LiteLLM proxy instance via its REST admin API — add/remove models, register new providers, update proxy
- `llm-model-diagnostics` — Diagnose LLM model quality issues: why a model feels 'dumb', responses get truncated, reasoning models return empty cont
- `mattermost-ops` — Operate, diagnose, and maintain a self-hosted Mattermost server (native or LXC). Covers plugin management, Agents plugin
- `mattermost-platform-ops` — Debug Hermes Mattermost adapter: typing, delivery, auth.
- `mcp-agent-integration` — Install and wire local stdio MCP servers (e.g. SearXNG AI Kit) into the user's coding agents — Claude Code, Goose, OpenC
- `memory-governance` — Govern durable memory writes for Hermes + Mnemosyne: audit admissions, lifecycle metadata, duplicate/contradiction check
- `mnemosyne-operations` — Operate, diagnose, and maintain Hermes Agent's native Mnemosyne memory system. Covers health checks, dream cycle consoli
- `multi-agent-git-ops` — Git push, rebase, and merge workflows for repos where multiple agents (Hermes, Nanobot, coding agents) commit. Covers Gi
- `nanobot-omniroute-skills` — Give Nanobot (CT333) access to OmniRoute's skills + tools via MCP. Approach A = add OmniRoute's streamable-HTTP MCP serv
- `nanobot-ops` — Operate, diagnose, and recover the Nanobot gateway service on Proxmox CT333 (clawtest). IP: 192.168.178.66. Access via P
- `nextcloud-ops` — Operate and recover Anthony's self-hosted Nextcloud (CT270, NextCloudPi). Covers the 503/outage diagnosis (maintenance m
- `obsidian-ops` — Operate, diagnose, and troubleshoot Anthony's Obsidian vault. For basic note read/write/search and evidence-based resear
- `omniroute-combo-troubleshooting` — Diagnose and fix OmniRoute model combos that disappear from the /v1/models catalog (and thus from the Hermes /model pick
- `omniroute-context-tuning` — >
- `omniroute-mcp-portal-cloudflare-auth` — Reference for authenticating Hermes to mcp-portal.martinwa.org/mcp via Cloudflare Access headers, plus Nando's config lo
- `omniroute-ops` — Operate, diagnose, and configure OmniRoute — Anthony's AI routing proxy. Covers database access, alias/routing diagnosti
- `omniroute-skill-caller` — Call any skill installed on the OmniRoute MCP server by NAME (e.g. "mattermost", "nanobot", "hermes-agent-framework") wi
- `omniroute-skill-execution` — Execute skills on an OmniRoute MCP server (the composure_statistic_module / omniroute endpoint). Covers the streamable-H
- `openclaw-ops` — Operate, diagnose, and configure the Open-Claw VM — Anthony's autonomous AI agent gateway node (QEMU VM 403). Covers con
- `pve-ops` — Proxmox VE operations via REST API and SSH. Covers guest management, disk reports, diagnostics, and credential-based API
- `radar-jobs-dashboard` — Use when operating or extending the RADAR jobs dashboard.
- `readlater-ops` — Operate, deploy, and fix Anthony's self-hosted ReadLater app (Tony0410/readlater) — Next.js + SQLite in Docker on debian
- `runtipi-container-management` — Update, clean up, and troubleshoot Docker containers inside a Runtipi LXC running on a Proxmox VE host. Covers inventory
- `sdlc-review` — Review Kanban handoffs and route verified outcomes.
- `self-hosted-app-deploy` — Deploy, update, and troubleshoot self-hosted Docker applications on Tailscale-connected homelab VMs. Covers the full lif
- `solutions-first-agent-conduct` — How this agent must behave when a task hits a wall or a tool fails — lead with the attempt and the fix, not the limitati
- `tailscale-ops` — Operate and manage Tailscale configuration — Serve routes, Funnel, DNS, ACLs, and debugging. Covers adding/removing serv
- `tailscale-serve-route-management` — Manage Tailscale Serve routes on CT460 — the `--set-path` replaces-all-rules pitfall, the full route inventory, and the
- `tailscale-serve-routes` — Manage Tailscale Serve routes on CT460 atomically. Use when routes need adding, fixing, or verifying.
- `task-approach` — How the agent should approach any task given by Anthony — pre-work, skill loading, quality gates, and delivery standards
- `telegram-create-forum-topic` — Create a Telegram forum topic (thread) in a group/supergroup that has topics enabled. Wraps the Telegram Bot API's creat
- `tmux-config` — Configure and troubleshoot tmux — fix scroll/scrollback (mouse mode, history-limit, wheel bindings) and live-apply confi
- `tool-ecosystem-evaluation` — Evaluate external tools, plugins, skills, and integrations for Hermes Agent and Nanobot. Use when auditing free add-ons,
- `uber-eats-monitor` — Monitor an Uber Eats order and announce via Sonos when delivery is imminent. Supports timer-based tracking (confidence)
- `vision-model-validation` — Validate and select free vision-capable models via OmniRoute for Hermes auxiliary.vision tool.
- `watchers` — Poll RSS, JSON APIs, and GitHub with watermark dedup.
- `web-content-extraction` — >-
- `weekly-comms-tracker` — Track all weekly communications sent to Anthony to prevent repetition across weeks. SQLite-backed with content hashing f
- `wiki-maintenance` — Maintain the systems wiki for staleness and drift.
- `zeroclaw-ops` — Operate, diagnose, and configure ZeroClaw — Anthony's autonomous AI daemon on Proxmox CT333. Covers config editing (TOML
## Hermes Desktop Plugins (1)
## Productivity (26)
- `hermes-desktop-plugins` — # Hermes Desktop Plugins Skill
- `airtable` — Airtable REST API via curl. Records CRUD, filters, upserts.
- `box` — Box manages cloud files, sharing, search, and metadata.
- `custom-pet-authoring` — Create custom animated pet spritesheets for Hermes' petdex system. Covers the pet format spec, Pillow-based spritesheet
- `document-to-action-items` — Extract cited obligations, deadlines, tasks from documents.
- `docx` — Create, read, edit, template, and review Word .docx files.
- `google-workspace` — Gmail, Calendar, Drive, Docs, Sheets via gws CLI or Python.
- `maps` — Geocode, POIs, routes, timezones via OpenStreetMap/OSRM.
- `meeting-action-items` — Turn meeting notes into cited decisions, owners, tickets.
- `nano-pdf` — Edit text in existing PDFs via natural-language prompts.
- `notion` — Notion API + ntn CLI: pages, databases, markdown, Workers.
- `notion-database-dedup` — Clean duplicates from Notion databases with quality scoring. Fetch all pages with pagination, compare by URL/company+rol
- `obsidian-git-sync` — >
- `obsidian-git-sync` — >
- `ocr-and-documents` — Extract text from PDFs/scans (pymupdf, marker-pdf).
- `pdf` — PDF files: create, read, merge, fill, OCR, edit text.
- `petdex` — Install and select animated petdex mascots for Hermes.
- `petdex-custom` — Create custom Hermes petdex pets by composing AI-generated or imported character art into spec-correct spritesheets. Com
- `powerpoint` — Create, read, edit .pptx decks with python-pptx.
- `product-price-monitor` — Watch product, flight, or listing prices; alert on target.
- `session-librarian` — Organize sessions by prompt: find, rename, archive, prune.
- `stratos-command-centre` — Run, verify, extend, and diagnose the Stratos command-centre dashboard and its backend.
- `teams-meeting-pipeline` — Teams meeting summaries, job replay, Graph subscriptions.
- `telegram-cron-table-formatting` — How to deliver tables/structured data to Telegram from a Hermes cron job (deliver='telegram') or regular chat message so
- `tui-widgets` — Author live widget apps for the Hermes TUI dock.
- `weekly-review-planning` — Weekly reset: commitments, stalled work, next-week plan.
- `xlsx` — Create, read, edit Excel .xlsx workbooks and CSVs.
## Hermes Gateway Troubleshooting (1)
## Creative (18)
- `hermes-gateway-troubleshooting` — # Hermes Gateway Troubleshooting & Migration Verification
## Hermes Mcp Troubleshooting (1)
- `hermes-mcp-troubleshooting` — # Hermes MCP Troubleshooting
## Hermes Runtime Worker Diagnostics (1)
- `hermes-runtime-worker-diagnostics` — # Hermes Runtime / Cron Worker Diagnostics
## Hermes Themes (1)
- `hermes-themes` — # Hermes Themes Skill
## Interactive Reference Pages (1)
- `interactive-reference-pages` — # Interactive Reference Pages
## Ios Hig Design (1)
- `ios-hig-design` — # iOS Human Interface Guidelines Design Skill
## Job Hunting (2)
- `job-hunting` — # Job Hunting & Career Transition Coaching
- `seek-extraction-patterns` — # SEEK Extraction Patterns
## Job Hunting Assistance (1)
- `job-hunting-assistance` — # Job Hunting Assistance
## Job Search Automation (1)
- `job-search-automation` — # Job Search Automation
## Job Tracker Enrichment (1)
- `job-tracker-enrichment` — # Job Tracker Enrichment (Anthony — Perth Marketing)
## Litprog (1)
- `litprog` — # Literate Programming
## Markdown To Anywhere (1)
- `markdown-to-anywhere` — # Markdown to Anywhere
## Mcp (2)
- `fastmcp` — # FastMCP
- `mcporter` — # mcporter
## Media (4)
- `gif-search` — # GIF Search (Tenor API)
- `heartmula` — # HeartMuLa - Open-Source Music Generation
- `songsee` — # songsee
- `youtube-content` — # YouTube Content Tool
## Mlops (10)
- `audiocraft` — # AudioCraft: Audio Generation
- `chroma` — # Chroma - Open-Source Embedding Database
- `clip` — # CLIP - Contrastive Language-Image Pre-Training
- `evaluating-llms-harness` — # lm-evaluation-harness - LLM Benchmarking
- `huggingface-hub` — # Hugging Face CLI (`hf`) Reference Guide
- `llama-cpp` — # llama.cpp + GGUF
- `qdrant` — # Qdrant - Vector Similarity Search Engine
- `segment-anything` — # Segment Anything Model (SAM)
- `serving-llms-vllm` — # vLLM - High-Performance LLM Serving
- `weights-and-biases` — # Weights & Biases: ML Experiment Tracking & MLOps
## Note Taking (2)
- `obsidian-vault-from-git` — ## Obsidian Vault from Git
- `obsidian-vault-manager` — # Obsidian Vault Manager
## Officecli (1)
- `officecli` — # officecli
## Omniroute (1)
- `omniroute-skills` — # OmniRoute Skills
## Pragmatic Programmer (1)
- `pragmatic-programmer` — # The Pragmatic Programmer Framework
## Productivity (25)
- `airtable` — # Airtable — Bases, Tables & Records
- `box` — # Box
- `custom-pet-authoring` — # Custom Pet Authoring
- `document-to-action-items` — # Document to Action Items
- `docx` — # Docx Skill
- `google-workspace` — # Google Workspace
- `maps` — # Maps Skill
- `meeting-action-items` — # Meeting Action Items
- `nano-pdf` — # nano-pdf
- `notion` — # Notion
- `notion-database-dedup` — # Notion Database Dedup
- `obsidian-git-sync` — # Obsidian Git Sync Workflow
- `ocr-and-documents` — # PDF & Document Extraction
- `pdf` — # PDF Skill
- `petdex` — # Petdex Skill
- `petdex-custom` — # Petdex Custom Pets
- `powerpoint` — # Powerpoint Skill
- `product-price-monitor` — # Product Price Monitor
- `session-librarian` — # Session Librarian
- `stratos-command-centre` — # Stratos — Command Centre
- `teams-meeting-pipeline` — # Teams Meeting Pipeline
- `telegram-cron-table-formatting` — # Telegram Table Formatting (Hermes)
- `tui-widgets` — # TUI Widgets Skill
- `weekly-review-planning` — # Weekly Review and Planning
- `xlsx` — # Xlsx Skill
## Proxmox Operations (1)
- `proxmox-operations` — # Proxmox Operations
## Refactoring Patterns (1)
- `refactoring-patterns` — # Refactoring Patterns Framework
## Refactoring Ui (1)
- `refactoring-ui` — # Refactoring UI Design System
## Research (14)
- `arxiv` — # arXiv Research
- `blogwatcher` — # Blogwatcher
- `chat-export-parsing` — # Chat Export Parsing
- `competitor-news-monitor` — # Competitor News Monitor
- `grounded-citations` — # Grounded Citations
- `llm-wiki` — # Karpathy's LLM Wiki
- `local-dining-scene-monitor` — # Local Dining Scene Monitor
- `operations-wiki` — # Operations Wiki
- `polymarket` — # Polymarket — Prediction Market Data
- `research-paper-writing` — # Research Paper Writing Pipeline
- `rss-feeds` — # RSS Feeds Skill
- `scrapling` — # Scrapling
- `searxng-search` — # SearXNG Search
- `wikilink-audit` — # Wikilink Auditor
## Restaurant Menu Extraction (1)
- `restaurant-menu-extraction` — # Restaurant Menu Extraction
## Smart Home (2)
- `openhue` — # OpenHue CLI
- `sonos` — # Sonos Skill
## Social Media (2)
- `reddit-reading` — # Reddit Reading Skill
- `xurl` — # xurl — X (Twitter) API via the Official CLI
## Software Design Philosophy (1)
- `software-design-philosophy` — # A Philosophy of Software Design Framework
- `architecture-diagram` — Dark-themed SVG architecture/cloud/infra diagrams as HTML.
- `ascii-art` — ASCII art: pyfiglet, cowsay, boxes, image-to-ascii.
- `ascii-video` — ASCII video: convert video/audio to colored ASCII MP4/GIF.
- `baoyu-infographic` — Infographics: 21 layouts x 21 styles (信息图, 可视化).
- `claude-design` — Design one-off HTML artifacts (landing, deck, prototype).
- `comfyui` — Generate images, video, and audio via diffusion workflows.
- `creative-ideation` — Generate ideas via named methods from creative practice.
- `design-md` — Author/validate/export Google's DESIGN.md token spec files.
- `excalidraw` — Hand-drawn Excalidraw JSON diagrams (arch, flow, seq).
- `humanizer` — Humanize text: strip AI-isms and add real voice.
- `hyperframes` — Render MP4/WebM videos from HTML compositions.
- `manim-video` — Manim CE animations: 3Blue1Brown math/algo videos.
- `p5js` — p5.js sketches: gen art, shaders, interactive, 3D.
- `popular-web-designs` — 54 real design systems (Stripe, Linear, Vercel) as HTML/CSS.
- `pretext` — Build creative browser demos with DOM-free text layout.
- `sketch` — Throwaway HTML mockups: 2-3 design variants to compare.
- `songwriting-and-ai-music` — Songwriting craft and Suno AI music prompts.
- `touchdesigner-mcp` — Control TouchDesigner via twozero MCP.
## Software Development (18)
- `code-wiki` — # Code Wiki Skill
- `codebase-inspection` — # Codebase Inspection with pygount
- `coding-router` — # Coding Router
- `dogfood` — # Dogfood: Systematic Web Application QA Testing
- `github` — # GitHub
- `hermes-agent-skill-authoring` — # Authoring Hermes-Agent Skills (in-repo)
- `inspecting-hermes-desktop-dom` — # Inspecting the live Hermes desktop DOM
- `node-inspect-debugger` — # Node.js Inspect Debugger
- `plan` — # Plan Mode
- `python-debugpy` — # Python Debugger (pdb + debugpy)
- `requesting-code-review` — # Pre-Commit Code Verification
- `rest-graphql-debug` — # API Testing & Debugging
- `self-hosted-personal-dashboard` — # Self-Hosted Personal Dashboard
- `simplify-code` — # Simplify Code — Parallel Review & Cleanup
- `spike` — # Spike
- `subagent-driven-development` — # Subagent-Driven Development
- `systematic-debugging` — # Systematic Debugging
- `test-driven-development` — # Test-Driven Development (TDD)
- `code-wiki` — Generate wiki docs + Mermaid diagrams for any codebase.
- `codebase-inspection` — Inspect codebases w/ pygount: LOC, languages, ratios.
- `coding-router` — Routes substantial software-development tasks to OpenCode CLI by default and Codex CLI as fallback, while Hermes remains
- `dogfood` — Exploratory QA of web apps: find bugs, evidence, reports.
- `github` — GitHub via gh CLI: PRs, issues, reviews, repos, auth.
- `hermes-agent-skill-authoring` — Author in-repo SKILL.md files: frontmatter and structure.
- `inspecting-hermes-desktop-dom` — Read the live Hermes desktop DOM/CSS over CDP.
- `node-inspect-debugger` — Debug Node.js via --inspect + Chrome DevTools Protocol CLI.
- `plan` — Write a markdown plan to .hermes/plans/; no execution.
- `python-debugpy` — Debug Python: pdb REPL + debugpy remote (DAP).
- `requesting-code-review` — Pre-commit review: security scan, quality gates, auto-fix.
- `rest-graphql-debug` — Debug REST/GraphQL APIs: status codes, auth, schemas, repro.
- `self-hosted-personal-dashboard` — >
- `simplify-code` — Parallel 4-agent cleanup of recent code changes.
- `spike` — Throwaway experiments to validate an idea before build.
- `subagent-driven-development` — Execute plans via delegate_task subagents (2-stage review).
- `systematic-debugging` — 4-phase root cause debugging: understand bugs before fixing.
- `test-driven-development` — TDD: enforce RED-GREEN-REFACTOR, tests before code.
## System Design (1)
## Research (14)
- `system-design` — # System Design Framework
- `arxiv` — Search arXiv papers by keyword, author, category, or ID.
- `blogwatcher` — Monitor blogs and RSS/Atom feeds via blogwatcher-cli tool.
- `chat-export-parsing` — Parse chat/messaging-platform export files (Telegram MTProto message.json, WhatsApp exports, Slack/Discord JSON) into re
- `competitor-news-monitor` — Watch named companies for material news; cited digests.
- `grounded-citations` — Ground answers and documents in cited, verifiable sources.
- `llm-wiki` — Karpathy's LLM Wiki: build/query interlinked markdown KB.
- `local-dining-scene-monitor` — Monitor city restaurant openings weekly with dedup.
- `operations-wiki` — Operational wiki for personal AI agents, self-hosted infrastructure, model routing and automation.
- `polymarket` — Query Polymarket: markets, prices, orderbooks, history.
- `research-paper-writing` — Write ML papers for NeurIPS/ICML/ICLR: design→submit.
- `rss-feeds` — Read RSS, Atom, JSON feeds; discover feeds behind a page.
- `scrapling` — Scrape sites with stealth browsing and Cloudflare bypass.
- `searxng-search` — Free keyless meta-search aggregating 70+ engines.
- `wikilink-audit` — Scan and fix broken [[wikilinks]] in an Obsidian vault or markdown knowledge base. Finds orphans, broken links, and temp
## Tailscale Serve Ops (1)
## Email (17)
- `tailscale-serve-ops` — # Tailscale Serve Ops 🦝🔗
- `agent-email-patterns` — Architecture patterns for AI agents that communicate over email -- why agents need dedicated inboxes rather than human e
- `agentmail` — Give the agent its own inbox: send and receive email.
- `agentmail-check-email` — Read, search, summarize, and triage AgentMail inboxes through the connected MCP server. Use for ANY request to look at,
- `agentmail-cli` — Operate AgentMail from a shell with the official CLI. Use when the user wants commands for listing or creating inboxes,
- `agentmail-manage-inboxes` — Create, list, inspect, update, or delete AgentMail inboxes through the connected MCP server. Use for ANY inbox lifecycle
- `agentmail-mcp` — Configure or troubleshoot the hosted AgentMail MCP server for Codex, Claude Code, Cursor, or another Streamable HTTP MCP
- `agentmail-rest-fallback` — AgentMail REST API operations for reading messages and downloading attachments when MCP tools are unavailable. Covers th
- `agentmail-sdk` — Deprecated alias of the agentmail skill, kept so existing installs and pinned URLs keep resolving. Prefer installing age
- `agentmail-send-email` — Draft, send, reply to, or forward email through the connected AgentMail MCP server. Use for ANY request to send, reply t
- `agentmail-toolkit` — Add AgentMail tools to agent frameworks with the TypeScript or Python AgentMail Toolkit. Use for Vercel AI SDK, LangChai
- `cron-email-briefing` — Produce structured email briefings from a cron job using IMAP (himalaya CLI). Covers the fallback workflow when no pre-i
- `email-for-ai-agents` — Deprecated alias of the agent-email-patterns skill, kept so existing installs and pinned URLs keep resolving. Prefer ins
- `email-inbox-triage` — Triage an inbox: prioritize threads, draft replies safely.
- `gmail-readonly-fetch` — Read-only email fetching from large Gmail inboxes via himalaya CLI or Python imaplib — cron-safe, paginated, with LLM ha
- `himalaya` — Himalaya CLI: IMAP/SMTP email from terminal.
- `mail-lookup` — Use when asked about one specific email, order, or receipt.
- `realtime-email-monitor` — >-
## Tech Ai Newsletter Digest (1)
## Mlops (10)
- `tech-ai-newsletter-digest` — # Tech & AI Newsletter Digest Skill
- `chroma` — Embedding database for RAG and semantic search.
- `clip` — Zero-shot image classification and image-text search.
- `evaluation` — W&B: log ML experiments, sweeps, model registry, dashboards.
- `evaluation` — lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).
- `huggingface-hub` — HuggingFace hf CLI: search/download/upload models, datasets.
- `inference` — llama.cpp local GGUF inference + HF Hub model discovery.
- `inference` — vLLM: high-throughput LLM serving, OpenAI API, quantization.
- `models` — AudioCraft: MusicGen text-to-music, AudioGen text-to-sound.
- `models` — SAM: zero-shot image segmentation via points, boxes, masks.
- `qdrant` — Vector search engine for production RAG systems.
## Ux Heuristics (1)
## Github (6)
- `ux-heuristics` — # UX Heuristics Framework
- `github-auth` — GitHub auth setup: HTTPS tokens, SSH keys, gh CLI login.
- `github-code-review` — Review PRs: diffs, inline comments via gh or REST.
- `github-issue-to-pr` — Carry a GitHub issue to a verified PR with honest CI state.
- `github-issues` — Create, triage, label, assign GitHub issues via gh or REST.
- `github-pr-workflow` — GitHub PR lifecycle: branch, commit, open, CI, merge.
- `github-repo-management` — Clone/create/fork repos; manage remotes, releases.
## Web (1)
## Apple (4)
- `blocked-page-recovery` — # Blocked-Page Recovery
- `apple-notes` — Manage Apple Notes via memo CLI: create, search, edit.
- `apple-reminders` — Apple Reminders via remindctl: add, list, complete.
- `findmy` — Track Apple devices/AirTags via FindMy.app on macOS.
- `imessage` — Send and receive iMessages/SMS via the imsg CLI on macOS.
## Web Design Reviewer (1)
## Frontend (4)
- `web-design-reviewer` — # Web Design Reviewer
- `design-alignment` — >-
- `impeccable` — Use when the user wants to design, redesign, shape, critique, audit, polish, clarify, distill, harden, optimize, adapt,
- `landing-dashboard-reference` — >
- `pixel-perfect-iteration` — >-
## Media (4)
- `gif-search` — Search/download GIFs from Tenor via curl + jq.
- `heartmula` — HeartMuLa: Suno-like song generation from lyrics + tags.
- `songsee` — Audio spectrograms/features (mel, chroma, MFCC) via CLI.
- `youtube-content` — YouTube transcripts to summaries, threads, blogs.
## Note Taking (2)
- `obsidian-vault-from-git` — Clone, initialise, or integrate a pre-existing Obsidian vault stored on Gitea or GitHub. Covers credential retrieval, gi
- `obsidian-vault-manager` — Manage Obsidian vaults — vault discovery, git sync, .obsidian/ state files, plugin config, rename safety, and troublesho
## Web Development (2)
- `interactive-html-pages` — # Interactive HTML Pages
- `page-agent` — # page-agent
- `interactive-html-pages` — Build self-contained interactive HTML pages served from a local HTTP server. Covers single-file pages, fetching JSON fro
- `page-agent` — Embed an in-page natural-language GUI copilot in web apps.
## Web Typography (1)
## Smart Home (2)
- `web-typography` — # Web Typography
- `openhue` — Control Philips Hue lights, scenes, rooms via OpenHue CLI.
- `sonos` — >
## Weekly Review (1)
## Mcp (2)
- `weekly-review` — # Weekly Review
- `fastmcp` — Build, test, and deploy Python MCP servers.
- `mcporter` — List, auth, and call MCP servers/tools from the terminal.
## Working With Legacy Code (1)
## Uncategorized (47)
- `working-with-legacy-code` — # Working Effectively with Legacy Code
- `chromadb` — >-
- `chromadb-preflight` — >
- `chromadb-skills-rag` — >
- `clean-architecture` — Structure software around the Dependency Rule: source code dependencies point inward from frameworks to use cases to ent
- `clean-code` — Write readable, maintainable code through disciplined naming, small functions, and clean error handling. Use when the us
- `cloudflare-tunnel-ops` — Operate Cloudflare Tunnels to expose internal services through public domains, bypassing network restrictions (e.g. work
- `cron-model-repair` — Repair cron jobs with dead model/provider pins.
- `ddia-systems` — Design data systems by understanding storage engines, replication, partitioning, transactions, and consistency models. U
- `devops`
- `devops-homelab-architecture-live-probe` — Build an accurate homelab architecture diagram by probing live infrastructure first, especially Proxmox VE and Tailscale
- `devops-proxmox-operations` — Use when operating Proxmox VE hosts, LXCs, VMs, storage, API tokens, noVNC consoles, DNS/hosts overrides, offline disk e
- `devops-service-management` — Umbrella for miscellaneous service operations skills — event logistics automation, Tailscale Serve, virtual desktop/X11,
- `devops-tailscale-web-routing` — Use when deploying or repairing self-hosted web apps behind Tailscale Serve/Funnel, especially path-prefix routing, stat
- `devops-telegram-bot-api-self-hosted` — |
- `domain-driven-design` — Model software around the business domain using bounded contexts, aggregates, and ubiquitous language. Use when the user
- `email-himalaya` — Himalaya CLI: IMAP/SMTP email from terminal. Covers configuration, cron automation, urgency assessment, and Gmail quirks
- `groktocrawl-deployment` — Self-hosted deployment guide for GroktoCrawl as a Firecrawl v2 replacement — resource requirements, port configuration,
- `hermes-api-vault` — Manage Hermes plugin/agent API credentials stored in a Notion vault — query the vault, map entries to ~/.hermes/.env var
- `hermes-desktop-plugins` — Write desktop app plugins that add UI panes and commands.
- `hermes-gateway-troubleshooting` — Gateway process management, MCP fleet debugging, post-migration verification, and config hardening for Hermes Agent LXC/
- `hermes-mcp-troubleshooting` — Diagnose and fix Hermes Agent MCP server connection issues, config corruption, startup banner false negatives, and relat
- `hermes-runtime-worker-diagnostics` — Use when many cron jobs fail at once, before job logic.
- `hermes-themes` — Author a Hermes color theme that skins every surface.
- `interactive-reference-pages` — Build clean, mobile-first, interactive HTML reference and checklist pages with proper dark-theme contrast, localStorage
- `ios-hig-design` — Design native iOS interfaces following Apple Human Interface Guidelines. Use when the user mentions "iPhone app", "iPad
- `job-hunting` — Comprehensive career transition coaching — research the user's background, discover opportunities, automate daily job-ra
- `job-hunting-assistance` — End-to-end job hunting support for a professional: research the user's background from available sources, locate existin
- `job-search-automation` — Operate, debug, and extend an autonomous job-board scraping pipeline (cron -> web_search_plus/web_extract_plus -> Camofo
- `job-tracker-enrichment` — Maintain and enrich Anthony's Notion "Job Opportunities 2026" Perth marketing/communications job tracker. Covers the bro
- `litprog`
- `markdown-to-anywhere` — On-demand cross-posting service: Anthony tells me to post a .md file to a platform, I format it appropriately and delive
- `officecli` — Create, analyze, proofread, and modify Office documents (.docx, .xlsx, .pptx) using the officecli CLI tool. Use when the
- `pragmatic-programmer` — Apply meta-principles of software craftsmanship: DRY, orthogonality, tracer bullets, and design by contract. Use when th
- `proxmox-operations` — Use when operating Proxmox VE hosts, LXCs, VMs, storage, API tokens, noVNC consoles, DNS/hosts overrides, offline disk e
- `refactoring-patterns` — Apply named refactoring transformations to improve code structure without changing behavior. Use when the user mentions
- `refactoring-ui` — Audit and fix visual hierarchy, spacing, color, and depth in web UIs. Use when the user mentions "my UI looks off" (or a
- `restaurant-menu-extraction` — Extract structured menu data (items, prices, ratings, delivery info) from food delivery sites like Uber Eats using Camof
- `software-design-philosophy` — Manage software complexity through deep modules, information hiding, and strategic programming. Use when the user mentio
- `system-design` — Design scalable distributed systems using structured approaches for load balancing, caching, database scaling, and messa
- `tailscale-serve-ops` — Operate Tailscale Serve to expose local web content through the tailnet. Covers route setup, backend management, path-pr
- `tech-ai-newsletter-digest` — Process and analyze tech/AI newsletter emails from Anthony's inbox using `himalaya` CLI, synthesizing key stories into s
- `ux-heuristics` — Evaluate and improve interface usability using heuristic analysis. Use when the user mentions "usability audit", "users
- `web-design-reviewer` — This skill enables visual inspection of websites running locally or remotely to identify and fix design issues. Triggers
- `web-typography` — Select, pair, and implement typefaces for web projects. Use when the user mentions "font pairing", "which typeface", "li
- `weekly-review` — Sunday 18:00 AWST cron that compiles a weekly review: job application stats from Notion, cron health from SQLite executi
- `working-with-legacy-code` — Safely change and test untested codebases using Feathers'' "Working Effectively with Legacy Code". Use when the user men
- `yuanbao` — Yuanbao (元宝) groups: @mention users, query info/members.
## Yuanbao (1)
## .Archive (45)
- `yuanbao` — # Yuanbao Group Interaction
- `agentmail-webhook-tailscale` — Real-time AgentMail webhooks via Tailscale funnel push.
- `anthropic-spend-forensics` — Use when unexplained Anthropic spend needs attribution.
- `briefing-visualization` — Research text → HTML briefing + PPTX deck.
- `container-cpu-forensics` — Find idle-CPU vampires: cgroup truth, crash loops.
- `cookie-jar-import` — Import browser cookies into Camofox for auth sessions.
- `courier-tracking-monitor` — Monitor a package/shipment; alert on change or delivery.
- `cover-letter-generation` — Draft, render, and deliver styled cover letters for Anthony's job applications. Covers content, format selection, PDF/DO
- `cron-prompt-service-integration` — Authed-API cron prompts: wrap calls in helper scripts.
- `cronjob-prompt-hardening` — Use when hardening cronjob prompts against fabrication.
- `cua-driver-linux-accessibility` — Fix AT-SPI bridge when computer_use returns 0x0 on Linux.
- `custom-dashboard-ops` — Operate or deploy the custom-dashboard on CT201.
- `data-share-to-ai-artefact` — Turn a file-share of KPI CSVs into one labelled xlsx for AI.
- `decision-briefing-dashboards` — Use when advice needs an offline decision dashboard.
- `employment-forms` — Fill out Anthony's employment and onboarding forms.
- `guanaco-ops` — Operate Guanaco LLM proxy on CT205.
- `hermes-config-backup` — Use when Hermes' Gitea config backup is stale or missing.
- `hermes-multiplex-telegram-ops` — Diagnose Hermes profile Telegram silence under multiplex.
- `hermes-profile-scoped-platform-troubleshooting` — Repair platform adapter mismatches across Hermes profiles.
- `hermes-session-context-ops` — Use when Hermes desktop chats lose or blend context.
- `hermes-ui-ops` — Operate Hermes desktop UI — Vite, HTTPS, mic, systemd.
- `hermes-web-backend-config` — Fix web_search backend selection and Tavily quota fallback.
- `homelab-migration-planning` — Plan homelab service migrations across Proxmox hosts.
- `in-repo-documentation` — Use for README and in-repository wiki documentation.
- `jobs-dashboard-server-ops` — Operate, debug, and extend the Notion-sourced job tracker dashboard (port 9099). Covers API routes, cover letter serving
- `live-application-verification` — Use when verifying a web change reaches the user's live app.
- `llm-proxy-streaming` — Use when an agent surfaces raw SSE chunks as errors.
- `media-download-ops` — Operate homelab torrent clients via Web API.
- `model-provider-comparison-research` — Use when comparing models inside an agent harness.
- `multi-agent-estate-knowledge` — Use for shared docs across multiple agent instances.
- `pre-flight-investigation` — Check the target and this box before installing anything.
- `pve-lxc-intrusion-detection` — Detect cryptominers on a Proxmox LXC via pct exec.
- `pve-vm-console-recovery` — Recover a stuck Proxmox VM by injecting console keystrokes.
- `qwenpaw-agent-operations` — Troubleshoot QwenPaw agents and model routing.
- `rclone-mount-ops` — Fix rclone FUSE mount recursion and CPU issues on PVE.
- `security-incident-response` — Investigate and remediate homelab security incidents.
- `self-hosted-dashboard-deployment-verification` — Use when verifying dashboard deployments.
- `session-verification` — Verify session claims — check DB, not memory.
- `simple-directives-execution` — Execute now when told 'just do X' — no investigation.
- `social-web-sentiment-research` — Use for current social/web sentiment scans.
- `soul-md-design` — Research SOUL.md patterns for structured agent personas.
- `tailnet-service-outage-triage` — Triage tailnet service-down alerts without PVE SSH.
- `tailscale-cert-for-docker-https` — Fix tailnet HTTPS cert errors with trusted Tailscale certs.
- `versioning-live-services` — Use when versioning a live service dir into a private repo.
- `video-translation-dubbing` — Translate and dub video audio to another language.
- `wiki-web-hosting` — Publish Markdown wikis as Quartz sites over Tailscale.
## Recent stocktake runs
## Finance (2)
- **2026-10-28 03:00** — regenerated this index (**302 skills**, mtime 03:01:40) and wrote `raw/stocktake/freshness-20261028.txt` (**2 stale / 103 fresh / 4 no-date of 109 pages**), then **failed** before committing: `RuntimeError: [400] … duplicate field 'reasoning_content' at line 1 column 512203` (provider `custom:omni-101 / free-stack`). Incident `1a8e90_f57ab4cafab8` (`detected`, no alert). This index and the freshness file were left uncommitted and were committed by the 2026-10-29 wiki audit — see [[systems/scheduled-tasks]] and [[incidents/2026-10-28-job-radar-stage1-iteration-cap]].
- `3-statement-model` — Build integrated IS/BS/CF financial workbooks in Excel.
- `australian-tax-return-review` — Review an Australian tax return against source records.
## Social Media (2)
- `reddit-reading` — Read Reddit: subreddits, search, threads, users. No browser.
- `xurl` — X/Twitter via xurl CLI: raw post search, posting, DM, media.
## Cron (1)
- `cron-digest-formatting` — Use when formatting cron Telegram news digests for Anthony.
## Data Science (1)
- `jupyter-live-kernel` — Iterative Python via live Jupyter kernel (hamelnb).
## Job Hunting (1)
- `seek-extraction-patterns` — Extract salary, closing dates, and job details from SEEK (au.seek.com) listings at scale. Use when enriching a batch of
## Omniroute (1)
- `omniroute-skills` — Interact with OmniRoute's Skills framework — list, install, enable, execute, and debug skills via REST API and MCP tools
## Web (1)
- `blocked-page-recovery` — Use when a fetch fails: 403/429, paywall, WAF, bot wall.
## Related
- [[systems/hermes-agent]]
- [[systems/omniroute]]