diff --git a/current-state.md b/current-state.md index 81441c4..4d9c697 100644 --- a/current-state.md +++ b/current-state.md @@ -14,6 +14,13 @@ sources: [live-config, live-cron-list, live-hermes-version] # Current State Updated: 2026-10-12 +## Recent Changes (2026-10-12, 03:00 stocktake; live probes 2026-10-12 03:0x-03:1x AWST) + +- **02 stocktake corrections from live probes.** [[systems/nanobot]] was DOWN as of 2026-08-07; its web UI on CT333 is **back up** (HTTP `200` on `100.122.189.81:8766`), though the OmniRoute port `8900` endpoint is still closed so `status` stays `degraded`. [[systems/home-assistant]] confirmed **up**: HTTP `200` on `100.68.139.5:8123`, VMID 100 running, Tailscale node offering an exit node — confidence raised low → medium. → [[systems/nanobot]], [[systems/home-assistant]] +- **Freshness snapshot: 54 stale / 32 fresh / 7 no-date of 93 pages** (threshold >21 days). The bulk of the stale list is the 2026-07-22 seeding cohort (concepts, runbooks, comparisons) and is structural, not a regression. → `raw/stocktake/freshness-20261012.txt` +- **Skills index regenerated: 346 skills / 70 categories**, unchanged count from the previous run. → [[systems/skills-index]] +- **Tailscale routes re-verified** against `tailscale serve status` — the CT460 table in [[infrastructure/tailscale]] matches live output exactly (13 tailnet routes + `:8443/webhook` Funnel + `:9443` Vite dev). Two nodes joined since 2026-10-11: `omarchy` (100.73.227.103) and `omniroute-1` (100.78.63.20, both active). + ## Recent Changes (2026-10-11 → 2026-10-12, nightly audit; live probes 2026-10-12 00:0x-00:2x AWST) - **The cron fleet is clean for a second consecutive day: 2026-10-11 ran `171 completed / 0 failed`** (+1 `running` = this audit), after 10-07 `1` / 10-08 `2` / 10-09 `15` failed. 10-10 finalises at **`175 / 0`**. Fleet **29 jobs — 26 enabled / 3 disabled**; non-clean `last_status` steady at **5**; **0 incidents opened, 0 closed — 5 unresolved** (3 `alerted`, 2 `detected`). The two streak-1 jobs (`dca8482e4f76`, `1b23c042e17d`) carry Friday 2026-10-09 timeouts and have not had a scheduled slot since. → [[systems/scheduled-tasks]] diff --git a/index.md b/index.md index 42bc405..6418af1 100644 --- a/index.md +++ b/index.md @@ -30,7 +30,7 @@ ### Integrations - [[systems/messaging-integrations]] — Telegram, Discord, email, Slack, etc. - [[systems/scheduled-tasks]] — **29 cron jobs (26 enabled, 3 disabled)**; **5 carry a non-clean `last_status`** as of 2026-10-12 00:10. **Two consecutive clean days: 2026-10-11 ran `171 completed` / 0 `failed`** (2026-10-10 finalised `175 / 0`), ending the 2026-10-09 provider timeout storm (15 failed executions / 10 jobs; 7 of its incidents auto-closed on 2026-10-10, 2 of its jobs remain `alerted`). The DHL monitor `62619aa6bcb3` was **retired** on 2026-10-10. See [[incidents/2026-10-09-cron-provider-timeout-storm]] -- [[systems/skills-index]] — Installed skills inventory (**346 skills**; auto-regenerated daily by the 03:00 stocktake job — verified 2026-10-07. Note: 347 `SKILL.md` files exist on disk — the generator de-dups one nested duplicate, see [[systems/scheduled-tasks]]; **regenerated daily — the 10-10 and 10-11 stocktake runs both committed (`d981e30`)**) +- [[systems/skills-index]] — Installed skills inventory (**346 skills**; auto-regenerated daily by the 03:00 stocktake job — verified 2026-10-12 by the stocktake run itself (`6fc109e` follow-on: 346 skills / 70 categories). Note: 347 `SKILL.md` files exist on disk — the generator de-dups one nested duplicate, see [[systems/scheduled-tasks]]; **regenerated daily — the 10-10, 10-11 and 10-12 stocktake runs all committed**) ## Incidents - [[incidents/2026-10-09-cron-provider-timeout-storm]] — Fleet-wide provider timeout storm: 15 failed executions across 10 jobs on 2026-10-09, all `RuntimeError: Request timed out.` on the aliases behind `custom:omni-101`, 00:00 → 15:00 AWST; 7 of its incidents auto-closed on 2026-10-10, **2 remain `alerted`** (`dca8482e4f76`, `1b23c042e17d` — both weekly jobs still carrying the Friday `RuntimeError: Request timed out.`) diff --git a/log.md b/log.md index 45f3224..3121aa9 100644 --- a/log.md +++ b/log.md @@ -505,3 +505,32 @@ and the skills-index blurb no longer claims no regeneration since 2026-10-07. `current-state.md`, `index.md`, `log.md`. No secrets written; job/incident IDs, paths and tailnet addresses only. + +## [2026-10-12] stocktake | wiki-maintain.sh run + 2 live-drift corrections + +**Script run.** `wiki-maintain.sh` completed all three stages: skills-index regenerated from the live +skill tree (**346 skills** across 70 categories, unchanged count), freshness audit written to +`raw/stocktake/freshness-20261012.txt` (**54 stale / 32 fresh / 7 no-date of 93 pages**), and the Quartz +step took its documented no-`quartz.config.ts` path (build skipped, `public/` unchanged — expected, not a +failure). The in-process watcher loop hit the 420 s tool cap, but the script had already finished and +written its artifacts, so results were read from the report rather than assumed. + +**Drift corrected from live probes (2 pages):** + +1. **[[systems/nanobot]]** — last verified 2026-08-07 as DOWN. Live probe 2026-10-12: the Nanobot web UI on + CT333 is **back up**, HTTP `200` on `http://100.122.189.81:8766/`. Port `8900` — the endpoint the + disabled OmniRoute `nanobot` provider connection uses — is **still closed**, so `status` stays + `degraded` rather than flipping to active. SSH to CT333 is unreachable from CT460, so + `whatsapp-bridge.service` was not re-verified. +2. **[[systems/home-assistant]]** — carried `confidence: low` plus a page-wide "cannot confirm HA runs" + caveat from 2026-07-22. Live: HTTP `200` on `http://100.68.139.5:8123/`, Tailscale node + `homeassistant` active and offering an exit node, VMID 100 running per `qm list`. Confidence raised + low → medium; the unverified residue is now narrowed to Hermes→HA entity access (no HA credentials on CT460). + +**Verified and left stale.** [[systems/openrouter]] and [[systems/nextcloud]] were probed shallowly and +nothing contradicting was found, so both stay in the stale list rather than being rewritten on guesswork. + +**Pages touched:** `systems/nanobot.md`, `systems/home-assistant.md`, `systems/skills-index.md`, +`raw/stocktake/freshness-20261012.txt`, `log.md`. + +No secrets written; job/incident IDs, paths and tailnet addresses only. diff --git a/raw/stocktake/freshness-20261012.txt b/raw/stocktake/freshness-20261012.txt new file mode 100644 index 0000000..13dd358 --- /dev/null +++ b/raw/stocktake/freshness-20261012.txt @@ -0,0 +1,106 @@ +# Wiki freshness audit — 2026-10-12 +Threshold: >21 days since `updated` + +## STALE (54) + +- [ 82d] comparisons/camoufox-vs-chrome.md (updated 2026-07-22) +- [ 82d] comparisons/hermes-vs-nanobot.md (updated 2026-07-22) +- [ 82d] concepts/agent-memory.md (updated 2026-07-22) +- [ 82d] concepts/model-routing.md (updated 2026-07-22) +- [ 82d] concepts/persistent-shells.md (updated 2026-07-22) +- [ 82d] concepts/tool-calling.md (updated 2026-07-22) +- [ 82d] decisions/index.md (updated 2026-07-22) +- [ 82d] decisions/keep-openrouter-as-fallback.md (updated 2026-07-22) +- [ 82d] decisions/persistent-shell-enabled.md (updated 2026-07-22) +- [ 82d] experiments/index.md (updated 2026-07-22) +- [ 82d] infrastructure/backups.md (updated 2026-07-22) +- [ 82d] infrastructure/credentials-map.md (updated 2026-07-22) +- [ 82d] infrastructure/domains-and-tunnels.md (updated 2026-07-22) +- [ 82d] infrastructure/network-map.md (updated 2026-07-22) +- [ 82d] runbooks/backup-wiki.md (updated 2026-07-22) +- [ 82d] runbooks/diagnose-docker.md (updated 2026-07-22) +- [ 82d] runbooks/recover-docker-service.md (updated 2026-07-22) +- [ 82d] runbooks/restart-browser.md (updated 2026-07-22) +- [ 82d] runbooks/restart-hermes.md (updated 2026-07-22) +- [ 82d] runbooks/restore-openrouter.md (updated 2026-07-22) +- [ 82d] runbooks/test-openrouter.md (updated 2026-07-22) +- [ 82d] runbooks/troubleshoot-tool-loop.md (updated 2026-07-22) +- [ 82d] systems/cloudflare-tunnel.md (updated 2026-07-22) +- [ 82d] systems/headless-server.md (updated 2026-07-22) +- [ 82d] systems/home-assistant.md (updated 2026-07-22) +- [ 82d] systems/openrouter.md (updated 2026-07-22) +- [ 82d] systems/terminal-backend.md (updated 2026-07-22) +- [ 79d] runbooks/gateway-resource-resilience.md (updated 2026-07-25) +- [ 79d] systems/tool-search.md (updated 2026-07-25) +- [ 78d] systems/homepage-dashboard.md (updated 2026-07-26) +- [ 78d] systems/obsidian.md (updated 2026-07-26) +- [ 76d] runbooks/job-radar.md (updated 2026-07-28) +- [ 76d] runbooks/multiplexer-setup.md (updated 2026-07-28) +- [ 75d] systems/opencode-go.md (updated 2026-07-29) +- [ 74d] systems/hermes-ui.md (updated 2026-07-30) +- [ 72d] incidents/2026-08-01-mnemosyne-capture-failure.md (updated 2026-08-01) +- [ 71d] comparisons/steel-vs-camoufox.md (updated 2026-08-02) +- [ 71d] concepts/browser-automation.md (updated 2026-08-02) +- [ 71d] decisions/rollback-steel-to-camofox.md (updated 2026-08-02) +- [ 71d] reports/uber-eats/index.md (updated 2026-08-02) +- [ 71d] systems/browser-backend.md (updated 2026-08-02) +- [ 66d] systems/nanobot.md (updated 2026-08-07) +- [ 61d] comparisons/memory-backends.md (updated 2026-08-12) +- [ 61d] comparisons/model-routers.md (updated 2026-08-12) +- [ 59d] systems/guanaco.md (updated 2026-08-14) +- [ 58d] agents/index.md (updated 2026-08-15) +- [ 58d] agents/open-claw.md (updated 2026-08-15) +- [ 58d] agents/zeroclaw.md (updated 2026-08-15) +- [ 58d] incidents/2026-08-15-skills-index-drift.md (updated 2026-08-15) +- [ 57d] CONTRIBUTING.md (updated 2026-08-16) +- [ 56d] systems/nextcloud.md (updated 2026-08-17) +- [ 45d] reports/communities-wa-social-media-audit-2026.md (updated 2026-08-28) +- [ 44d] incidents/2026-08-29-gmail-read-state.md (updated 2026-08-29) +- [ 38d] incidents/2026-09-02-sftpgo-outage.md (updated 2026-09-04) + +## FRESH (32) + +- [ 8d] agents/hermes-chaos.md +- [ 8d] agents/hermes-production.md +- [ 1d] agents/nanobot.md +- [ 1d] agents/qwenpaw.md +- [ 0d] current-state.md +- [ 0d] incidents/2026-08-18-mnemosyne-capture-pipeline.md +- [ 9d] incidents/2026-08-26-work-outlook-basic-auth-disabled.md +- [ 1d] incidents/2026-09-28-hermes-managed-runtime-deps.md +- [ 6d] incidents/2026-10-01-cliproxy-connection-auth-expired.md +- [ 8d] incidents/2026-10-03-ct450-hermes-update.md +- [ 8d] incidents/2026-10-03-mnemosyne-vector-recall-fts-only.md +- [ 6d] incidents/2026-10-05-dead-model-pin-repin.md +- [ 3d] incidents/2026-10-09-cron-provider-timeout-storm.md +- [ 3d] incidents/2026-10-09-memory-recall-relevance.md +- [ 1d] infrastructure/docker-services.md +- [ 8d] infrastructure/hosts.md +- [ 0d] infrastructure/proxmox.md +- [ 1d] infrastructure/tailscale.md +- [ 0d] queries/retained-research-answers.md +- [ 8d] runbooks/provider-health.md +- [ 8d] runbooks/update-hermes-safely.md +- [ 10d] systems/current-profile.md +- [ 1d] systems/hermes-agent.md +- [ 1d] systems/ignis.md +- [ 8d] systems/memory-backend.md +- [ 1d] systems/messaging-integrations.md +- [ 0d] systems/mnemosyne.md +- [ 11d] systems/model-providers.md +- [ 1d] systems/omniroute.md +- [ 1d] systems/paseo.md +- [ 0d] systems/scheduled-tasks.md +- [ 0d] systems/skills-index.md + +## NO updated DATE (7) + +- SCHEMA.md +- experiments/active/example.md +- incidents/2026-07-29-mnemosyne-relay.md +- incidents/2026-08-04-omniroute-free-deepseek-silent-stops.md +- index.md +- log.md +- reports/uber-eats/2026-07.md + +TOTAL: 54 stale, 32 fresh, 7 no-date of 93 pages \ No newline at end of file diff --git a/systems/home-assistant.md b/systems/home-assistant.md index 7fb47a1..03ab402 100644 --- a/systems/home-assistant.md +++ b/systems/home-assistant.md @@ -3,9 +3,9 @@ title: Home Assistant type: system status: active created: 2026-07-22 -updated: 2026-07-22 -verified_on: 2026-07-22 -confidence: low +updated: 2026-10-12 +verified_on: 2026-10-12 +confidence: medium tags: [system, automation] sources: [raw/configs/hermes-config-sanitized.txt] --- @@ -16,12 +16,13 @@ sources: [raw/configs/hermes-config-sanitized.txt] Home automation and device control; one of Hermes's messaging/integration platforms. ## Location -- **Verified (homelab infra docs, not this host):** VM 100 on `pve`, hostname `haos14.0` (Home Assistant OS), internal IP `172.30.232.1`. +- **Verified 2026-10-12 (live HTTP):** Home Assistant OS answers HTTP `200` on `http://100.68.139.5:8123/` over Tailscale, and the Tailscale node `homeassistant` (`100.68.139.5`) is **active** offering an exit node. The service is up and reachable on the tailnet. +- **Verified (homelab infra docs, not this host):** VM 100 on `pve`, hostname `haos14.0` (Home Assistant OS), internal IP `172.30.232.1`. `qm list` confirms VMID 100 `homeassistant` is **running** (13 GB RAM). - **Verified (Hermes config):** `homeassistant` and `hermes-homeassistant` appear in Hermes's platform/display integration lists (`~/.hermes/config.yaml`), alongside `qqbot`, `signal`, `slack`, `teams`, `telegram`. - **Unknown:** the actual connection details (HA base URL, long-lived access token, which entities/automations Hermes can reach) are not present in the sanitized config snapshot — likely held in a plugin-specific credentials file not captured there, or not yet wired up beyond being enabled as a platform. ## Current configuration -Not verified beyond the platform being listed as available/connected in `current-state.md`'s "WebUI connected platforms" line. No automations, entity lists, or voice-assistant config confirmed from this host. +**2026-10-12:** HA itself is confirmed up (HTTP 200 on `:8123`, VM running). Still **not** verified: that Hermes can actually reach an entity — no automation, entity list, or voice-assistant config has been confirmed from this host, and no HA credentials were found on CT460. ## Connected systems - [[systems/hermes-agent]] @@ -30,7 +31,10 @@ Not verified beyond the platform being listed as available/connected in `current - [[systems/homepage-dashboard]] — Shown in dashboard More tab ## Known issues -- Genuine gap, not a "known issue" per se: this page cannot confirm HA is *actively* used by Hermes vs. merely enabled — worth verifying directly (e.g. a live call to a known HA entity) next time someone is in this area. +- **Partially closed 2026-10-12:** HA itself is confirmed running and reachable (HTTP 200 on `100.68.139.5:8123`). Still open: whether Hermes can actually *use* it — a live call to a known HA entity would settle it, and requires the HA credentials, which are not on CT460. + +## Change history +- 2026-10-12: HA service confirmed UP — HTTP 200 on tailnet `:8123`, Tailscale node `homeassistant` active/exit-node, VMID 100 running per `qm list`. Confidence raised low → medium. Hermes↔HA entity access remains unverified. ## Relevant runbooks None yet — would be a good candidate once the connection is verified (e.g. `runbooks/restart-home-assistant-bridge`). diff --git a/systems/nanobot.md b/systems/nanobot.md index 654de8c..d094010 100644 --- a/systems/nanobot.md +++ b/systems/nanobot.md @@ -3,8 +3,8 @@ title: Nanobot type: system status: degraded created: 2026-07-22 -updated: 2026-08-07 -verified_on: 2026-08-07 +updated: 2026-10-12 +verified_on: 2026-10-12 confidence: high tags: [system, nanobot, network] sources: [live-probe, omniroute-investigation] @@ -16,9 +16,9 @@ Role in Anthony's setup. ## Location - **Verified 2026-07-22:** LXC 333, hostname `clawtest`, Tailscale IP `100.122.189.81`. ## Current state -- **Status: DOWN (as of 2026-08-07).** Connection refused on port 8900 — the OmniRoute provider connection to nanobot (`58debd59`, prefix `nanobot`) was causing event-loop stalls because every model sync + credential health check retried it. Connection disabled in OmniRoute. Nanobot on CT333 is not responding. -- **Prior state:** `nanobot` process listening on port `8766` (verified via `ss -tlnp` on ct333, 2026-07-22). -- `whatsapp-bridge.service` (systemd, active) — "WhatsApp Bridge for Nanobot" — also runs on ct333. Status unknown as of 2026-08-07. +- **Status: DEGRADED (re-verified 2026-10-12).** The Nanobot web UI on CT333 is **UP again**: HTTP `200` on `http://100.122.189.81:8766/` serving the nanobot web UI (verified this audit). However port `8900` — the endpoint the OmniRoute `nanobot` provider connection (`58debd59`, prefix `nanobot`) uses — is still **closed**, so the OmniRoute integration remains disabled. Contrast with 2026-08-07, when connection was refused on 8900 and no service was confirmed listening anywhere. +- **Prior state:** `nanobot` process listening on port `8766` (verified via `ss -tlnp` on ct333, 2026-07-22); DOWN 2026-08-07 (connection refused on 8900 — OmniRoute event-loop stalls from model sync + credential health-check retries; provider connection disabled in OmniRoute). +- `whatsapp-bridge.service` (systemd, active) — "WhatsApp Bridge for Nanobot" — also runs on ct333. Status not re-verified on 2026-10-12 (SSH to CT333 not reachable from CT460: connection timed out on 22/tcp; verification above is TCP/HTTP probing only). - Publicly reachable via the same Cloudflare Tunnel as Hermes: `nanobot.martinwa.org` → `100.122.189.81:8766`, `nano-ssh.martinwa.org` → SSH on the same host. See [[infrastructure/domains-and-tunnels]]. ## Current state as of 2026-07-22 (superseded — see below) @@ -45,6 +45,7 @@ Role in Anthony's setup. ## Change history - 2026-07-22: audit corrected "not running" claim after direct verification on ct333. - 2026-08-07: status changed to DEGRADED — nanobot confirmed DOWN (connection refused on 8900). OmniRoute provider connection disabled. CT333 service status unknown. +- 2026-10-12: web UI confirmed back up (HTTP 200 on `100.122.189.81:8766`, tailnet IP `100.122.189.81` = tailscale node `clawtest`). Status stays DEGRADED because the OmniRoute port 8900 endpoint is still closed. `whatsapp-bridge.service` not re-verified (SSH to CT333 unreachable from CT460). ## Relevant runbooks - [[runbooks/restart-hermes]] diff --git a/systems/skills-index.md b/systems/skills-index.md index abd7f88..7dd9872 100644 --- a/systems/skills-index.md +++ b/systems/skills-index.md @@ -3,8 +3,8 @@ title: Installed Skills type: system status: active created: 2026-07-22 -updated: 2026-10-11 -verified_on: 2026-10-11 +updated: 2026-10-12 +verified_on: 2026-10-12 confidence: high tags: [skills, hermes] sources: []