agent-estate-wiki/log.md

# Wiki Log

> Chronological record of wiki actions. Append-only.

## [2026-07-22] init | Fresh Hermes Systems Wiki
- Created technical wiki at `/home/hermes/wiki`
- Structure: systems/, infrastructure/, concepts/, decisions/, comparisons/, runbooks/, experiments/, queries/, raw/, _archive/
- Wrote SCHEMA.md with 11 operating rules
- Wrote index.md as curated Start here dashboard
- Reseeded system/infrastructure/runbook stubs for existing homelab
- Git initialized, local-only
- No secrets, passwords, tokens, or private keys written to vault

## [2026-07-22] audit | External audit pass (Claude, at Anthony's request)
- Merged duplicate pages: `systems/omniRoute.md` (empty stub) -> kept `systems/omniroute.md` (populated); repointed 5 links
- Merged duplicate pages: `runbooks/safely-update-hermes.md` -> kept as `runbooks/update-hermes-safely.md` (fuller content); repointed 2 links
- Added missing YAML frontmatter to 19 pages that lacked it (SCHEMA rule violation)
- Flagged 17 skeleton/placeholder pages explicitly as stubs (status: unresolved, confidence: low) instead of leaving them silently thin
- Linked 5 previously-orphaned pages (network-map, domains-and-tunnels, backups, cloudflare-tunnel, home-assistant) into index.md so they're reachable from Start Here
- Verified: no secrets/keys/passwords found in any wiki page (rule 1 compliance holds)
- Did not fabricate comparison/concept content — left for Hermes/Anthony to fill with verified facts

## [2026-07-22] populate | Fill stub pages with verified system facts (Claude, at Anthony's request)
- Populated with real, verified data (not fabricated): infrastructure/network-map, infrastructure/domains-and-tunnels, infrastructure/backups, systems/cloudflare-tunnel, systems/home-assistant, systems/headless-server, systems/hindsight, comparisons/camoufox-vs-chrome, comparisons/model-routers, comparisons/hermes-vs-nanobot, comparisons/memory-backends, concepts/persistent-shells, concepts/tool-calling, concepts/browser-automation, concepts/agent-memory
- Corrected systems/nanobot.md: Nanobot is actually running, on LXC 333 ("clawtest"), not this host — earlier "not running" claim was true only for ct460 and got conflated with pve's own IP
- Found and flagged a real inconsistency: decisions/use-hindsight-for-memory.md still names Hindsight as primary backend while current-state.md/systems/memory-backend.md say Mnemosyne — marked unresolved rather than silently overwritten
- Fixed a broken `review_after: 0001-01-01` in decisions/keep-openrouter-as-fallback.md
- New security-relevant findings recorded in-place: x11vnc on this host has no password and binds to all interfaces (systems/headless-server.md)
- queries/retained-research-answers.md intentionally left as a stub — it's a running research log to be filled by Hermes over time, not something system inspection can populate

## [2026-07-22] correction | Purge Hindsight (per Anthony, direct correction)
- Anthony confirmed Hindsight has nothing to do with this Hermes box (ct460) — it was confused with a different Hermes instance during earlier wiki authoring
- Deleted: systems/hindsight.md, decisions/use-hindsight-for-memory.md, runbooks/check-hindsight.md
- Rewrote comparisons/memory-backends.md and concepts/agent-memory.md to drop Hindsight as this box's predecessor backend
- Removed Hindsight references from: systems/memory-backend.md, systems/hermes-agent.md, systems/mnemosyne.md, current-state.md, decisions/index.md, index.md
- Left in place (not deleted, just flagged): archived hindsight-*.py scripts under ~/.hermes/scripts/ on this host — their presence is noted in comparisons/memory-backends.md as unexplained, possibly generic install-template leftovers, not evidence of local use
- Confirmed: Mnemosyne is, and has only ever been, this box's memory backend

## [2026-07-22] cleanup | Remove leftover Hindsight scripts (per Anthony)
- Deleted /home/hermes/.hermes/scripts/hindsight-watchdog.py.archived, hindsight_migration_watchdog.py.archived, honcho_to_hindsight_import.py.archived
- These were the last physical trace of the earlier Hindsight/another-Hermes mix-up (see prior wiki + decisions correction); nothing Hindsight-related remains on this host or in this wiki

## [2026-07-25] operations | Tool Search and gateway resilience
- Documented always-on native Hermes Tool Search, its verified context reduction, new-tool behavior, and rollback backup
- Documented how new skill categories are derived from the first folder below `~/.hermes/skills/`
- Documented CT460 gateway reclaim/CPU protection and the memory containment applied to Camofox and Hermes Web UI
- Recorded the I/O-controller delegation limitation so an ineffective `IOWeight` is not mistaken for active protection
- Added health checks, exact effective settings, backup location, and rollback procedure
- No secrets, tokens, or private credentials were written to the wiki

## [2026-07-28] dashboard | Jobs Dashboard feature additions
- Fixed bulk status update (payload shape mismatch: `{ids, status}` → `{ids, updates: {status}}`)
- Fixed column header sorting (sort key suffix mismatch, added all 8 column sort handlers)
- Added Phone Screen as a dedicated KPI filter card (separated from Interviews)
- Added Market Intel section at top of page (lazy-loaded on expand, refreshes on Refresh)
- Added rich text notes editor (Write/Preview with markdown: bold, italic, links, bullets) syncing to Notion `Notes / JD Summary`
- Added document attachment system (bookmark blocks on Notion page body, lazy-loaded, add/delete via API)
- Added 2 strong-match roles manually: Steel Blue Marketing Manager – APAC, Revo Fitness Campaign Manager
- Updated Synergy and Dept of Communities to Phone Screen status
- Updated wiki runbook with full dashboard feature documentation and API endpoint table

## [2026-07-27] incident | Job Radar search and dashboard recovery
- Documented the contaminated morning radar run, 16-record cleanup, deterministic seven-day freshness/direct-advert rules, Pacific Energy exclusion, full Notion pagination and strong-match cover-letter workflow
- Added the non-negotiable same-day cover-letter self-healing audit: retry once, verify all five artifacts, require canonical HTML/HTTP 200, and require `MISSING_COVER_LETTER_COUNT=0`
- Recorded the production cron model (`gpt-5.6-terra`) and verified first inherited fallback (`opencode-go/mimo-v2.5`)
- Documented dashboard direct-advert links, one-click KPI/stat/funnel filters, and retained Notion-backed status editing
- Recorded the CT460 Tailscale route-table loss and corrected atomic restoration of both `:443` Serve and `:8443/webhook` Funnel
- Added [[runbooks/job-radar]] and updated scheduled-task, Tailscale and index references
- Verified no secrets, API keys or private credentials were written to the wiki

## [2026-07-29] update | Steel Browser deployment — browser backend switch
- **New page:** `systems/browser-backend.md` — rewritten to reflect Steel Browser as primary, Camoufox as fallback
- **New page:** `comparisons/steel-vs-camoufox.md` — comparison of the two backends
- **Updated:** `infrastructure/hosts.md` — added CT201 and CT460 to verified hosts table
- **Updated:** `infrastructure/docker-services.md` — added Steel Browser compose file
- **Updated:** `infrastructure/tailscale.md` — added CT201 Steel serve routes
- **Updated:** `index.md` — browser-backend link description
- **Updated:** `current-state.md` — added 2026-07-29 changes section
- **Config change:** `browser.cdp_url` set to `http://100.96.244.39:9223` in ops profile

## [2026-07-30] update | Open WebUI updated on CT245
- Updated image to `ghcr.io/open-webui/open-webui:main` (new digest e97bf9531916, Jul 27)
- Bumped CT245 memory from 2592→4096 MB, swap 1600→2048 MB to prevent OOM on latest image
- Restarted container after OOM hang
- Updated `infrastructure/proxmox.md`

## [2026-08-02] incident | Mnemosyne embedding coverage gap discovered and fixed
- Health monitor was reporting vector index counts but not actual embedding coverage, hiding a critical gap
- Working memory embeddings were at 12.4% (186/1,504) — early memories from July 17-26 were never embedded
- Ran `mnemosyne reindex --yes` to rebuild all embeddings — working memory now at 100%
- Updated health check script (`mnemosyne-health-check.sh`) to report actual embedding coverage via new helper script (`check_embedding_coverage.py`)
- Created embedding coverage monitoring cron job (b63ed5817212) — runs every 6 hours, alerts if working memory coverage drops below 95%
- Added auto-reindex rule to MEMORY.md — will run reindex automatically if coverage drops, no user prompt needed
- DB grew from 8.7M to 21M with the new embeddings

## [2026-08-01] incident | Mnemosyne capture pipeline repaired
- Confirmed automatic conversation capture failure with a pre-fix unique-marker E2E test: Hermes completed the turn but no `working_memory` row existed.
- Corrected stringified structured YAML (`sync_roles`, `skip_contexts`, `reflect`, `sleep_threshold`, `ignore_patterns`) and enabled profile isolation; `hermes config check` passed.
- Verified the repair with a second CLI marker that persisted as an exact normalized `[USER]` database row.
- Corrected stale canonical model facts for light-only design, numbered Obsidian structure, and CT460 root route `:9119`; added authoritative identity/preference/workflow slots.
- Reduced legacy always-injected memory files from 8,013 bytes to 1,080 bytes, with timestamped backups retained.
- Added [[incidents/2026-08-01-mnemosyne-capture-failure]] and refreshed [[systems/mnemosyne]] plus [[systems/memory-backend]].

## [2026-08-02] update | Wiki audit and Steel→Camoufox rollback documentation
- Rewrote [[comparisons/steel-vs-camoufox]] — full Steel experiment story, timeline, lessons learned
- Rewrote [[systems/browser-backend]] — Camoufox primary, Steel disabled
- Updated [[current-state]] — browser revert, Mnemosyne vector gap, DeepSeek 0731, Uber Eats upgrade, GH Trending removal, Tembo→Tusk rename
- Updated [[concepts/browser-automation]] — added Steel experiment lesson
- Created [[decisions/rollback-steel-to-camofox]] — formal decision record
- Updated [[systems/scheduled-tasks]] — job count 25, Uber Eats upgrade, GH Trending Daily removed
- Updated [[systems/mnemosyne]] — vector embedding coverage gap documented
- Updated [[systems/model-providers]] — DeepSeek V4 Flash 0731 details
- Created [[reports/uber-eats/index]] — landing page for monthly spend reports
- Updated [[decisions/index]] — linked new decision page
- Updated [[index.md]] — browser description, reports section, last-updated date
- Created nightly wiki audit cron job (23:59 AWST)

## [2026-08-02] update | Ops profile Mnemosyne enabled and wiki sync
- Ops profile (`~/.hermes/profiles/ops/config.yaml`): enabled `mnemosyne` plugin with `profile_isolation: true`, `allow_tool_override: false`, `shared_surface_read: false`
- Tusk (ops) and Rhino (default) now have independent memory banks — zero cross-contamination
- Verified end-to-end: memory writes land correctly in both profiles independently
- Updated [[systems/mnemosyne]] — profile isolation section now documents ops config
- Updated [[current-state]] — added ops profile Mnemosyne entry
- Updated [[runbooks/multiplexer-setup]] — fixed "Elephant ops" → "Tusk ops" in architecture diagram
- Fixed [[systems/memory-backend]] — "Shared surface empty" section renamed to reflect intentional isolation decision

## [2026-08-04] update | OmniRoute free-DeepSeek silent-stop mitigation
- Incident: free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on `deepseek-flash-free` and `smart-route`/`good-free` across both instances
- Reworked `deepseek-flash-free` and `good-free` on CT600: strategy `priority` → `fill-first`, dropped proven culprits (`238d5978/deepseek-v4-flash` + `-0731`, `nvidia` deepseek), healthy free models promoted, deepseek demoted to last resort
- Created [[incidents/2026-08-04-omniroute-free-deepseek-silent-stops]]
- Updated [[current-state]] — added OmniRoute free-DeepSeek mitigation to Recent Changes
- Updated [[systems/omniroute]] — added Recent Changes (2026-08-04) section
- Updated [[systems/model-providers]] — added mitigation entry under Notable Model Updates, updated `updated` date

## [2026-08-07] update | Nightly wiki audit — OmniRoute fix, Nanobot degraded, scheduled tasks rewrite
- **OmniRoute three-layer fix documented:** npm package .env override (ENABLE_SOCKS5_PROXY=true re-installed by npm update), dead nanobot provider connection (CT333 DOWN), PVE host load leaking into CT600 via nesting=1. First-request-after-cache-expiry stall pattern. `is_active=0` does NOT stop CredentialHealth probes. All lessons banked in omniroute-ops skill.
- **Updated [[systems/omniroute]]** — added Recent Changes (2026-08-07) section with full root cause analysis
- **Updated [[systems/nanobot]]** — status changed from "active" to "degraded" (DOWN, connection refused on 8900). Updated confidence, sources, verified_on.
- **Updated [[current-state]]** — added 2026-08-07 changes: OmniRoute fix, Nanobot degraded, CT460 memory bump
- **Rewrote [[systems/scheduled-tasks]]** — fixed duplicate entries (5 duplicates removed), added 8 missing jobs (Important-mail monitor, CT221 Miner Watchdog, Job Tracker Dashboard Refresh, Evening Stand-Down Briefing, Daily Birthday Checker, Mnemosyne Embedding Coverage Monitor, Hermes Config Backup, Review Agent Estate). Count corrected: 25 → 28.
- **Wiki audit cron error noted:** model drift (smart-route → deepseek-paid-smart) causing RuntimeError on Wiki Audit job
- **Updated [[index.md]]** — last-updated date 2026-08-04 → 2026-08-07

## [2026-08-09] update | Nightly wiki audit — Portainer reset, SFTPGo review
- **Portainer password reset documented:** CT245 admin user confirmed as `anthony` (not `admin`). Password reset via helper container, verified with auth API. Recovery procedure recorded. Added to [[current-state]].
- **SFTPGo security review documented:** CT299 findings — root access, unpinned fingerprint, `//` root path, F1 quota mystery. Added to [[current-state]] and [[infrastructure/proxmox]].

## [2026-08-08] update | Cron model repinning, hermes-ui revert, Phase One review
- **Cron job model repinning:** 13 jobs repinned from dead/stale provider combos. 3 from `freeapi/minimax-m2.7` → `smart-route` (IDs: `c8d52904e1d6`, `1b23c042e17d`, `d40085da631c`). Mosquito Tracker from `freeapi/auto` → `smart-route` (ID: `1b31187397a5`). 9 stale-named jobs (e.g. `vertex/gemini-2.5-flash`, `deepseek-v4-flash-free`) → `smart-route`. Daily Marketing Job Radar → `gpt-5.6-terra`.
- **Hermes UI vite config revert:** Vite config was changed to HTTPS with self-signed certs (~16:00), breaking iPad access. Reverted to pre-HTTPS config, restarted `hermes-ui.service`. Production build serves `/ui/` path correctly with relative asset paths. `hermes-desktop.martinwa.org` identified as separate Cloudflare infrastructure (likely CT450), not fixable from CT460.
- **Phase One estate review completed:** Manual cron run (ID `a9359b7db27b`) verified CT450's 11 duplicate publishing jobs remain paused (zero executions since Aug 1), CT460's 11 production jobs remain enabled with `last_status=ok`, CT333 Nanobot has live `jobs.json` with 9 jobs. Cron review report contained two inaccuracies (wrong date, false claim about CT333) — verified via live system probes.
- **Updated [[current-state]]** — added 2026-08-08 changes section
- **Updated [[systems/scheduled-tasks]]** — Phase One review marked completed, model repinning noted in Known Issues

## [2026-08-10] audit | Nightly wiki audit — cron failures, session storage issue
- **Cron job failures documented:** Four jobs failed on 2026-08-10 with two error patterns:
  - TimeoutError (idle 601–602s): Daily Marketing Job Radar, Political News Digest
  - RuntimeError (session storage write failure): Tech & AI Newsletter Digest, GitHub Trending Radar — Weekly Roundup
- **Root cause identified:** Disk at 83% (11GB free on 65GB), `state.db` at 593MB. Session storage likely experiencing WAL checkpoint or disk pressure issues.
- **Updated [[current-state]]** — added 2026-08-10 changes section with cron failures and OpenCode WebSocket errors
- **Updated [[systems/scheduled-tasks]]** — added session storage failures to Known Issues, updated verified_on to 2026-08-10
- **OpenCode WebSocket proxy errors noted:** Repeated "OpenCode service unavailable" in hermes node journal (Aug 11 00:08). Process running, likely transient.
- **No new pages created** — all changes applied to existing pages

## [2026-08-12] update | Nightly wiki audit — OmniRoute AutoRoutes, OpenViking evaluation, FreeLLMAPI internals
- **OmniRoute AutoRoute variants documented:** `auto/best-free` (Auto Cheap) and `auto/coding-free` are **virtual** variants — not among the 54 persisted combos. LKGP engine resolves them dynamically by scoring live connected accounts (antigravity, opencode-zen, nvidia, openrouter) on health/quota/latency/errors/cost-inverse, with instant failover. Example observed: `auto/best-free` routed to Antigravity. Updated [[systems/omniroute]].
- **OpenViking memory provider evaluated, NOT adopted:** single-slot config would fully replace Mnemosyne (no parallel mode); risk assessment flagged schema/embedding/scope/provenance loss and no rollback. Config remains `memory.provider: mnemosyne`. Added to [[comparisons/memory-backends]].
- **FreeLLMAPI router internals documented:** 3-axis weighted scoring (Reliability/Speed/Intelligence) + guardrail multiplier + Penalty Inspector + vision/tool capability matching. Router behind Hermes's default `freellmapi-auto`. Added to [[comparisons/model-routers]].
- **No new pages created** — all changes applied to existing pages. Routine cron (Mnemosyne health, embedding coverage, CT221 miner watchdog, important-mail monitor) passed clean; no incidents.

## [2026-08-13] update | Nightly wiki audit — OmniRoute CT600→CT601 migration
- **OmniRoute migrated from CT600 to CT601:** Provider renamed `custom:omniroute` → `custom:omni`; base URL `http://omniroute:20128/v1` → `http://omni:20128/v1`. Host `omni.kangaroo-eel.ts.net` (100.93.204.4, active) replaces `omniroute.kangaroo-eel.ts.net` (100.88.81.19, offline). CT600 stopped, CT601 running. DB path `/var/lib/omniroute/storage.sqlite` → `/opt/omniroute/storage.sqlite`; API key unchanged. All config (14) + cron-job (22) `custom:omniroute` refs repointed to `custom:omni`. Pre-migration backup at `~/.hermes/config.yaml.bak.omniroute-removal`. All facts verified live (`pct list`, `grep`, Tailscale DNS).
- **Updated [[systems/omniroute]]** — Current Configuration rewritten for `omni`/CT601, added Recent Changes (2026-08-13), bumped `updated`/`verified_on` to 2026-08-13.
- **Updated [[infrastructure/proxmox]]** — CT600 marked stopped/retired, CT601 added running; bumped `updated`.
- **Updated [[current-state]]** — attested provider/host facts updated, added Recent Changes (2026-08-13) including evening stand-down weather bugfix.
- **Updated [[infrastructure/hosts]]** — OmniRoute proxy entry → `omni`; CT600 reference marked verified/retired.
- **Updated [[index]]** — last-updated date 2026-08-10 → 2026-08-13.
- No new pages created; all changes applied to existing pages. No secrets written.

## [2026-08-15] update | Nightly wiki audit — Skills index drift fix, agent pages added, scheduled-tasks updated
- **Skills index drift incident:** `systems/skills-index.md` claimed 66 skills when 330 were actually installed (4x undercount). Page was hand-written once on 2026-07-22/25 and never regenerated. Fixed by regenerating from live skill tree, creating `regenerate-skills-index.sh` script, and adding cron job `1a8e90ea2271` (daily 03:00 AWST) to auto-regenerate and commit to Gitea. See [[incidents/2026-08-15-skills-index-drift]].
- **Updated [[systems/skills-index]]** — added auto-regeneration notice, updated counts (66→330), bumped `updated`/`verified_on` to 2026-08-15.
- **Updated [[systems/scheduled-tasks]]** — added Skills Stock-Take Regeneration job, bumped count (28→29), updated dates.
- **Updated [[systems/hermes-agent]]** — skills count (66→330), cron jobs count (25→29), added Mattermost gateway warning note, updated dates.
- **Updated [[index.md]]** — added Incidents section with link to skills drift incident, updated last-updated date to 2026-08-15.
- **Created [[incidents/2026-08-15-skills-index-drift]]** — new incident page documenting the 4x undercount, root cause (static docs for mutable inventory), resolution (auto-regeneration), and prevention (daily cron).
- No secrets written.

## [2026-08-17] update | Nightly wiki audit — Nextcloud ncdownloader crash
- **New page:** `systems/nextcloud.md` — documented Nextcloud CT270: version 33.0.2.2, apps inventory, ncdownloader incompatibility incident, access details, related pages.
- **Updated [[current-state]]** — added Recent Changes (2026-08-17) section with ncdownloader fix details.
- **Updated [[infrastructure/proxmox]]** — CT270 entry now shows Nextcloud version and ncdownloader status.
- **Updated [[index.md]]** — added Nextcloud link to Core systems, bumped last-updated to 2026-08-17.
- No secrets written.

## [2026-08-16] update | Nightly wiki audit — OmniRoute capacity pressure, Agent Zero sluggishness, Hermes verification quirk
- **OmniRoute upstream capacity pressure documented:** `ServiceUnavailableError: Structurally heavy chat request capacity is busy` reported across agents via OmniRoute. Root cause: upstream OpenAI capacity pressure (recent incident Aug 11, now resolved). OmniRoute's context compression not always applied before sending oversized requests, amplifying the issue. Diagnostic requirements recorded.
- **Agent Zero sluggishness confirmed as orchestration overhead:** GPT-5.6 Luna is fast — the latency comes from Agent Zero's orchestration layer (large system prompts, multi-step reasoning, memory lookups, sub-agent spawning). GPT-5.6 Luna confirmed as real model (lightweight/fast tier, launched Jul 9, 2026).
- **Hermes model verification quirk documented:** Custom endpoint verification can fail with "could not reach" when the endpoint returns HTTP 402 (auth required) during unauthenticated transient checks. Config `http://omni:20128/v1` is correct — `/v1` is required.
- **Updated [[systems/omniroute]]** — added Recent Changes (2026-08-16) section, bumped `updated`/`verified_on` to 2026-08-16.
- **Updated [[current-state]]** — added Recent Changes (2026-08-16) section, bumped `updated` to 2026-08-16.
- **Updated [[index.md]]** — bumped last-updated date to 2026-08-16.
- No new pages created. No secrets written.

## [2026-08-18] incident | Mnemosyne capture pipeline broken (active)
- **New page:** `incidents/2026-08-18-mnemosyne-capture-pipeline` — working memory capture (`sync_turn`) stopped functioning after 2026-08-16 23:15:39, ~48 hours of conversation turns not captured. Root cause: silent guard conditions in `run_agent.py:_sync_external_memory_for_turn()`. MEMDEBUG logging recommended. Config verified clean (native YAML types). Still active as of verification.
- **Updated [[systems/mnemosyne]]** — added 2026-08-18 incident to History and Evidence, bumped `updated`/`verified_on` to 2026-08-18, added cross-reference to Related.
- **Updated [[current-state]]** — added Recent Changes (2026-08-18) section with capture pipeline incident, bumped `updated` to 2026-08-18.
- **Updated [[index.md]]** — added 2026-08-18 incident to Incidents section, bumped last-updated to 2026-08-18.
- **Marked [[incidents/2026-08-01-mnemosyne-capture-failure]]** — status changed from `active` to `resolved` (that earlier incident was fixed).
- No secrets written.

## [2026-08-23] update | Nightly wiki audit — Cron failures, wiki-stocktake drift, Mnemosyne watchdog history
- **Cron job failures documented (Weekly Review session):** Three distinct failure patterns identified in past 7 days:
  - `wiki-stocktake-maintenance` (1a8e90ea2271) failing ×4 (Aug 20–23) — config drift skip: model changed from `cx/gpt-5.6-luna` → `go-cheap`/`opencode-go/gpt-5.6-luna`, job unpinned. Requires explicit provider/model pinning.
  - CT221 gitea miner watchdog (d5516d0a5147) failing ×6+ (Aug 19) — SSH to PVE (root@100.96.100.82) failed repeatedly; CT 221 status = UNKNOWN, never inspected.
  - Mnemosyne Watchdog (96eb95ce12f2) major incident (Aug 19 early AM) — no consolidation for 36–41.5+ hours (last seen Aug 17 07:15). Watchdog cycled for ~2 hours before stopping.
- **Updated [[systems/scheduled-tasks]]** — added Known Issues section with cron failure patterns, bumped `updated`/`verified_on` to 2026-08-23.
- **Updated [[systems/mnemosyne]]** — added Mnemosyne Watchdog Aug 19 incident to History section, bumped `updated`/`verified_on` to 2026-08-23.
- **Updated [[current-state]]** — added Recent Changes (2026-08-23) section with cron failures summary.
- **Updated [[index.md]]** — bumped last-updated date to 2026-08-23.
- No secrets written.

## [2026-08-19] update | Nightly wiki audit — Ignis deployment, Command Code provider, Tavili search status
- **New page:** `systems/ignis.md` — documented Ignis browser-native Obsidian (Docker `ignis-subpath:0.8.9`, port 8080, Tailscale Serve at `hermes.kangaroo-eel.ts.net/ignis`). Volume mount: `/home/hermes/wiki/agent-estate-wiki:/vaults/wiki`. Verified live: container running, Tailscale route active.
- **Updated [[systems/model-providers]]** — added `command-code` provider (`https://api.commandcode.ai/provider/v1`) to Current Provider Stack. Bumped `updated`/`verified_on` to 2026-08-19.
- **Updated [[systems/omniroute]]** — added Recent Changes (2026-08-19) section documenting Command Code provider, `deepseek-flash-mix` combo, and Tavili search proxy status. Bumped `updated`/`verified_on` to 2026-08-19.
- **Updated [[infrastructure/docker-services]]** — added Ignis compose file and service definition.
- **Updated [[infrastructure/hosts]]** — added Ignis to CT460 row.
- **Updated [[systems/obsidian]]** — added Ignis as browser interface, corrected vault path to `agent-estate-wiki/`, added Quartz static site reference.
- **Updated [[current-state]]** — added Recent Changes (2026-08-19) section with Ignis deployment, Command Code provider, and Tavili search status.
- **Updated [[index.md]]** — added Ignis link to Core systems, bumped last-updated to 2026-08-19.
- No secrets written.
## [2026-08-26] update | Nightly wiki audit — Cron job reconciliation
- **Updated [[systems/scheduled-tasks]]** — reconciled live `hermes cron list` (27 jobs) against wiki table (29 entries). Removed: `Workday start reminder` (no longer in live list), `CT 221 Miner Watchdog` (job `d5516d0a5147` removed; CT221 status unknown), `Review Agent Estate Phase One` (one-time job completed 2026-08-08). Added: `weekly-save-restricted-bot-restart` (`0 3 * * 0`, local, no-agent). Updated count from 29 to 27.
- **Updated [[current-state]]** — added Recent Changes (2026-08-26) section documenting cron reconciliation and wiki-stocktake-maintenance drift persistence.
- **Updated [[index.md]]** — bumped last-updated to 2026-08-26, corrected job count reference (29 → 27).
- **Known Issues cleanup** — removed stale CT221 miner watchdog SSH failure entry (job no longer exists).
- No secrets written.

## [2026-09-02] incident | SFTPGo outage on CT299
- **New page:** `incidents/2026-09-02-sftpgo-outage.md` — documented CT299 SFTPGo outage (02:18 AWST). Root cause: CT299 swap exhausted (43Gi used, 100%), `tailscaled` died; PVE `pct start` failed due to 5 stale `lxc-attach` processes pinning cgroup. Killed PIDs 3037951/3334928/3352203/3402511/4004148, `pct stop 299 --skiplock` + `pct start 299`. Verified end-to-end: direct Tailscale connection, SFTPGo WebAdmin HTTP 401, SFTP banner `SSH-2.0-SFTPGo_2.7.0`.
- **Updated [[current-state]]** — added Recent Changes (2026-09-02) section, bumped `updated`/`verified_on` to 2026-09-04.
- **Updated [[infrastructure/proxmox]]** — CT299 entry already documented incident; no change needed.
- No secrets written.

## [2026-08-28] update | Department of Communities role and social-media audit
- **New page:** `reports/communities-wa-social-media-audit-2026` — durable strategic summary of Anthony's supplied public-source audit for his Department of Communities Social Media Officer role.
- **Updated [[systems/current-profile]]** — recorded current employment context and linked the report.
- **Updated [[index.md]]** — added the report to the Reports section and refreshed the last-updated date.
- No secrets written.

## [2026-09-20] update | Nightly wiki audit — No changes in past 24h
- **No new pages created** — no new systems, decisions, or incidents in the past 24 hours.
- **No existing pages updated** — all wiki content remains current.
- **Freshness audit run:** `freshness-20260920.txt` generated at 03:01 AWST (scheduled cron).
- **Skills index auto-regenerated:** `systems/skills-index.md` updated at 03:01 AWST (scheduled cron `1a8e90ea2271`).
- **Scheduled tasks reconciled:** `systems/scheduled-tasks.md` updated at 03:02 AWST — 26 active jobs verified from live `hermes cron list`.
- No secrets written.