Files
agent-estate-wiki/incidents/2026-08-15-skills-index-drift.md
T

5.5 KiB

title, type, status, created, updated, resolved_on, reopened_on, verified_on, confidence, tags, sources
title type status created updated resolved_on reopened_on verified_on confidence tags sources
Skills Index Drift — 4x Undercount incident active 2026-08-15 2026-10-23 2026-08-15 2026-10-23 2026-10-23 high
skills
wiki
drift
auto-regeneration
session-investigation
agent-estate-wiki
live-filesystem-probe
cron-executions-db

Skills Index Drift — 4x Undercount

Summary

The systems/skills-index.md page in both wikis claimed 66 skills when 330 were actually installed — a 4x undercount. The page had been hand-written once on 2026-07-22/25 and never regenerated as the estate grew.

Root Cause

Static documentation for a mutable inventory. The skills tree at ~/.hermes/skills/ grew from ~66 to 330 skills over three weeks, but the wiki page was never updated because there was no automation to keep it in sync.

Impact

  • Wiki page was 4x out of date
  • Any inventory-based decisions (skill loading, capacity planning) would have been based on wrong numbers
  • No functional impact — skills still loaded correctly via Tool Search

Resolution

  1. Regenerated systems/skills-index.md from the live skill tree (all 330 skills, 26 categories)
  2. Created a regeneration script for automated regeneration — superseded on 2026-10-21: the regeneration now lives inside wiki-maintain.sh (step 1/3) in the wiki repo; no standalone regenerate-skills-index.sh exists on disk any more.
  3. Set up cron job 1a8e90ea2271 to run daily at 03:00 AWST
  4. Script auto-commits and pushes to Gitea (Anthony/agent-estate-wiki)
  5. Quartz site at http://100.118.5.51:9120/systems/skills-index rebuilt and redeployed

Lesson

Static wiki pages that reflect mutable inventory (skills, services, cron jobs) MUST be auto-regenerated on a schedule, not hand-maintained. The moment the estate grows, a hand-written page becomes stale.

Re-verified 2026-10-21

The prevention mechanism is live and working. wiki-maintain.sh step 1/3 regenerated systems/skills-index.md from ~/.hermes/skills/ on the 2026-10-21 03:00 scheduled run: 347 skills across 70 categories (the "26 categories" figure in the original write-up was the count at the time of the incident, not the current topology — the skill tree has since nested deeper). The daily cron job 1a8e90ea2271 is active with last_status: ok.

Prevention

  • Daily cron job regenerates skills-index.md from live tree
  • Page includes "auto-regenerated, do not hand-edit" warning
  • Same script pattern is reusable for other inventory pages (cron jobs, services, agents)

Reopened 2026-10-23 — the count includes archived skills, and two generators disagree

The 2026-10-21 all-clear above was premature. Two defects were confirmed on the night of 2026-10-22/23:

1. The stocktake ran six times long and committed nothing. Cron job 1a8e90ea2271 ran 03:00:28 → 04:38:19 AWST on 2026-10-22 (97 m 58 s), against 16 m 01 s on 2026-10-21, and finished completed with delivery suppressed — but git status in the wiki repo still shows the run's regenerated systems/skills-index.md (mtime 10-22 03:00) uncommitted, and none of the fixes the run had diagnosed were applied. A completed row is therefore not evidence that the stocktake did its job.

2. Two scripts generate the same page with different shapes.

Generator Skill scan 2026-10-22 output
/home/hermes/workspace/agent-estate-wiki/wiki-maintain.sh (the one the cron job executes) find … -name SKILL.md — does not prune dot-directories 347 skills, with a ## .Archive (41) section
/home/hermes/.hermes/scripts/wiki-maintain.sh (stale copy, mtime 2026-08-17) Path.rglob('SKILL.md') 348 skills, no archive section

The 00:18 maintenance step committed the rglob shape; the 03:00 stocktake wrote the find shape. Whoever runs last wins, so the page oscillates.

Live counts, verified 2026-10-23 00:1x AWST (filesystem probes + hermes skills list):

  • find ~/.hermes/skills -name SKILL.md → 348
  • find -L ~/.hermes/skills -name SKILL.md → 376 (28 more skills reachable only through symlinks)
  • excluding .archive/ (41) and apple/ (4) → 331
  • the loader's own catalogue — hermes skills list → 329 enabled (34 hub-installed, 52 builtin, 243 local — 329 enabled, 0 disabled)

So the page total counts 45 artefacts the loader does not expose (the parked .archive/ set plus apple/), which is why its number has never matched the loader.

Fix applied 2026-10-23: the repo's wiki-maintain.sh now excludes */.archive/* from the scan, prints the archived count as an explicit exclusion, and states that hermes skills list is the authoritative loadable figure while the page remains a filesystem inventory. The page was regenerated through the patched script and committed with the fix.

Still open:

  • The stale ~/.hermes/scripts/wiki-maintain.sh copy must never be run — it produces the divergent shape. It should be deleted or made to delegate to the repo script.
  • The page will still read higher than the loader's 329: nested collections (context-engineering-collection/…), apple/ and hub-installed trees are counted as installed files. Whether the page should instead report the loader's figure is a decision for Anthony.
  • Watch the 2026-10-23 03:00 stocktake run: it should complete in ~16 minutes and commit the regenerated page.