snapshot: preserve central wiki state 2026-08-15 (7 modified + 1 new file)

2026-08-15 22:11:30 +08:00
commit 92e8028bb8
71 changed files with 9897 additions and 0 deletions
@@ -0,0 +1,103 @@
+---
+title: Provider Health Check
+type: runbook
+status: active
+created: 2026-07-22
+updated: 2026-07-22
+verified_on: 2026-07-22
+last_tested: 2026-07-22
+confidence: high
+tags: [runbook, providers, health, routing]
+sources: [raw/configs/hermes-config-sanitized.txt]
+---
+
+# Provider Health Check
+
+## Purpose
+Verify Hermes can reach LiteLLM, OmniRoute, OpenRouter-style fallbacks, and the WebUI, and report which paths are healthy or degraded.
+
+## Symptoms that match this runbook
+- Models list is slow or blank in Hermes
+- Fallback model retries happen immediately
+- Chat returns 5xx or gateway errors
+
+## Prerequisites
+- Network access from this host to `litellm:4000`, `omniroute:20128`, `opencode.ai`, `api.telegram.org`, Telegram fallback IPs
+- Hermes config slocated at `~/.hermes/config.yaml`
+
+## Procedure
+
+### 1. Check local Hermes services
+```bash
+hermes gateway status
+hermes dashboard status
+```
+
+### 2. Check main provider reachability
+```bash
+python3 -c "import urllib.request; urllib.request.urlopen('http://litellm:4000/v1/models', timeout=4)"
+```
+Current install behavior: LiteLLM responds but requires auth, so HTTP 401 is expected from an unauthenticated check.
+
+### 3. Check OmniRoute reachability
+```bash
+python3 -c "
+import urllib.request, json, yaml
+with open('/home/hermes/.hermes/config.yaml') as f:
+    cfg = yaml.safe_load(f)
+key = [p['api_key'] for p in cfg.get('custom_providers',[]) if p.get('name')=='omniroute'][0]
+req = urllib.request.Request('http://omniroute:20128/v1/models', headers={'Authorization': f'Bearer {key}'})
+with urllib.request.urlopen(req, timeout=5) as r:
+    data = json.loads(r.read())
+    print('OmniRoute OK, models:', len(data.get('data', [])))
+"
+```
+Current verified result: reachable and returns a non-empty model catalog.
+
+### 4. Check fallback provider endpoints
+```bash
+curl -s -o /dev/null -w "%{http_code}" --max-time 5 https://opencode.ai/zen/v1/models
+curl -s -o /dev/null -w "%{http_code}" --max-time 5 https://inference-api.nousresearch.com/v1/models
+curl -s -o /dev/null -w "%{http_code}" --max-time 5 https://ollama.com/v1/models
+```
+Any non-200 here means that fallback path is currently offline.
+
+### 5. Check WebUI health
+```bash
+curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:8787/health
+```
+Expected: `200`
+
+### 6. Check gateway platform reachability
+Tail the gateway journal for errors:
+```bash
+journalctl --user -u hermes-gateway.service -n 100 --no-pager
+```
+Look for repeated failures on Telegram IMAP, Telegram API, Slack Socket Mode, or Email fetch.
+
+## Verification
+- Gateway status shows `active (running)`.
+- LiteLLM 401 from unauthenticated check is expected.
+- OmniRoute returns model count > 0.
+- Fallback URLs return 200 or expected auth codes.
+- WebUI `/health` returns `200`.
+- Gateway journal shows no red alarm pattern.
+
+## Current install references
+- Hermes: v0.19.0, git install: `/home/hermes/.hermes/hermes-agent`
+- LiteLLM base URL: `http://litellm:4000/v1`
+- OmniRoute base URL: `http://omniroute:20128/v1`
+- OmniRoute direct IP: `100.88.81.19:20128`
+- WebUI: `http://0.0.0.0:8787/health`
+
+## Rollback
+- This read-only runbook has no rollback side effects
+- If you change fallback providers during remediation, record and revert via `hermes fallback list` + previous wiki state
+
+## Last tested
+2026-07-22
+
+## Related
+- [[model-providers]]
+- [[messaging-integrations]]
+- [[restart-hermes]]