Files
agent-estate-wiki/runbooks/test-openrouter.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

125 lines
5.5 KiB
Markdown

---
title: Test OpenRouter
type: runbook
status: active
created: 2026-07-22
updated: 2026-11-11
verified_on: 2026-11-11
last_tested: 2026-10-20
confidence: high
tags: [runbook, openrouter, providers, fallback]
sources: [raw/configs/hermes-config-sanitized.txt]
---
# Test OpenRouter
## Purpose
Test whether OpenRouter is available to Hermes and whether fallback providers are actually callable in this installation.
## Symptoms that match this runbook
- Hermes reports fallback model errors
- `hermes fallback list` shows expected providers, but actual requests fail
- User wants to confirm OpenRouter is usable without relying on catalog names
## Prerequisites
- Hermes CLI available: `hermes` on PATH (`/home/hermes/.local/bin/hermes` is a shim to the
uv-managed runtime — see the note under **Last tested**)
- Internet egress for opencode.ai, inference-api.nousresearch.com, kilo.ai (ollama.com is no
longer in the live chain — entry 3 uses the internal `guacano` endpoint instead)
## Procedure
### 1. List fallback providers
```bash
hermes fallback list
```
Expected active fallbacks on this install (live `hermes fallback list`, 2026-11-11 — **6** entries):
1. `meituan/longcat-2.5-preview:free` via nous
2. `deepseek-v4.1-flash` via opencode-zen — `https://opencode.ai/zen/v1`
3. `deepseek-v4.1-flash` via custom:guacano — `http://ollama:11000/v1`
4. `kilo-reflection-beam-free` via custom:litellm — `http://litellm:4000/v1`
5. `poolside/laguna-m.1:free` via kilocode — `https://api.kilo.ai/api/gateway`
6. `or-inclusionai_ling-2_6-1t` via custom — `http://litellm:4000/v1`
**Primary:** `smart-route` (via `custom:omni-101`, `http://100.78.63.20:20128`).
> **Chain rewritten 2026-11-11 — the 2026-10-20 list recorded above was materially stale.**
> Live `hermes fallback list` now reads (verbatim):
>
> ```
> Primary: smart-route (via custom:omni-101)
> Fallback chain (6 entries):
> 1. meituan/longcat-2.5-preview:free (via nous)
> 2. deepseek-v4.1-flash (via opencode-zen) [https://opencode.ai/zen/v1]
> 3. deepseek-v4.1-flash (via custom:guacano)
> 4. kilo-reflection-beam-free (via custom:litellm)
> 5. poolside/laguna-m.1:free (via kilocode) [https://api.kilo.ai/api/gateway]
> 6. or-inclusionai_ling-2_6-1t (via custom) [http://litellm:4000/v1]
> ```
>
> What changed since the 2026-10-20 reading: the **primary moved back to OmniRoute**
> (`smart-route` via `custom:omni-101`, matching the live `model:` block — see
> [[concepts/model-routing]]); `mimo-v2.5` via opencode-go and `stepfun/step-3.7-flash:free` via
> nous have **left** the chain; `qwen3.5:397b` via ollama-cloud has **left** (so `ollama.com` no
> longer needs probing); `deepseek-v4.1-flash` via `custom:guacano` and `kilo-reflection-beam-free`
> via `custom:litellm` have **joined** at positions 3 and 4; positions 5 and 6 are unchanged.
> The standing *"Claude Subscription DirectSDK … Claude Code is not installed"* notice recorded on
> 2026-10-20 **did not print** on the 2026-11-11 run.
### 2. Test OpenRouter-style direct access if present
```bash
curl -s -o /dev/null -w "%{http_code}" --max-time 8 https://openrouter.ai/api/v1/models
```
Current install state as of last run: OpenRouter is not configured as a standalone provider in Hermes, so this is an optional confirmation step only.
### 3. Test each verified fallback URL
```bash
for url in \
"https://opencode.ai/zen/v1/models" \
"https://inference-api.nousresearch.com/v1/models" \
"https://api.kilo.ai/api/gateway/models"; do
printf "%s -> " "$url"
curl -s -o /dev/null -w "%{http_code}" --max-time 8 "$url"
echo
done
```
Entries 3 (`custom:guacano` → `http://ollama:11000/v1`) and 6 (`custom` → `http://litellm:4000/v1`)
are **internal to the estate** and both now **require authentication**: a bare `curl` returns
**401 `invalid_api_key`**. Their keys live in `~/.hermes/.env` behind each provider's `key_env`
(e.g. `HERMES_CUSTOM_LITELLM_API_KEY`). `guacano` resolves through CT205 over the tailnet/DNS name
`ollama`, so it is only reachable from inside the estate — a 401 there proves auth reached the
node, not that it is down (see [[runbooks/provider-health]]).
### 4. Check Hermes fallback config reflects reality
```bash
grep -A5 "^fallback:" ~/.hermes/config.yaml
```
Verify provider names and URLs in config match what the user wants to use.
## Verification
- `hermes fallback list` shows the exact same providers as the live config
- Each URL returns an HTTP response within timeout
- No leftover undefined OpenRouter provider entries exist in `custom_providers`
## Rollback
- No system changes in this runbook
- If a provider looks wrong but is part of live chat behavior, treat it like a config issue, not a transient health failure
## Notes
- In this installation, OpenRouter is not a live first-class provider in `custom_providers`
- This runbook tests confirmed fallback paths, not assumed ones
## Last tested
2026-07-22 — re-run 2026-10-20 against live `hermes fallback list` output (6 entries, primary
`meituan/longcat-2.5-preview:free`), and again on **2026-11-11** against the live output recorded
above (6 entries, primary `smart-route` via `custom:omni-101`). Note the CLI reports Hermes Agent **v0.22.2 (2026.9.24),
local `d923a5eb487a`**, and the `/home/hermes/.local/bin/hermes` launcher is a 73-byte shim that
delegates to the uv-managed runtime at
`/home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/` — *not* the
`~/.hermes/hermes-agent/venv/` path named in some older runbooks. Use the `hermes` on PATH.
## Related
- [[provider-health]]
- [[model-providers]]