--- title: Test OpenRouter type: runbook status: active created: 2026-07-22 updated: 2026-11-11 verified_on: 2026-11-11 last_tested: 2026-10-20 confidence: high tags: [runbook, openrouter, providers, fallback] sources: [raw/configs/hermes-config-sanitized.txt] --- # Test OpenRouter ## Purpose Test whether OpenRouter is available to Hermes and whether fallback providers are actually callable in this installation. ## Symptoms that match this runbook - Hermes reports fallback model errors - `hermes fallback list` shows expected providers, but actual requests fail - User wants to confirm OpenRouter is usable without relying on catalog names ## Prerequisites - Hermes CLI available: `hermes` on PATH (`/home/hermes/.local/bin/hermes` is a shim to the uv-managed runtime — see the note under **Last tested**) - Internet egress for opencode.ai, inference-api.nousresearch.com, kilo.ai (ollama.com is no longer in the live chain — entry 3 uses the internal `guacano` endpoint instead) ## Procedure ### 1. List fallback providers ```bash hermes fallback list ``` Expected active fallbacks on this install (live `hermes fallback list`, 2026-11-11 — **6** entries): 1. `meituan/longcat-2.5-preview:free` via nous 2. `deepseek-v4.1-flash` via opencode-zen — `https://opencode.ai/zen/v1` 3. `deepseek-v4.1-flash` via custom:guacano — `http://ollama:11000/v1` 4. `kilo-reflection-beam-free` via custom:litellm — `http://litellm:4000/v1` 5. `poolside/laguna-m.1:free` via kilocode — `https://api.kilo.ai/api/gateway` 6. `or-inclusionai_ling-2_6-1t` via custom — `http://litellm:4000/v1` **Primary:** `smart-route` (via `custom:omni-101`, `http://100.78.63.20:20128`). > **Chain rewritten 2026-11-11 — the 2026-10-20 list recorded above was materially stale.** > Live `hermes fallback list` now reads (verbatim): > > ``` > Primary: smart-route (via custom:omni-101) > Fallback chain (6 entries): > 1. meituan/longcat-2.5-preview:free (via nous) > 2. deepseek-v4.1-flash (via opencode-zen) [https://opencode.ai/zen/v1] > 3. deepseek-v4.1-flash (via custom:guacano) > 4. kilo-reflection-beam-free (via custom:litellm) > 5. poolside/laguna-m.1:free (via kilocode) [https://api.kilo.ai/api/gateway] > 6. or-inclusionai_ling-2_6-1t (via custom) [http://litellm:4000/v1] > ``` > > What changed since the 2026-10-20 reading: the **primary moved back to OmniRoute** > (`smart-route` via `custom:omni-101`, matching the live `model:` block — see > [[concepts/model-routing]]); `mimo-v2.5` via opencode-go and `stepfun/step-3.7-flash:free` via > nous have **left** the chain; `qwen3.5:397b` via ollama-cloud has **left** (so `ollama.com` no > longer needs probing); `deepseek-v4.1-flash` via `custom:guacano` and `kilo-reflection-beam-free` > via `custom:litellm` have **joined** at positions 3 and 4; positions 5 and 6 are unchanged. > The standing *"Claude Subscription DirectSDK … Claude Code is not installed"* notice recorded on > 2026-10-20 **did not print** on the 2026-11-11 run. ### 2. Test OpenRouter-style direct access if present ```bash curl -s -o /dev/null -w "%{http_code}" --max-time 8 https://openrouter.ai/api/v1/models ``` Current install state as of last run: OpenRouter is not configured as a standalone provider in Hermes, so this is an optional confirmation step only. ### 3. Test each verified fallback URL ```bash for url in \ "https://opencode.ai/zen/v1/models" \ "https://inference-api.nousresearch.com/v1/models" \ "https://api.kilo.ai/api/gateway/models"; do printf "%s -> " "$url" curl -s -o /dev/null -w "%{http_code}" --max-time 8 "$url" echo done ``` Entries 3 (`custom:guacano` → `http://ollama:11000/v1`) and 6 (`custom` → `http://litellm:4000/v1`) are **internal to the estate** and both now **require authentication**: a bare `curl` returns **401 `invalid_api_key`**. Their keys live in `~/.hermes/.env` behind each provider's `key_env` (e.g. `HERMES_CUSTOM_LITELLM_API_KEY`). `guacano` resolves through CT205 over the tailnet/DNS name `ollama`, so it is only reachable from inside the estate — a 401 there proves auth reached the node, not that it is down (see [[runbooks/provider-health]]). ### 4. Check Hermes fallback config reflects reality ```bash grep -A5 "^fallback:" ~/.hermes/config.yaml ``` Verify provider names and URLs in config match what the user wants to use. ## Verification - `hermes fallback list` shows the exact same providers as the live config - Each URL returns an HTTP response within timeout - No leftover undefined OpenRouter provider entries exist in `custom_providers` ## Rollback - No system changes in this runbook - If a provider looks wrong but is part of live chat behavior, treat it like a config issue, not a transient health failure ## Notes - In this installation, OpenRouter is not a live first-class provider in `custom_providers` - This runbook tests confirmed fallback paths, not assumed ones ## Last tested 2026-07-22 — re-run 2026-10-20 against live `hermes fallback list` output (6 entries, primary `meituan/longcat-2.5-preview:free`), and again on **2026-11-11** against the live output recorded above (6 entries, primary `smart-route` via `custom:omni-101`). Note the CLI reports Hermes Agent **v0.22.2 (2026.9.24), local `d923a5eb487a`**, and the `/home/hermes/.local/bin/hermes` launcher is a 73-byte shim that delegates to the uv-managed runtime at `/home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/` — *not* the `~/.hermes/hermes-agent/venv/` path named in some older runbooks. Use the `hermes` on PATH. ## Related - [[provider-health]] - [[model-providers]]