Files
agent-estate-wiki/runbooks/test-openrouter.md
T

101 lines
4.0 KiB
Markdown

---
title: Test OpenRouter
type: runbook
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
last_tested: 2026-10-20
confidence: high
tags: [runbook, openrouter, providers, fallback]
sources: [raw/configs/hermes-config-sanitized.txt]
---
# Test OpenRouter
## Purpose
Test whether OpenRouter is available to Hermes and whether fallback providers are actually callable in this installation.
## Symptoms that match this runbook
- Hermes reports fallback model errors
- `hermes fallback list` shows expected providers, but actual requests fail
- User wants to confirm OpenRouter is usable without relying on catalog names
## Prerequisites
- Hermes CLI available: `hermes` on PATH (`/home/hermes/.local/bin/hermes` is a shim to the
uv-managed runtime — see the note under **Last tested**)
- Internet egress for opencode.ai, inference-api.nousresearch.com, ollama.com, kilo.ai, litellm:4000
## Procedure
### 1. List fallback providers
```bash
hermes fallback list
```
Expected active fallbacks on this install (live `hermes fallback list`, 2026-10-20 — **6** entries):
1. `mimo-v2.5` via opencode-go
2. `deepseek-v4-flash-free` via opencode-zen — `https://opencode.ai/zen/v1`
3. `stepfun/step-3.7-flash:free` via nous — `https://inference-api.nousresearch.com/v1`
4. `qwen3.5:397b` via ollama-cloud — `https://ollama.com/v1`
5. `poolside/laguna-m.1:free` via kilocode — `https://api.kilo.ai/api/gateway`
6. `or-inclusionai_ling-2_6-1t` via custom — `http://litellm:4000/v1`
**Primary:** `meituan/longcat-2.5-preview:free` (via nous).
> **Changes since 2026-07-22:** `mimo-v2.5` via **opencode-go** has been **added at position 1** of the
> chain, and the primary changed to `meituan/longcat-2.5-preview:free` (was `freellmapi-auto`).
> The other five entries and their order are unchanged.
> `hermes fallback list` also prints a standing notice on every invocation:
> *"Claude Subscription DirectSDK (Experimental): Claude Code is not installed"* — expected on this
> box, since `claude` is not on PATH. It is a notice, not a failure of the fallback chain.
### 2. Test OpenRouter-style direct access if present
```bash
curl -s -o /dev/null -w "%{http_code}" --max-time 8 https://openrouter.ai/api/v1/models
```
Current install state as of last run: OpenRouter is not configured as a standalone provider in Hermes, so this is an optional confirmation step only.
### 3. Test each verified fallback URL
```bash
for url in \
"https://opencode.ai/zen/v1/models" \
"https://inference-api.nousresearch.com/v1/models" \
"https://ollama.com/v1/models" \
"https://api.kilo.ai/api/gateway/models"; do
printf "%s -> " "$url"
curl -s -o /dev/null -w "%{http_code}" --max-time 8 "$url"
echo
done
```
### 4. Check Hermes fallback config reflects reality
```bash
grep -A5 "^fallback:" ~/.hermes/config.yaml
```
Verify provider names and URLs in config match what the user wants to use.
## Verification
- `hermes fallback list` shows the exact same providers as the live config
- Each URL returns an HTTP response within timeout
- No leftover undefined OpenRouter provider entries exist in `custom_providers`
## Rollback
- No system changes in this runbook
- If a provider looks wrong but is part of live chat behavior, treat it like a config issue, not a transient health failure
## Notes
- In this installation, OpenRouter is not a live first-class provider in `custom_providers`
- This runbook tests confirmed fallback paths, not assumed ones
## Last tested
2026-07-22 — re-run 2026-10-20 against live `hermes fallback list` output (6 entries, primary
`meituan/longcat-2.5-preview:free`). Note the CLI reports Hermes Agent **v0.22.2 (2026.9.24),
local `d923a5eb487a`**, and the `/home/hermes/.local/bin/hermes` launcher is a 73-byte shim that
delegates to the uv-managed runtime at
`/home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/` — *not* the
`~/.hermes/hermes-agent/venv/` path named in some older runbooks. Use the `hermes` on PATH.
## Related
- [[provider-health]]
- [[model-providers]]