agent-estate-wiki/runbooks/test-openrouter.md

title, type, status, created, updated, verified_on, last_tested, confidence, tags, sources

title	type	status	created	updated	verified_on	last_tested	confidence	tags	sources
Test OpenRouter	runbook	active	2026-07-22	2026-07-22	2026-07-22	2026-07-22	high	runbook openrouter providers fallback	raw/configs/hermes-config-sanitized.txt

Test OpenRouter

Purpose

Test whether OpenRouter is available to Hermes and whether fallback providers are actually callable in this installation.

Symptoms that match this runbook

• Hermes reports fallback model errors
• hermes fallback list shows expected providers, but actual requests fail
• User wants to confirm OpenRouter is usable without relying on catalog names

Prerequisites

• Hermes CLI available: /home/hermes/.local/bin/hermes
• Internet egress for opencode.ai, inference-api.nousresearch.com, ollama.com, kilo.ai, litellm:4000

Procedure

1. List fallback providers

hermes fallback list

Expected active fallbacks on this install:

1. deepseek-v4-flash-free via opencode-zen
2. stepfun/step-3.7-flash:free via nous
3. qwen3.5:397b via ollama-cloud
4. poolside/laguna-m.1:free via kilocode
5. or-inclusionai_ling-2_6-1t via custom/litellm

2. Test OpenRouter-style direct access if present

curl -s -o /dev/null -w "%{http_code}" --max-time 8 https://openrouter.ai/api/v1/models

Current install state as of last run: OpenRouter is not configured as a standalone provider in Hermes, so this is an optional confirmation step only.

3. Test each verified fallback URL

for url in \
  "https://opencode.ai/zen/v1/models" \
  "https://inference-api.nousresearch.com/v1/models" \
  "https://ollama.com/v1/models" \
  "https://api.kilo.ai/api/gateway/models"; do
  printf "%s -> " "$url"
  curl -s -o /dev/null -w "%{http_code}" --max-time 8 "$url"
  echo
done

4. Check Hermes fallback config reflects reality

grep -A5 "^fallback:" ~/.hermes/config.yaml

Verify provider names and URLs in config match what the user wants to use.

Verification

• hermes fallback list shows the exact same providers as the live config
• Each URL returns an HTTP response within timeout
• No leftover undefined OpenRouter provider entries exist in custom_providers

Rollback

• No system changes in this runbook
• If a provider looks wrong but is part of live chat behavior, treat it like a config issue, not a transient health failure

Notes

• In this installation, OpenRouter is not a live first-class provider in custom_providers
• This runbook tests confirmed fallback paths, not assumed ones

Last tested

2026-07-22

Related

• provider-health
• model-providers