Files
agent-estate-wiki/runbooks/test-openrouter.md
T

4.0 KiB

title, type, status, created, updated, verified_on, last_tested, confidence, tags, sources
title type status created updated verified_on last_tested confidence tags sources
Test OpenRouter runbook active 2026-07-22 2026-10-20 2026-10-20 2026-10-20 high
runbook
openrouter
providers
fallback
raw/configs/hermes-config-sanitized.txt

Test OpenRouter

Purpose

Test whether OpenRouter is available to Hermes and whether fallback providers are actually callable in this installation.

Symptoms that match this runbook

  • Hermes reports fallback model errors
  • hermes fallback list shows expected providers, but actual requests fail
  • User wants to confirm OpenRouter is usable without relying on catalog names

Prerequisites

  • Hermes CLI available: hermes on PATH (/home/hermes/.local/bin/hermes is a shim to the uv-managed runtime — see the note under Last tested)
  • Internet egress for opencode.ai, inference-api.nousresearch.com, ollama.com, kilo.ai, litellm:4000

Procedure

1. List fallback providers

hermes fallback list

Expected active fallbacks on this install (live hermes fallback list, 2026-10-20 — 6 entries):

  1. mimo-v2.5 via opencode-go
  2. deepseek-v4-flash-free via opencode-zen — https://opencode.ai/zen/v1
  3. stepfun/step-3.7-flash:free via nous — https://inference-api.nousresearch.com/v1
  4. qwen3.5:397b via ollama-cloud — https://ollama.com/v1
  5. poolside/laguna-m.1:free via kilocode — https://api.kilo.ai/api/gateway
  6. or-inclusionai_ling-2_6-1t via custom — http://litellm:4000/v1

Primary: meituan/longcat-2.5-preview:free (via nous).

Changes since 2026-07-22: mimo-v2.5 via opencode-go has been added at position 1 of the chain, and the primary changed to meituan/longcat-2.5-preview:free (was freellmapi-auto). The other five entries and their order are unchanged. hermes fallback list also prints a standing notice on every invocation: "Claude Subscription DirectSDK (Experimental): Claude Code is not installed" — expected on this box, since claude is not on PATH. It is a notice, not a failure of the fallback chain.

2. Test OpenRouter-style direct access if present

curl -s -o /dev/null -w "%{http_code}" --max-time 8 https://openrouter.ai/api/v1/models

Current install state as of last run: OpenRouter is not configured as a standalone provider in Hermes, so this is an optional confirmation step only.

3. Test each verified fallback URL

for url in \
  "https://opencode.ai/zen/v1/models" \
  "https://inference-api.nousresearch.com/v1/models" \
  "https://ollama.com/v1/models" \
  "https://api.kilo.ai/api/gateway/models"; do
  printf "%s -> " "$url"
  curl -s -o /dev/null -w "%{http_code}" --max-time 8 "$url"
  echo
done

4. Check Hermes fallback config reflects reality

grep -A5 "^fallback:" ~/.hermes/config.yaml

Verify provider names and URLs in config match what the user wants to use.

Verification

  • hermes fallback list shows the exact same providers as the live config
  • Each URL returns an HTTP response within timeout
  • No leftover undefined OpenRouter provider entries exist in custom_providers

Rollback

  • No system changes in this runbook
  • If a provider looks wrong but is part of live chat behavior, treat it like a config issue, not a transient health failure

Notes

  • In this installation, OpenRouter is not a live first-class provider in custom_providers
  • This runbook tests confirmed fallback paths, not assumed ones

Last tested

2026-07-22 — re-run 2026-10-20 against live hermes fallback list output (6 entries, primary meituan/longcat-2.5-preview:free). Note the CLI reports Hermes Agent v0.22.2 (2026.9.24), local d923a5eb487a, and the /home/hermes/.local/bin/hermes launcher is a 73-byte shim that delegates to the uv-managed runtime at /home/hermes/.hermes/tools/python-3.14.7+20260901-linux-x64/ — not the ~/.hermes/hermes-agent/venv/ path named in some older runbooks. Use the hermes on PATH.