Files
agent-estate-wiki/comparisons/model-routers.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

6.3 KiB
Raw Blame History

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Model Routers comparison active 2026-07-22 2026-11-11 2026-11-11 high
comparison
routing
raw/configs/hermes-config-sanitized.txt

Model Routers

Purpose

Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.

At a glance

LiteLLM OmniRoute OpenRouter
Role in Hermes Fallback + a custom_providers entry Auxiliary + the pin on essentially every model-calling cron job Not configured as a standalone provider
Endpoint http://litellm:4000/v1 http://omniroute:20128/v1 (public: omniroute.martinwa.org) and http://100.78.63.20:20128 (omni-101) n/a
Model count (live, 2026-10-20) 1869 (/v1/models with the configured key) 6457 at omniroute:20128/v1/models n/a
Model count (declared in config.yaml) 1728 5398 (Omniroute:20128) / 5384 (omni-101) n/a
Routing style Straight proxy Combo strategy, LKGP scoring, session affinity n/a
Aux roles used for — skills_hub, approval, mcp —
Config presence custom_providers in ~/.hermes/config.yaml same (two separate entries) absent from custom_providers; only referenced via a kept decisions/keep-openrouter-as-fallback decision

The "LiteLLM is the default/primary" row was wrong by 2026-10-20 (verified and corrected). The live model: block in ~/.hermes/config.yaml is:

model:
  provider: nous
  default: meituan/longcat-2.5-preview:free
  base_url: https://inference-api.nousresearch.com/v1
  api_mode: chat_completions
  persist_switch_by_default: false

So the primary model path is Nous Research direct, not LiteLLM. LiteLLM is the last entry in the fallback chain (or-inclusionai_ling-2_6-1t via custom → http://litellm:4000/v1).

The two OmniRoute endpoints matter. Omniroute:20128 is a DNS name (m93p CT101); omni-101 is the raw IP 100.78.63.20:20128 (node omniroute-1) and is the alias pinned on effectively every model-calling cron job. The two carry different catalogs (5398 vs 5384 declared models) and the omni-101 entry's own model: is smart-route while Omniroute:20128's is go-cheap.

Also: both OmniRoute endpoints and LiteLLM now require authentication. curl without a key returns 401 invalid_api_key on all three /v1/models endpoints. The keys live in ~/.hermes/.env via each provider's key_env (e.g. HERMES_CUSTOM_OMNIROUTE_20128_API_KEY). Any runbook that probes these endpoints with a bare curl will see 401 and may wrongly conclude the provider is down — it is not.

Re-verified 2026-11-11 — the corrections recorded here all still hold. The live model: block is OmniRoute-primary (custom:omni-101 / smart-route / http://100.78.63.20:20128), and LiteLLM is the tail of the six-entry fallback chain (or-inclusionai_ling-2_6-1t via custom → http://litellm:4000/v1). The five custom_providers and their base URLs are exactly as listed: litellm http://litellm:4000/v1, guacano http://ollama:11000/v1, Omniroute:20128 http://omniroute:20128/v1, command-code https://api.commandcode.ai/provider/v1, omni-101 http://100.78.63.20:20128. OpenRouter is still absent from custom_providers. Not re-probed this pass: the live /v1/models counts (1869 / 6457) and their 2026-10-20 timestamps stand as recorded — treat them as unverified.

Detail

LiteLLM — fallback tail + a custom_providers entry

LiteLLM hosts the freellmapi-auto model alias and is reachable at http://litellm:4000/v1, but it is no longer the default chat path — that is Nous direct (meituan/longcat-2.5-preview:free, per the live model: block). LiteLLM appears in the fallback chain as or-inclusionai_ling-2_6-1t via the custom provider. See systems/model-providers and concepts/model-routing.

FreeLLMAPI — the router behind freellmapi-auto

Unified LLM router at 100.82.40.23:3001 (also freellmapi.martinwa.org). Verified 2026-08-12:

  • 3-axis weighted scoring: Reliability + Speed + Intelligence, with user-configurable weights.
  • Dynamic guardrail multiplier: Final score = weighted avg × (free-quota headroom × live rate-limit penalty). Below 1.0 means the model is held back.
  • Penalty Inspector ("Router Pressure"): granular penalty factors (e.g. score × 0.5) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps.
  • Capability-aware routing: filters candidates on whether the request has images (vision) or tool calls.
  • Exposes /api/fallback/routing, /api/fallback/penalty-inspector, /api/fallback/token-usage. This is the router that resolves Hermes's default model freellmapi-auto — distinct from OmniRoute's LKGP (below).

OmniRoute — ⚠ main chat path again since 2026-10-24 (was "fallback / auxiliary")

Re-verified 2026-10-26: this section is out of date. Hermes's default is custom:omni-101 / smart-route on the omniroute-1 node (http://100.78.63.20:20128) — i.e. OmniRoute is the main chat path, as it was before the 2026-10-20 → 10-24 nous detour → systems/current-profile. It also still handles auxiliary roles and appears in the fallback_providers chain.` Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what's actually resolvable — combo discovery vs. resolution can differ (documented known issue). See systems/omniroute.

OpenRouter — kept only as a fallback decision, not wired in

Verified: absent from custom_providers in the live config. A decision page (decisions/keep-openrouter-as-fallback) records intent to retain it as a fallback option, but there's no live routing through it today — see systems/openrouter for the "previously used, largely replaced" history.

Verdict

Two routers are live (OmniRoute primary for the default path and auxiliary slots since 2026-10-24 19:06; LiteLLM as the deeper fallback), OpenRouter is a retained intent, not a working path — re-verified 2026-10-26. If decisions/keep-openrouter-as-fallback is still desired, it needs an actual custom_providers entry — currently it's fallback-in-name-only.