5.5 KiB
title, type, status, created, updated, verified_on, confidence, tags, sources
| title | type | status | created | updated | verified_on | confidence | tags | sources | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model Routers | comparison | active | 2026-07-22 | 2026-10-20 | 2026-10-20 | high |
|
|
Model Routers
Purpose
Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
At a glance
| LiteLLM | OmniRoute | OpenRouter | |
|---|---|---|---|
| Role in Hermes | Fallback + a custom_providers entry | Auxiliary + the pin on essentially every model-calling cron job | Not configured as a standalone provider |
| Endpoint | http://litellm:4000/v1 |
http://omniroute:20128/v1 (public: omniroute.martinwa.org) and http://100.78.63.20:20128 (omni-101) |
n/a |
| Model count (live, 2026-10-20) | 1869 (/v1/models with the configured key) |
6457 at omniroute:20128/v1/models |
n/a |
Model count (declared in config.yaml) |
1728 | 5398 (Omniroute:20128) / 5384 (omni-101) |
n/a |
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
| Aux roles used for | — | skills_hub, approval, mcp |
— |
| Config presence | custom_providers in ~/.hermes/config.yaml |
same (two separate entries) | absent from custom_providers; only referenced via a kept decisions/keep-openrouter-as-fallback decision |
The "LiteLLM is the default/primary" row was wrong by 2026-10-20 (verified and corrected). The live
model:block in~/.hermes/config.yamlis:model: provider: nous default: meituan/longcat-2.5-preview:free base_url: https://inference-api.nousresearch.com/v1 api_mode: chat_completions persist_switch_by_default: falseSo the primary model path is Nous Research direct, not LiteLLM. LiteLLM is the last entry in the fallback chain (
or-inclusionai_ling-2_6-1tviacustom→http://litellm:4000/v1).The two OmniRoute endpoints matter.
Omniroute:20128is a DNS name (m93p CT101);omni-101is the raw IP100.78.63.20:20128(nodeomniroute-1) and is the alias pinned on effectively every model-calling cron job. The two carry different catalogs (5398 vs 5384 declared models) and theomni-101entry's ownmodel:issmart-routewhileOmniroute:20128's isgo-cheap.Also: both OmniRoute endpoints and LiteLLM now require authentication.
curlwithout a key returns 401invalid_api_keyon all three/v1/modelsendpoints. The keys live in~/.hermes/.envvia each provider'skey_env(e.g.HERMES_CUSTOM_OMNIROUTE_20128_API_KEY). Any runbook that probes these endpoints with a barecurlwill see 401 and may wrongly conclude the provider is down — it is not.
Detail
LiteLLM — fallback tail + a custom_providers entry
LiteLLM hosts the freellmapi-auto model alias and is reachable at http://litellm:4000/v1, but
it is no longer the default chat path — that is Nous direct
(meituan/longcat-2.5-preview:free, per the live model: block). LiteLLM appears in the
fallback chain as or-inclusionai_ling-2_6-1t via the custom provider.
See systems/model-providers and concepts/model-routing.
FreeLLMAPI — the router behind freellmapi-auto
Unified LLM router at 100.82.40.23:3001 (also freellmapi.martinwa.org). Verified 2026-08-12:
- 3-axis weighted scoring: Reliability + Speed + Intelligence, with user-configurable weights.
- Dynamic guardrail multiplier:
Final score = weighted avg × (free-quota headroom × live rate-limit penalty). Below 1.0 means the model is held back. - Penalty Inspector ("Router Pressure"): granular penalty factors (e.g.
score × 0.5) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps. - Capability-aware routing: filters candidates on whether the request has images (vision) or tool calls.
- Exposes
/api/fallback/routing,/api/fallback/penalty-inspector,/api/fallback/token-usage. This is the router that resolves Hermes's default modelfreellmapi-auto— distinct from OmniRoute's LKGP (below).
OmniRoute — ⚠ main chat path again since 2026-10-24 (was "fallback / auxiliary")
Re-verified 2026-10-26: this section is out of date. Hermes's default is custom:omni-101 /
smart-route on the omniroute-1 node (http://100.78.63.20:20128) — i.e. OmniRoute is the main chat
path, as it was before the 2026-10-20 → 10-24 nous detour → systems/current-profile.
It also still handles auxiliary roles and appears in the fallback_providers chain.` Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what's actually resolvable — combo discovery vs. resolution can differ (documented known issue). See systems/omniroute.
OpenRouter — kept only as a fallback decision, not wired in
Verified: absent from custom_providers in the live config. A decision page (decisions/keep-openrouter-as-fallback) records intent to retain it as a fallback option, but there's no live routing through it today — see systems/openrouter for the "previously used, largely replaced" history.
Verdict
Two routers are live (OmniRoute primary for the default path and auxiliary slots since 2026-10-24 19:06; LiteLLM as the deeper fallback), OpenRouter is a retained intent, not a working path — re-verified 2026-10-26. If decisions/keep-openrouter-as-fallback is still desired, it needs an actual custom_providers entry — currently it's fallback-in-name-only.