Model Routers
Purpose
Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
At a glance
| LiteLLM | OmniRoute | OpenRouter | |
|---|---|---|---|
| Role in Hermes | Default/primary provider | Fallback/auxiliary provider | Not configured as a standalone provider |
| Endpoint | http://litellm:4000/v1 | http://omniroute:20128/v1 (public: omniroute.martinwa.org) | n/a |
| Advertised model count | ~1512 | ~3430 | n/a |
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
| Aux roles used for | — | skills_hub, approval, mcp | — |
| Config presence | custom_providers in ~/.hermes/config.yaml | same | absent from custom_providers; only referenced via a kept keep-openrouter-as-fallback decision |
Detail
LiteLLM — primary
Default provider for main chat traffic (freellmapi-auto default model resolves through it). See model-providers.
FreeLLMAPI — the router behind freellmapi-auto
Unified LLM router at 100.82.40.23:3001 (also freellmapi.martinwa.org). Verified 2026-08-12:
- 3-axis weighted scoring: Reliability + Speed + Intelligence, with user-configurable weights.
- Dynamic guardrail multiplier:
Final score = weighted avg × (free-quota headroom × live rate-limit penalty). Below 1.0 means the model is held back. - Penalty Inspector (“Router Pressure”): granular penalty factors (e.g.
score × 0.5) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps. - Capability-aware routing: filters candidates on whether the request has images (vision) or tool calls.
- Exposes
/api/fallback/routing,/api/fallback/penalty-inspector,/api/fallback/token-usage. This is the router that resolves Hermes’s default modelfreellmapi-auto— distinct from OmniRoute’s LKGP (below).
OmniRoute — fallback / auxiliary
Not used for Hermes’s main chat path but handles specific auxiliary roles (skills_hub, approval, mcp) and serves as the fallback provider. Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what’s actually resolvable — combo discovery vs. resolution can differ (documented known issue). See omniroute.
OpenRouter — kept only as a fallback decision, not wired in
Verified: absent from custom_providers in the live config. A decision page (keep-openrouter-as-fallback) records intent to retain it as a fallback option, but there’s no live routing through it today — see openrouter for the “previously used, largely replaced” history.
Verdict
Two routers are actually live (LiteLLM primary, OmniRoute fallback/auxiliary); OpenRouter is a retained intent, not a working path, as of 2026-07-22. If keep-openrouter-as-fallback is still desired, it needs an actual custom_providers entry — currently it’s fallback-in-name-only.