54 lines
3.2 KiB
Markdown
54 lines
3.2 KiB
Markdown
---
|
||
title: Model Routers
|
||
type: comparison
|
||
status: active
|
||
created: 2026-07-22
|
||
updated: 2026-08-12
|
||
verified_on: 2026-08-12
|
||
confidence: high
|
||
tags: [comparison, routing]
|
||
sources: [raw/configs/hermes-config-sanitized.txt]
|
||
---
|
||
|
||
# Model Routers
|
||
|
||
## Purpose
|
||
Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
|
||
|
||
## At a glance
|
||
| | LiteLLM | OmniRoute | OpenRouter |
|
||
|---|---|---|---|
|
||
| Role in Hermes | **Default/primary** provider | **Fallback/auxiliary** provider | Not configured as a standalone provider |
|
||
| Endpoint | `http://litellm:4000/v1` | `http://omniroute:20128/v1` (public: `omniroute.martinwa.org`) | n/a |
|
||
| Advertised model count | ~1512 | ~3430 | n/a |
|
||
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
|
||
| Aux roles used for | — | `skills_hub`, `approval`, `mcp` | — |
|
||
| Config presence | `custom_providers` in `~/.hermes/config.yaml` | same | absent from `custom_providers`; only referenced via a kept [[decisions/keep-openrouter-as-fallback]] decision |
|
||
|
||
## Detail
|
||
|
||
### LiteLLM — primary
|
||
Default provider for main chat traffic (`freellmapi-auto` default model resolves through it). See [[systems/model-providers]].
|
||
|
||
### FreeLLMAPI — the router behind `freellmapi-auto`
|
||
Unified LLM router at `100.82.40.23:3001` (also `freellmapi.martinwa.org`). Verified 2026-08-12:
|
||
- **3-axis weighted scoring:** Reliability + Speed + Intelligence, with user-configurable weights.
|
||
- **Dynamic guardrail multiplier:** `Final score = weighted avg × (free-quota headroom × live rate-limit penalty)`. Below 1.0 means the model is held back.
|
||
- **Penalty Inspector ("Router Pressure"):** granular penalty factors (e.g. `score × 0.5`) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps.
|
||
- **Capability-aware routing:** filters candidates on whether the request has images (vision) or tool calls.
|
||
- Exposes `/api/fallback/routing`, `/api/fallback/penalty-inspector`, `/api/fallback/token-usage`.
|
||
This is the router that resolves Hermes's **default model** `freellmapi-auto` — distinct from OmniRoute's LKGP (below).
|
||
|
||
### OmniRoute — fallback / auxiliary
|
||
Not used for Hermes's main chat path but handles specific auxiliary roles (`skills_hub`, `approval`, `mcp`) and serves as the fallback provider. Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what's actually resolvable — combo discovery vs. resolution can differ (documented known issue). See [[systems/omniroute]].
|
||
|
||
### OpenRouter — kept only as a fallback decision, not wired in
|
||
**Verified:** absent from `custom_providers` in the live config. A decision page ([[decisions/keep-openrouter-as-fallback]]) records intent to retain it as a fallback option, but there's no live routing through it today — see [[systems/openrouter]] for the "previously used, largely replaced" history.
|
||
|
||
## Verdict
|
||
Two routers are actually live (LiteLLM primary, OmniRoute fallback/auxiliary); OpenRouter is a retained *intent*, not a working path, as of 2026-07-22. If [[decisions/keep-openrouter-as-fallback]] is still desired, it needs an actual `custom_providers` entry — currently it's fallback-in-name-only.
|
||
|
||
## Related
|
||
- [[systems/model-providers]]
|
||
- [[concepts/model-routing]]
|