- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13) - skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299) - runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated - comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest - incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised - incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused) - 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists) - log.md addendum + index.md date bump; no page deleted, no secrets written
97 lines
6.3 KiB
Markdown
97 lines
6.3 KiB
Markdown
---
|
||
title: Model Routers
|
||
type: comparison
|
||
status: active
|
||
created: 2026-07-22
|
||
updated: 2026-11-11
|
||
verified_on: 2026-11-11
|
||
confidence: high
|
||
tags: [comparison, routing]
|
||
sources: [raw/configs/hermes-config-sanitized.txt]
|
||
---
|
||
|
||
# Model Routers
|
||
|
||
## Purpose
|
||
Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
|
||
|
||
## At a glance
|
||
| | LiteLLM | OmniRoute | OpenRouter |
|
||
|---|---|---|---|
|
||
| Role in Hermes | Fallback + a *custom_providers* entry | **Auxiliary + the pin on essentially every model-calling cron job** | Not configured as a standalone provider |
|
||
| Endpoint | `http://litellm:4000/v1` | `http://omniroute:20128/v1` (public: `omniroute.martinwa.org`) **and** `http://100.78.63.20:20128` (`omni-101`) | n/a |
|
||
| Model count (live, 2026-10-20) | **1869** (`/v1/models` with the configured key) | **6457** at `omniroute:20128/v1/models` | n/a |
|
||
| Model count (declared in `config.yaml`) | 1728 | 5398 (`Omniroute:20128`) / 5384 (`omni-101`) | n/a |
|
||
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
|
||
| Aux roles used for | — | `skills_hub`, `approval`, `mcp` | — |
|
||
| Config presence | `custom_providers` in `~/.hermes/config.yaml` | same (two separate entries) | absent from `custom_providers`; only referenced via a kept [[decisions/keep-openrouter-as-fallback]] decision |
|
||
|
||
> **The "LiteLLM is the default/primary" row was wrong by 2026-10-20 (verified and corrected).**
|
||
> The live `model:` block in `~/.hermes/config.yaml` is:
|
||
> ```yaml
|
||
> model:
|
||
> provider: nous
|
||
> default: meituan/longcat-2.5-preview:free
|
||
> base_url: https://inference-api.nousresearch.com/v1
|
||
> api_mode: chat_completions
|
||
> persist_switch_by_default: false
|
||
> ```
|
||
> So the **primary model path is Nous Research direct**, not LiteLLM. LiteLLM is the *last* entry
|
||
> in the fallback chain (`or-inclusionai_ling-2_6-1t` via `custom` → `http://litellm:4000/v1`).
|
||
>
|
||
> **The two OmniRoute endpoints matter.** `Omniroute:20128` is a DNS name (m93p CT101);
|
||
> `omni-101` is the raw IP `100.78.63.20:20128` (node `omniroute-1`) and is the alias pinned on
|
||
> effectively every model-calling cron job. The two carry **different catalogs** (5398 vs 5384
|
||
> declared models) and the `omni-101` entry's own `model:` is `smart-route` while
|
||
> `Omniroute:20128`'s is `go-cheap`.
|
||
>
|
||
> **Also: both OmniRoute endpoints and LiteLLM now require authentication.** `curl` without a key
|
||
> returns **401 `invalid_api_key`** on all three `/v1/models` endpoints. The keys live in
|
||
> `~/.hermes/.env` via each provider's `key_env` (e.g. `HERMES_CUSTOM_OMNIROUTE_20128_API_KEY`).
|
||
> Any runbook that probes these endpoints with a bare `curl` will see 401 and may wrongly conclude
|
||
> the provider is down — it is not.
|
||
|
||
> **Re-verified 2026-11-11 — the corrections recorded here all still hold.** The live `model:` block
|
||
> is OmniRoute-primary (`custom:omni-101` / `smart-route` / `http://100.78.63.20:20128`), and
|
||
> LiteLLM is the **tail** of the six-entry fallback chain (`or-inclusionai_ling-2_6-1t` via
|
||
> `custom` → `http://litellm:4000/v1`). The five `custom_providers` and their base URLs are exactly
|
||
> as listed: `litellm` `http://litellm:4000/v1`, `guacano` `http://ollama:11000/v1`,
|
||
> `Omniroute:20128` `http://omniroute:20128/v1`, `command-code`
|
||
> `https://api.commandcode.ai/provider/v1`, `omni-101` `http://100.78.63.20:20128`. OpenRouter is
|
||
> still absent from `custom_providers`. **Not re-probed this pass:** the live `/v1/models` counts
|
||
> (1869 / 6457) and their 2026-10-20 timestamps stand as recorded — treat them as unverified.
|
||
|
||
## Detail
|
||
|
||
### LiteLLM — fallback tail + a `custom_providers` entry
|
||
LiteLLM hosts the `freellmapi-auto` model alias and is reachable at `http://litellm:4000/v1`, but
|
||
it is **no longer the default chat path** — that is Nous direct
|
||
(`meituan/longcat-2.5-preview:free`, per the live `model:` block). LiteLLM appears in the
|
||
fallback chain as `or-inclusionai_ling-2_6-1t` via the `custom` provider.
|
||
See [[systems/model-providers]] and [[concepts/model-routing]].
|
||
|
||
### FreeLLMAPI — the router behind `freellmapi-auto`
|
||
Unified LLM router at `100.82.40.23:3001` (also `freellmapi.martinwa.org`). Verified 2026-08-12:
|
||
- **3-axis weighted scoring:** Reliability + Speed + Intelligence, with user-configurable weights.
|
||
- **Dynamic guardrail multiplier:** `Final score = weighted avg × (free-quota headroom × live rate-limit penalty)`. Below 1.0 means the model is held back.
|
||
- **Penalty Inspector ("Router Pressure"):** granular penalty factors (e.g. `score × 0.5`) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps.
|
||
- **Capability-aware routing:** filters candidates on whether the request has images (vision) or tool calls.
|
||
- Exposes `/api/fallback/routing`, `/api/fallback/penalty-inspector`, `/api/fallback/token-usage`.
|
||
This is the router that resolves Hermes's **default model** `freellmapi-auto` — distinct from OmniRoute's LKGP (below).
|
||
|
||
### OmniRoute — ⚠ main chat path again since 2026-10-24 (was "fallback / auxiliary")
|
||
**Re-verified 2026-10-26: this section is out of date.** Hermes's **default** is `custom:omni-101` /
|
||
`smart-route` on the `omniroute-1` node (`http://100.78.63.20:20128`) — i.e. OmniRoute **is** the main chat
|
||
path, as it was before the 2026-10-20 → 10-24 `nous` detour → [[systems/current-profile]].
|
||
It also still handles auxiliary roles and appears in the `fallback_providers` chain.` Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what's actually resolvable — combo discovery vs. resolution can differ (documented known issue). See [[systems/omniroute]].
|
||
|
||
### OpenRouter — kept only as a fallback decision, not wired in
|
||
**Verified:** absent from `custom_providers` in the live config. A decision page ([[decisions/keep-openrouter-as-fallback]]) records intent to retain it as a fallback option, but there's no live routing through it today — see [[systems/openrouter]] for the "previously used, largely replaced" history.
|
||
|
||
## Verdict
|
||
Two routers are live (**OmniRoute primary** for the default path and auxiliary slots since 2026-10-24 19:06; **LiteLLM** as the deeper fallback), OpenRouter is a retained *intent*, not a working path — re-verified 2026-10-26. If [[decisions/keep-openrouter-as-fallback]] is still desired, it needs an actual `custom_providers` entry — currently it's fallback-in-name-only.
|
||
|
||
## Related
|
||
- [[systems/model-providers]]
|
||
- [[concepts/model-routing]]
|