Files
agent-estate-wiki/comparisons/model-routers.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

97 lines
6.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Model Routers
type: comparison
status: active
created: 2026-07-22
updated: 2026-11-11
verified_on: 2026-11-11
confidence: high
tags: [comparison, routing]
sources: [raw/configs/hermes-config-sanitized.txt]
---
# Model Routers
## Purpose
Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
## At a glance
| | LiteLLM | OmniRoute | OpenRouter |
|---|---|---|---|
| Role in Hermes | Fallback + a *custom_providers* entry | **Auxiliary + the pin on essentially every model-calling cron job** | Not configured as a standalone provider |
| Endpoint | `http://litellm:4000/v1` | `http://omniroute:20128/v1` (public: `omniroute.martinwa.org`) **and** `http://100.78.63.20:20128` (`omni-101`) | n/a |
| Model count (live, 2026-10-20) | **1869** (`/v1/models` with the configured key) | **6457** at `omniroute:20128/v1/models` | n/a |
| Model count (declared in `config.yaml`) | 1728 | 5398 (`Omniroute:20128`) / 5384 (`omni-101`) | n/a |
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
| Aux roles used for | — | `skills_hub`, `approval`, `mcp` | — |
| Config presence | `custom_providers` in `~/.hermes/config.yaml` | same (two separate entries) | absent from `custom_providers`; only referenced via a kept [[decisions/keep-openrouter-as-fallback]] decision |
> **The "LiteLLM is the default/primary" row was wrong by 2026-10-20 (verified and corrected).**
> The live `model:` block in `~/.hermes/config.yaml` is:
> ```yaml
> model:
> provider: nous
> default: meituan/longcat-2.5-preview:free
> base_url: https://inference-api.nousresearch.com/v1
> api_mode: chat_completions
> persist_switch_by_default: false
> ```
> So the **primary model path is Nous Research direct**, not LiteLLM. LiteLLM is the *last* entry
> in the fallback chain (`or-inclusionai_ling-2_6-1t` via `custom` → `http://litellm:4000/v1`).
>
> **The two OmniRoute endpoints matter.** `Omniroute:20128` is a DNS name (m93p CT101);
> `omni-101` is the raw IP `100.78.63.20:20128` (node `omniroute-1`) and is the alias pinned on
> effectively every model-calling cron job. The two carry **different catalogs** (5398 vs 5384
> declared models) and the `omni-101` entry's own `model:` is `smart-route` while
> `Omniroute:20128`'s is `go-cheap`.
>
> **Also: both OmniRoute endpoints and LiteLLM now require authentication.** `curl` without a key
> returns **401 `invalid_api_key`** on all three `/v1/models` endpoints. The keys live in
> `~/.hermes/.env` via each provider's `key_env` (e.g. `HERMES_CUSTOM_OMNIROUTE_20128_API_KEY`).
> Any runbook that probes these endpoints with a bare `curl` will see 401 and may wrongly conclude
> the provider is down — it is not.
> **Re-verified 2026-11-11 — the corrections recorded here all still hold.** The live `model:` block
> is OmniRoute-primary (`custom:omni-101` / `smart-route` / `http://100.78.63.20:20128`), and
> LiteLLM is the **tail** of the six-entry fallback chain (`or-inclusionai_ling-2_6-1t` via
> `custom` → `http://litellm:4000/v1`). The five `custom_providers` and their base URLs are exactly
> as listed: `litellm` `http://litellm:4000/v1`, `guacano` `http://ollama:11000/v1`,
> `Omniroute:20128` `http://omniroute:20128/v1`, `command-code`
> `https://api.commandcode.ai/provider/v1`, `omni-101` `http://100.78.63.20:20128`. OpenRouter is
> still absent from `custom_providers`. **Not re-probed this pass:** the live `/v1/models` counts
> (1869 / 6457) and their 2026-10-20 timestamps stand as recorded — treat them as unverified.
## Detail
### LiteLLM — fallback tail + a `custom_providers` entry
LiteLLM hosts the `freellmapi-auto` model alias and is reachable at `http://litellm:4000/v1`, but
it is **no longer the default chat path** — that is Nous direct
(`meituan/longcat-2.5-preview:free`, per the live `model:` block). LiteLLM appears in the
fallback chain as `or-inclusionai_ling-2_6-1t` via the `custom` provider.
See [[systems/model-providers]] and [[concepts/model-routing]].
### FreeLLMAPI — the router behind `freellmapi-auto`
Unified LLM router at `100.82.40.23:3001` (also `freellmapi.martinwa.org`). Verified 2026-08-12:
- **3-axis weighted scoring:** Reliability + Speed + Intelligence, with user-configurable weights.
- **Dynamic guardrail multiplier:** `Final score = weighted avg × (free-quota headroom × live rate-limit penalty)`. Below 1.0 means the model is held back.
- **Penalty Inspector ("Router Pressure"):** granular penalty factors (e.g. `score × 0.5`) from recent 4xx/5xx, cooldowns, RPM/RPD limits, monthly token caps.
- **Capability-aware routing:** filters candidates on whether the request has images (vision) or tool calls.
- Exposes `/api/fallback/routing`, `/api/fallback/penalty-inspector`, `/api/fallback/token-usage`.
This is the router that resolves Hermes's **default model** `freellmapi-auto` — distinct from OmniRoute's LKGP (below).
### OmniRoute — ⚠ main chat path again since 2026-10-24 (was "fallback / auxiliary")
**Re-verified 2026-10-26: this section is out of date.** Hermes's **default** is `custom:omni-101` /
`smart-route` on the `omniroute-1` node (`http://100.78.63.20:20128`) — i.e. OmniRoute **is** the main chat
path, as it was before the 2026-10-20 → 10-24 `nous` detour → [[systems/current-profile]].
It also still handles auxiliary roles and appears in the `fallback_providers` chain.` Has a much larger advertised catalog (~3430 vs ~1512) but that catalog can diverge from what's actually resolvable — combo discovery vs. resolution can differ (documented known issue). See [[systems/omniroute]].
### OpenRouter — kept only as a fallback decision, not wired in
**Verified:** absent from `custom_providers` in the live config. A decision page ([[decisions/keep-openrouter-as-fallback]]) records intent to retain it as a fallback option, but there's no live routing through it today — see [[systems/openrouter]] for the "previously used, largely replaced" history.
## Verdict
Two routers are live (**OmniRoute primary** for the default path and auxiliary slots since 2026-10-24 19:06; **LiteLLM** as the deeper fallback), OpenRouter is a retained *intent*, not a working path — re-verified 2026-10-26. If [[decisions/keep-openrouter-as-fallback]] is still desired, it needs an actual `custom_providers` entry — currently it's fallback-in-name-only.
## Related
- [[systems/model-providers]]
- [[concepts/model-routing]]