wiki: publish pending edits (2026-10-20)
This commit is contained in:
@@ -3,8 +3,8 @@ title: Model Routers
|
||||
type: comparison
|
||||
status: active
|
||||
created: 2026-07-22
|
||||
updated: 2026-08-12
|
||||
verified_on: 2026-08-12
|
||||
updated: 2026-10-20
|
||||
verified_on: 2026-10-20
|
||||
confidence: high
|
||||
tags: [comparison, routing]
|
||||
sources: [raw/configs/hermes-config-sanitized.txt]
|
||||
@@ -18,17 +18,47 @@ Compare LiteLLM, OmniRoute, and OpenRouter as model-routing layers for Hermes.
|
||||
## At a glance
|
||||
| | LiteLLM | OmniRoute | OpenRouter |
|
||||
|---|---|---|---|
|
||||
| Role in Hermes | **Default/primary** provider | **Fallback/auxiliary** provider | Not configured as a standalone provider |
|
||||
| Endpoint | `http://litellm:4000/v1` | `http://omniroute:20128/v1` (public: `omniroute.martinwa.org`) | n/a |
|
||||
| Advertised model count | ~1512 | ~3430 | n/a |
|
||||
| Role in Hermes | Fallback + a *custom_providers* entry | **Auxiliary + the pin on essentially every model-calling cron job** | Not configured as a standalone provider |
|
||||
| Endpoint | `http://litellm:4000/v1` | `http://omniroute:20128/v1` (public: `omniroute.martinwa.org`) **and** `http://100.78.63.20:20128` (`omni-101`) | n/a |
|
||||
| Model count (live, 2026-10-20) | **1869** (`/v1/models` with the configured key) | **6457** at `omniroute:20128/v1/models` | n/a |
|
||||
| Model count (declared in `config.yaml`) | 1728 | 5398 (`Omniroute:20128`) / 5384 (`omni-101`) | n/a |
|
||||
| Routing style | Straight proxy | Combo strategy, LKGP scoring, session affinity | n/a |
|
||||
| Aux roles used for | — | `skills_hub`, `approval`, `mcp` | — |
|
||||
| Config presence | `custom_providers` in `~/.hermes/config.yaml` | same | absent from `custom_providers`; only referenced via a kept [[decisions/keep-openrouter-as-fallback]] decision |
|
||||
| Config presence | `custom_providers` in `~/.hermes/config.yaml` | same (two separate entries) | absent from `custom_providers`; only referenced via a kept [[decisions/keep-openrouter-as-fallback]] decision |
|
||||
|
||||
> **The "LiteLLM is the default/primary" row was wrong by 2026-10-20 (verified and corrected).**
|
||||
> The live `model:` block in `~/.hermes/config.yaml` is:
|
||||
> ```yaml
|
||||
> model:
|
||||
> provider: nous
|
||||
> default: meituan/longcat-2.5-preview:free
|
||||
> base_url: https://inference-api.nousresearch.com/v1
|
||||
> api_mode: chat_completions
|
||||
> persist_switch_by_default: false
|
||||
> ```
|
||||
> So the **primary model path is Nous Research direct**, not LiteLLM. LiteLLM is the *last* entry
|
||||
> in the fallback chain (`or-inclusionai_ling-2_6-1t` via `custom` → `http://litellm:4000/v1`).
|
||||
>
|
||||
> **The two OmniRoute endpoints matter.** `Omniroute:20128` is a DNS name (m93p CT101);
|
||||
> `omni-101` is the raw IP `100.78.63.20:20128` (node `omniroute-1`) and is the alias pinned on
|
||||
> effectively every model-calling cron job. The two carry **different catalogs** (5398 vs 5384
|
||||
> declared models) and the `omni-101` entry's own `model:` is `smart-route` while
|
||||
> `Omniroute:20128`'s is `go-cheap`.
|
||||
>
|
||||
> **Also: both OmniRoute endpoints and LiteLLM now require authentication.** `curl` without a key
|
||||
> returns **401 `invalid_api_key`** on all three `/v1/models` endpoints. The keys live in
|
||||
> `~/.hermes/.env` via each provider's `key_env` (e.g. `HERMES_CUSTOM_OMNIROUTE_20128_API_KEY`).
|
||||
> Any runbook that probes these endpoints with a bare `curl` will see 401 and may wrongly conclude
|
||||
> the provider is down — it is not.
|
||||
|
||||
## Detail
|
||||
|
||||
### LiteLLM — primary
|
||||
Default provider for main chat traffic (`freellmapi-auto` default model resolves through it). See [[systems/model-providers]].
|
||||
### LiteLLM — fallback tail + a `custom_providers` entry
|
||||
LiteLLM hosts the `freellmapi-auto` model alias and is reachable at `http://litellm:4000/v1`, but
|
||||
it is **no longer the default chat path** — that is Nous direct
|
||||
(`meituan/longcat-2.5-preview:free`, per the live `model:` block). LiteLLM appears in the
|
||||
fallback chain as `or-inclusionai_ling-2_6-1t` via the `custom` provider.
|
||||
See [[systems/model-providers]] and [[concepts/model-routing]].
|
||||
|
||||
### FreeLLMAPI — the router behind `freellmapi-auto`
|
||||
Unified LLM router at `100.82.40.23:3001` (also `freellmapi.martinwa.org`). Verified 2026-08-12:
|
||||
|
||||
Reference in New Issue
Block a user