73 lines
3.4 KiB
Markdown
73 lines
3.4 KiB
Markdown
---
|
|
title: Model Routing
|
|
type: concept
|
|
status: active
|
|
created: 2026-07-22
|
|
updated: 2026-10-20
|
|
verified_on: 2026-10-20
|
|
confidence: high
|
|
tags: [routing, model, inference]
|
|
sources: [raw/configs/hermes-config-sanitized.txt, current-profile.md]
|
|
---
|
|
|
|
# Model Routing
|
|
|
|
## Purpose
|
|
How Hermes selects providers, applies combos, and enforces context limits.
|
|
|
|
## Current Flow
|
|
- Main path: `custom:litellm` → `http://litellm:4000/v1`
|
|
- Default model: `freellmapi-auto`
|
|
- OmniRoute used for the **default chat path** (`custom:omni-101` / `smart-route`, since 2026-10-24 19:06) and for **all eight auxiliary slots** (`approval`, `compression`, `curator`, `mcp`, `profile_describer`, `skills_hub`, `title_generation`, `vision` — all `no-think/freeapi/auto` as of 2026-10-25 19:13)
|
|
|
|
> **Live `model:` block re-read 2026-10-26 — the default has moved TWICE.**
|
|
> `model.default` is **`smart-route`** on provider **`custom:omni-101`**
|
|
> (`http://100.78.63.20:20128`), `api_mode: chat_completions`. It moved to `nous` /
|
|
> `meituan/longcat-2.5-preview:free` on 2026-10-20 22:01 and **back** on **2026-10-24 19:06:10**.
|
|
> A six-provider top-level `fallback_providers` chain is live again, headed by the **dead `nous`
|
|
> credential** (`Invalid refresh token` since 2026-10-22).
|
|
> → [[systems/current-profile]] · [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]
|
|
>
|
|
|
|
> **The five `custom_providers` and their live base URLs:**
|
|
>
|
|
> | Name | Base URL | Host |
|
|
> |---|---|---|
|
|
> | `litellm` | `http://litellm:4000/v1` | LiteLLM proxy (m93p CT500) |
|
|
> | `guacano` | `http://ollama:11000/v1` | Guanaco on CT205 |
|
|
> | `Omniroute:20128` | `http://omniroute:20128/v1` | OmniRoute on m93p CT101 |
|
|
> | `command-code` | `https://api.commandcode.ai/provider/v1` | remote |
|
|
> | `omni-101` | `http://100.78.63.20:20128` | OmniRoute node `omniroute-1` (direct IP, not a DNS name) |
|
|
>
|
|
> Note the **two distinct OmniRoute endpoints**: `Omniroute:20128` (DNS, m93p CT101) and
|
|
> `omni-101` (raw IP `100.78.63.20:20128`). `omni-101` is the pin on effectively every
|
|
> model-calling cron job, and it has been the single point of failure in the 10-14, 10-16/17 and
|
|
> 10-19 incidents — its `/api/health` reads `200 {"status":"ok"}` throughout, so **reachability
|
|
> probes do not detect that failure class**; only job-level error text does.
|
|
|
|
## Context Enforcement
|
|
- Three layers: model native → combo `data.config.context_length` → Hermes client cap
|
|
- Effective context is the minimum of the three
|
|
|
|
## Combo Behavior
|
|
- `priority` retries same provider up to `maxRetries`, then errors; it does NOT fall through
|
|
- `fill-first` walks the list on failure
|
|
- `auto`/`lkgp` recalculates per request using health scoring
|
|
- Same-provider entries in a combo share rate-limit buckets
|
|
- Session affinity caches provider in `session_model_history`
|
|
|
|
## Aliases and Mappings
|
|
- Model aliases resolved via `key_value` table (`modelAliases` namespace)
|
|
- Wildcard mappings via `model_combo_mappings`
|
|
- OpenCode/Zen entries may resolve back to `opencode-zen` provider internally
|
|
|
|
## History
|
|
- User-provided: OpenRouter was used previously; provider lists now centered on LiteLLM + OmniRoute
|
|
- User-provided: `persist_switch_by_default=false`; `/model` is per-session unless `--global` used
|
|
- Inferred: Configuration undergoes periodic corrections related to vision, model fallback, and browser backend tuning
|
|
|
|
## Related
|
|
- [[model-providers]]
|
|
- [[systems/omniroute]]
|
|
- [[openrouter]]
|