Files
agent-estate-wiki/concepts/model-routing.md
T

68 lines
2.8 KiB
Markdown

---
title: Model Routing
type: concept
status: active
created: 2026-07-22
updated: 2026-10-20
verified_on: 2026-10-20
confidence: high
tags: [routing, model, inference]
sources: [raw/configs/hermes-config-sanitized.txt, current-profile.md]
---
# Model Routing
## Purpose
How Hermes selects providers, applies combos, and enforces context limits.
## Current Flow
- Main path: `custom:litellm` → `http://litellm:4000/v1`
- Default model: `freellmapi-auto`
- OmniRoute used for auxiliary roles: vision, skills_hub, approval, mcp, title_generation, web_extract, compression
> **Live `model:` block re-read 2026-10-20 — the default has moved.**
> `model.default` is now **`meituan/longcat-2.5-preview:free`** (provider `nous`), not
> `freellmapi-auto`. The block also carries `base_url`, `persist_switch_by_default` and `api_mode`.
>
> **The five `custom_providers` and their live base URLs:**
>
> | Name | Base URL | Host |
> |---|---|---|
> | `litellm` | `http://litellm:4000/v1` | LiteLLM proxy (m93p CT500) |
> | `guacano` | `http://ollama:11000/v1` | Guanaco on CT205 |
> | `Omniroute:20128` | `http://omniroute:20128/v1` | OmniRoute on m93p CT101 |
> | `command-code` | `https://api.commandcode.ai/provider/v1` | remote |
> | `omni-101` | `http://100.78.63.20:20128` | OmniRoute node `omniroute-1` (direct IP, not a DNS name) |
>
> Note the **two distinct OmniRoute endpoints**: `Omniroute:20128` (DNS, m93p CT101) and
> `omni-101` (raw IP `100.78.63.20:20128`). `omni-101` is the pin on effectively every
> model-calling cron job, and it has been the single point of failure in the 10-14, 10-16/17 and
> 10-19 incidents — its `/api/health` reads `200 {"status":"ok"}` throughout, so **reachability
> probes do not detect that failure class**; only job-level error text does.
## Context Enforcement
- Three layers: model native → combo `data.config.context_length` → Hermes client cap
- Effective context is the minimum of the three
## Combo Behavior
- `priority` retries same provider up to `maxRetries`, then errors; it does NOT fall through
- `fill-first` walks the list on failure
- `auto`/`lkgp` recalculates per request using health scoring
- Same-provider entries in a combo share rate-limit buckets
- Session affinity caches provider in `session_model_history`
## Aliases and Mappings
- Model aliases resolved via `key_value` table (`modelAliases` namespace)
- Wildcard mappings via `model_combo_mappings`
- OpenCode/Zen entries may resolve back to `opencode-zen` provider internally
## History
- User-provided: OpenRouter was used previously; provider lists now centered on LiteLLM + OmniRoute
- User-provided: `persist_switch_by_default=false`; `/model` is per-session unless `--global` used
- Inferred: Configuration undergoes periodic corrections related to vision, model fallback, and browser backend tuning
## Related
- [[model-providers]]
- [[systems/omniroute]]
- [[openrouter]]