68 lines
2.8 KiB
Markdown
68 lines
2.8 KiB
Markdown
---
|
|
title: Model Routing
|
|
type: concept
|
|
status: active
|
|
created: 2026-07-22
|
|
updated: 2026-10-20
|
|
verified_on: 2026-10-20
|
|
confidence: high
|
|
tags: [routing, model, inference]
|
|
sources: [raw/configs/hermes-config-sanitized.txt, current-profile.md]
|
|
---
|
|
|
|
# Model Routing
|
|
|
|
## Purpose
|
|
How Hermes selects providers, applies combos, and enforces context limits.
|
|
|
|
## Current Flow
|
|
- Main path: `custom:litellm` → `http://litellm:4000/v1`
|
|
- Default model: `freellmapi-auto`
|
|
- OmniRoute used for auxiliary roles: vision, skills_hub, approval, mcp, title_generation, web_extract, compression
|
|
|
|
> **Live `model:` block re-read 2026-10-20 — the default has moved.**
|
|
> `model.default` is now **`meituan/longcat-2.5-preview:free`** (provider `nous`), not
|
|
> `freellmapi-auto`. The block also carries `base_url`, `persist_switch_by_default` and `api_mode`.
|
|
>
|
|
> **The five `custom_providers` and their live base URLs:**
|
|
>
|
|
> | Name | Base URL | Host |
|
|
> |---|---|---|
|
|
> | `litellm` | `http://litellm:4000/v1` | LiteLLM proxy (m93p CT500) |
|
|
> | `guacano` | `http://ollama:11000/v1` | Guanaco on CT205 |
|
|
> | `Omniroute:20128` | `http://omniroute:20128/v1` | OmniRoute on m93p CT101 |
|
|
> | `command-code` | `https://api.commandcode.ai/provider/v1` | remote |
|
|
> | `omni-101` | `http://100.78.63.20:20128` | OmniRoute node `omniroute-1` (direct IP, not a DNS name) |
|
|
>
|
|
> Note the **two distinct OmniRoute endpoints**: `Omniroute:20128` (DNS, m93p CT101) and
|
|
> `omni-101` (raw IP `100.78.63.20:20128`). `omni-101` is the pin on effectively every
|
|
> model-calling cron job, and it has been the single point of failure in the 10-14, 10-16/17 and
|
|
> 10-19 incidents — its `/api/health` reads `200 {"status":"ok"}` throughout, so **reachability
|
|
> probes do not detect that failure class**; only job-level error text does.
|
|
|
|
## Context Enforcement
|
|
- Three layers: model native → combo `data.config.context_length` → Hermes client cap
|
|
- Effective context is the minimum of the three
|
|
|
|
## Combo Behavior
|
|
- `priority` retries same provider up to `maxRetries`, then errors; it does NOT fall through
|
|
- `fill-first` walks the list on failure
|
|
- `auto`/`lkgp` recalculates per request using health scoring
|
|
- Same-provider entries in a combo share rate-limit buckets
|
|
- Session affinity caches provider in `session_model_history`
|
|
|
|
## Aliases and Mappings
|
|
- Model aliases resolved via `key_value` table (`modelAliases` namespace)
|
|
- Wildcard mappings via `model_combo_mappings`
|
|
- OpenCode/Zen entries may resolve back to `opencode-zen` provider internally
|
|
|
|
## History
|
|
- User-provided: OpenRouter was used previously; provider lists now centered on LiteLLM + OmniRoute
|
|
- User-provided: `persist_switch_by_default=false`; `/model` is per-session unless `--global` used
|
|
- Inferred: Configuration undergoes periodic corrections related to vision, model fallback, and browser backend tuning
|
|
|
|
## Related
|
|
- [[model-providers]]
|
|
- [[systems/omniroute]]
|
|
- [[openrouter]]
|