Files
agent-estate-wiki/concepts/model-routing.md
T

3.4 KiB

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Model Routing concept active 2026-07-22 2026-10-20 2026-10-20 high
routing
model
inference
raw/configs/hermes-config-sanitized.txt
current-profile.md

Model Routing

Purpose

How Hermes selects providers, applies combos, and enforces context limits.

Current Flow

  • Main path: custom:litellm → http://litellm:4000/v1
  • Default model: freellmapi-auto
  • OmniRoute used for the default chat path (custom:omni-101 / smart-route, since 2026-10-24 19:06) and for all eight auxiliary slots (approval, compression, curator, mcp, profile_describer, skills_hub, title_generation, vision — all no-think/freeapi/auto as of 2026-10-25 19:13)

Live model: block re-read 2026-10-26 — the default has moved TWICE. model.default is smart-route on provider custom:omni-101 (http://100.78.63.20:20128), api_mode: chat_completions. It moved to nous / meituan/longcat-2.5-preview:free on 2026-10-20 22:01 and back on 2026-10-24 19:06:10. A six-provider top-level fallback_providers chain is live again, headed by the dead nous credential (Invalid refresh token since 2026-10-22). → systems/current-profile · incidents/2026-10-22-nous-portal-refresh-token-invalid

The five custom_providers and their live base URLs:

Name Base URL Host
litellm http://litellm:4000/v1 LiteLLM proxy (m93p CT500)
guacano http://ollama:11000/v1 Guanaco on CT205
Omniroute:20128 http://omniroute:20128/v1 OmniRoute on m93p CT101
command-code https://api.commandcode.ai/provider/v1 remote
omni-101 http://100.78.63.20:20128 OmniRoute node omniroute-1 (direct IP, not a DNS name)

Note the two distinct OmniRoute endpoints: Omniroute:20128 (DNS, m93p CT101) and omni-101 (raw IP 100.78.63.20:20128). omni-101 is the pin on effectively every model-calling cron job, and it has been the single point of failure in the 10-14, 10-16/17 and 10-19 incidents — its /api/health reads 200 {"status":"ok"} throughout, so reachability probes do not detect that failure class; only job-level error text does.

Context Enforcement

  • Three layers: model native → combo data.config.context_length → Hermes client cap
  • Effective context is the minimum of the three

Combo Behavior

  • priority retries same provider up to maxRetries, then errors; it does NOT fall through
  • fill-first walks the list on failure
  • auto/lkgp recalculates per request using health scoring
  • Same-provider entries in a combo share rate-limit buckets
  • Session affinity caches provider in session_model_history

Aliases and Mappings

  • Model aliases resolved via key_value table (modelAliases namespace)
  • Wildcard mappings via model_combo_mappings
  • OpenCode/Zen entries may resolve back to opencode-zen provider internally

History

  • User-provided: OpenRouter was used previously; provider lists now centered on LiteLLM + OmniRoute
  • User-provided: persist_switch_by_default=false; /model is per-session unless --global used
  • Inferred: Configuration undergoes periodic corrections related to vision, model fallback, and browser backend tuning