Model Providers

Purpose

Inference providers available to Hermes, their routing paths, and notable model updates.

Provider	URL	Role
`custom:litellm`	`http://litellm:4000/v1`	Main inference (slower path, 6–26s/call)
`custom:omni`	`http://omni:20128/v1`	Auxiliary roles (vision, skills, approval, mcp, web_extract, compression) — factory-reset 2026-08-14
`opencode-zen`	(internal)	Free-tier model access for cron jobs
Guanaco	`ollama:11434`	LLM proxy/router on CT205 (Ollama stack)

resilient-free — resolves through OmniRoute (changed from auto/best-free on 2026-08-14 after factory reset)
Previous default auto/best-free was misconfigured — had no combo definition and routed to paid gpt-5.6-sol
freellmapi-auto (LiteLLM path) still available but slower

Community consensus: Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper
Luna strengths: visual/UI work, final code review, bounded high-stakes tasks, cleaner structured output
DeepSeek strengths: automation, persistent executor work, high-volume subtasks, long-context economics
Key caveat: Provider route matters more than model name — openai-codex/gpt-5.6-luna, openrouter/openai/gpt-5.6-luna, deepseek/deepseek-v4-flash, and openrouter/deepseek/deepseek-v4-flash are four different tests
Pricing: Direct OpenAI $0.20/M input, $1.20/M output; OpenRouter $0.10/M input, $0.60/M output
Context: 1.05M tokens, 128K output, function calling, structured outputs, image input, configurable reasoning effort
Recommendation: Luna plans and reviews; DeepSeek executes — but test on a clean direct route before judging

Updated from v0.8.11 on CT205 — three incremental bug-fix releases focused on DSML/tool-call output hygiene
DSML tag leakage fixes: malformed tool-call output no longer leaks raw syntax junk into model context
Dashboard performance fix: cached content size, fixed wrong column names in history query
No new features, no breaking changes — low-risk update
See systems/guanaco

Same model ID: deepseek-v4-flash — no ID change, just a post-training bump
Same architecture: 284B total params, 13B active per token, 1M context window
Same pricing: $0.14/M input, $0.28/M output
Benchmark jump: Terminal Bench 2.1: 82.7 (up from 61.8 preview — 34% relative gain)
Now surpasses V4-Pro preview (72.1) on agent/coding benchmarks
Stricter tool name validation: Rejects tool names not matching ^[a-zA-Z0-9_-]+$ — may cause issues with OmniRoute-transformed tool names
Impact: Your existing cron jobs using deepseek-v4-flash get the upgrade automatically. The empty-stream fallback issue (opencode-go/deepseek-v4-flash returning empty 200s) is a provider plumbing problem, not a model quality problem.

Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on deepseek-flash-free and smart-route/good-free across both instances
Reworked both combos: strategy priority → fill-first, dropped proven culprits (238d5978/deepseek-v4-flash + -0731, nvidia deepseek), healthy free models promoted, deepseek demoted to last resort
See incidents/2026-08-04-omniroute-free-deepseek-silent-stops

Three layers: model native → combo data.config.context_length → Hermes client cap
Effective context is the minimum of the three

priority retries same provider up to maxRetries, then errors; it does NOT fall through
fill-first walks the list on failure
auto/lkgp recalculates per request using health scoring
Same-provider entries in a combo share rate-limit buckets
Session affinity caches provider in session_model_history