Files
agent-estate-wiki/systems/model-providers.md
T

12 KiB
Raw Blame History

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Model Providers system active 2026-07-22 2026-11-17 2026-11-17 high
model
provider
inference
routing
live-probe
config
web-search

Model Providers

Purpose

Inference providers available to Hermes, their routing paths, and notable model updates.

Current Provider Stack

Verified from live ~/.hermes/config.yaml → custom_providers, 2026-10-01.

Provider URL Models Role
custom:omni-101 http://100.78.63.20:20128 5,384 Primary. OmniRoute on the estate (omni-101). 21 of 26 enabled jobs pin it (2026-11-17)
custom:Omniroute:20128 http://omniroute:20128/v1 5,398 Secondary OmniRoute host — legacy naming, kept alongside omni-101
custom:litellm http://litellm:4000/v1 1,728 Fallback inference (slower path, 6–26s/call)
custom:command-code https://api.commandcode.ai/provider/v1 71 DeepSeek Flash primary via the deepseek-flash-mix combo (added 2026-08-19); also the source of the MODEL_NOT_IN_PLAN 403 on gpt-5.6-terra
custom:guacano http://ollama:11000/v1 46 Guanaco LLM proxy/router on CT205 (Ollama stack). See systems/guanaco
custom:omni (removed) — No longer configured. It once served auxiliary roles (vision, skills, approval, mcp, web_extract, compression). Two cron jobs still name it as their provider key — see systems/scheduled-tasks

Built-in (non-custom_providers) routes still in use as fallbacks: opencode-go / mimo-v2.5, opencode-zen / deepseek-v4-flash-free, nous / stepfun/step-3.7-flash:free, ollama-cloud / qwen3.5:397b, kilocode / poolside/laguna-m.1:free. charm-hyper (Charmbracelet, hyper.charm.land/v1, free tier 100 credits/mo) is no longer in the live stack. The live fallback_providers chain (2026-10-26) is nous / meituan/longcat-2.5-preview:free → opencode-zen / deepseek-v4.1-flash → custom:guacano / deepseek-v4.1-flash → custom:litellm / kilo-reflection-beam-free → kilocode / poolside/laguna-m.1:free → custom / or-inclusionai_ling-2_6-1t.

Default provider: custom:omni-101 → http://100.78.63.20:20128. Default model: smart-route. ⚠ The default moved BACK to OmniRoute on 2026-10-24 19:06:10 AWST (session switch note: "model was just switched from meituan/longcat-2.5-preview:free to smart-route via omni-101"). The nous direct window ran 2026-10-20 22:01 → 2026-10-24 19:06 and the 2026-10-22/10-24/10-25 audit entries recorded it as current — stale. config.yaml last written 2026-10-25 19:13. → incidents/2026-10-22-nous-portal-refresh-token-invalid

Provider names that appear in job configs but not in custom_providers do not necessarily fail: the default custom block resolves to the same base URL, so a stale provider key can go unnoticed. Prefer routing aliases (smart-route) over raw upstream model names in pins.

Default Model

  • smart-route on custom:omni-101 (http://100.78.63.20:20128, api_mode: chat_completions) — the live default since 2026-10-24 19:06:10 AWST, re-verified against live ~/.hermes/config.yaml 2026-10-26 00:1x.
  • ⚠ A six-provider top-level fallback_providers chain is live again (re-verified 2026-10-26): nous / meituan/longcat-2.5-preview:free → opencode-zen / deepseek-v4.1-flash → custom:guacano / deepseek-v4.1-flash → custom:litellm / kilo-reflection-beam-free → kilocode / poolside/laguna-m.1:free → custom (http://litellm:4000/v1) / or-inclusionai_ling-2_6-1t. The 2026-10-21 claim that no fallback chain exists is stale; the model block itself still carries only base_url, default, provider, api_mode, persist_switch_by_default.
  • ⚠ The first fallback entry is the dead nous credential (Invalid refresh token, since 2026-10-22) → incidents/2026-10-22-nous-portal-refresh-token-invalid
  • Anthony has explicitly rejected free-lkgp as a default; stable defaults are smart-route / deepseek-paid-smart (paid, not free)
  • resilient-free was the default after the 2026-08-14 OmniRoute factory reset, but that combo is now in the broken set
  • Previous default auto/best-free was misconfigured — had no combo definition and routed to a paid gpt-5.6-sol
  • 21 of 26 enabled scheduled jobs pin OmniRoute explicitly (smart-route ×14, free-stack ×6, deepseek-paid-smart ×1; verified from live jobs.json 2026-11-17). The remaining 5 enabled jobs carry no pin (no-agent/script jobs) and follow the global default, which is also OmniRoute — the fleet runs a single upstream path. The two jobs that still named the removed custom:omni key on 2026-10-26 no longer do. → systems/scheduled-tasks

Auxiliary Role Routing (re-verified live 2026-11-17)

All eight auxiliary: slots — approval, compression, curator, mcp, profile_describer, skills_hub, title_generation, vision — are no-think/freeapi/auto on custom:omni-101, each with the chain no-think/freeapi/deepseek-v4-flash → no-think/kg/kilo-auto/free and free_only: true. Compression timeout 90 s; vision timeout 180 s. A ninth slot, pulse_memory, is nous / anthropic/claude-haiku-4.5 — i.e. it sits on the dead nous credential; a pulse-memory call that hits it will fail until the slot or the credential is fixed.

  • Re-placed 2026-10-25 19:13 AWST, replacing the nous / stepfun/step-3.7-flash:free and custom:omni aux paths recorded on 2026-10-02 and 2026-10-25 (17:50).
  • ⚠ The move was made because the Nous Portal credential is dead (Invalid refresh token), so it masks rather than fixes that fault → incidents/2026-10-22-nous-portal-refresh-token-invalid
  • Probe results from that session (17:50–19:13, not re-run 2026-10-26): freeapi/auto healthy for text and vision; kg/kilo-auto/free returns empty content on image input; stepfun/step-3.7-flash:free → 401 no active credentials; freeapi pinned models in cooldown (429). ⚠ A vision fallback returning empty content is a silent failure mode — check freeapi/auto itself, not the fallback order, if image calls go blank.

Retired / Unavailable Models (2026-09-25 → 2026-10-01)

Pins that fail at request time, with the exact upstream error. Each is an open cron-repin in systems/scheduled-tasks.

Pin Upstream Error Affected job
qwen3.5:397b ollama-cloud HTTP 410: qwen3.5:397b was retired at 2026-09-25 1a8e90ea2271 (wiki stocktake, streak 7)
gpt-5.6-terra command-code 403 MODEL_NOT_IN_PLAN: GPT-5.6 Terra available in Pro and above plans dca8482e4f76 (Daily Marketing Job Radar)
nvidia/deepseek-ai/deepseek-v4-flash nvidia via custom:omni-101 HTTP 400: Model 'deepseek-ai/deepseek-v4-flash' is not available in the active live catalog for provider 'nvidia' fad5a86979d4 (Facebook birthday sync)

Lesson: a retired or plan-gated raw model pin never self-recovers — it fails identically on every run until someone changes the pin. Pin to a routing alias instead.

Auxiliary Role Routing

  • Vision: live vision.model is stepfun/step-3.7-flash:free on provider nous. Anthony asked on 2026-09-30 that vision calls use the local Ollama on http://anthonys-mac-mini:11434 (gemma4) going forward — as of 2026-10-01 the config still points at nous, so the switch is recorded but not applied.
  • Auxiliary roles (vision, skills, approval, mcp, web_extract, compression) previously routed through custom:omni, which no longer exists.

Notable Model Updates

HyperCharm (Charmbracelet) — Provider Research (2026-08-20)

  • What: Hyper (hyper.charm.land) is Charmbracelet's LLM inference service, purpose-built for agentic coding models
  • OmniRoute integration: Already merged — PR #5961 (2026-07-03) added as charm-hyper provider with passthrough models from hyper.charm.land/v1/models
  • Free plan: 100 Hypercredits/month (~$5 value), no rollover — use it or lose it monthly
  • Credit mechanics: 1 Hypercredit = $0.05; balance queryable via GET /v1/credits (returns {"balance": 100})
  • Catalog pricing (cheap models): deepseek-v4-flash $0.20/$0.40 per 1M in/out, qwen3.7-flash $0.20/$0.80, gemma-4-26b $0.12/$0.42
  • No config changes made — research only; charm-hyper is available but not wired as a Hermes default
  • Status: Candidate for free-tier OmniRoute combo; passthrough model import means zero maintenance

GPT-5.6 Luna — Model Research (2026-08-14)

  • Community consensus: Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper
  • Luna strengths: visual/UI work, final code review, bounded high-stakes tasks, cleaner structured output
  • DeepSeek strengths: automation, persistent executor work, high-volume subtasks, long-context economics
  • Key caveat: Provider route matters more than model name — openai-codex/gpt-5.6-luna, openrouter/openai/gpt-5.6-luna, deepseek/deepseek-v4-flash, and openrouter/deepseek/deepseek-v4-flash are four different tests
  • Pricing: Direct OpenAI $0.20/M input, $1.20/M output; OpenRouter $0.10/M input, $0.60/M output
  • Context: 1.05M tokens, 128K output, function calling, structured outputs, image input, configurable reasoning effort
  • Recommendation: Luna plans and reviews; DeepSeek executes — but test on a clean direct route before judging

Guanaco v0.8.14 (2026-08-14)

  • Updated from v0.8.11 on CT205 — three incremental bug-fix releases focused on DSML/tool-call output hygiene
  • DSML tag leakage fixes: malformed tool-call output no longer leaks raw syntax junk into model context
  • Dashboard performance fix: cached content size, fixed wrong column names in history query
  • No new features, no breaking changes — low-risk update
  • See systems/guanaco

DeepSeek V4 Flash 0731 (2026-07-31)

  • Same model ID: deepseek-v4-flash — no ID change, just a post-training bump
  • Same architecture: 284B total params, 13B active per token, 1M context window
  • Same pricing: $0.14/M input, $0.28/M output
  • Benchmark jump: Terminal Bench 2.1: 82.7 (up from 61.8 preview — 34% relative gain)
  • Now surpasses V4-Pro preview (72.1) on agent/coding benchmarks
  • Stricter tool name validation: Rejects tool names not matching ^[a-zA-Z0-9_-]+$ — may cause issues with OmniRoute-transformed tool names
  • Impact: Your existing cron jobs using deepseek-v4-flash get the upgrade automatically. The empty-stream fallback issue (opencode-go/deepseek-v4-flash returning empty 200s) is a provider plumbing problem, not a model quality problem.

OmniRoute Free-DeepSeek Silent-Stop Mitigation (2026-08-04)

  • Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on deepseek-flash-free and smart-route/good-free across both instances
  • Reworked both combos: strategy priority → fill-first, dropped proven culprits (238d5978/deepseek-v4-flash + -0731, nvidia deepseek), healthy free models promoted, deepseek demoted to last resort
  • See incidents/2026-08-04-omniroute-free-deepseek-silent-stops

Context Enforcement

  • Three layers: model native → combo data.config.context_length → Hermes client cap
  • Effective context is the minimum of the three

Combo Behavior

  • priority retries same provider up to maxRetries, then errors; it does NOT fall through
  • fill-first walks the list on failure
  • auto/lkgp recalculates per request using health scoring
  • Same-provider entries in a combo share rate-limit buckets
  • Session affinity caches provider in session_model_history