Files
agent-estate-wiki/systems/model-providers.md
T

8.2 KiB
Raw Blame History

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Model Providers system active 2026-07-22 2026-10-01 2026-10-01 high
model
provider
inference
routing
live-probe
config
web-search

Model Providers

Purpose

Inference providers available to Hermes, their routing paths, and notable model updates.

Current Provider Stack

Verified from live ~/.hermes/config.yaml → custom_providers, 2026-10-01.

Provider URL Models Role
custom:omni-101 http://100.78.63.20:20128 5,384 Primary. OmniRoute on the estate (omni-101). 15 of 30 cron jobs reference it
custom:Omniroute:20128 http://omniroute:20128/v1 5,398 Secondary OmniRoute host — legacy naming, kept alongside omni-101
custom:litellm http://litellm:4000/v1 1,728 Fallback inference (slower path, 6–26s/call)
custom:command-code https://api.commandcode.ai/provider/v1 71 DeepSeek Flash primary via the deepseek-flash-mix combo (added 2026-08-19); also the source of the MODEL_NOT_IN_PLAN 403 on gpt-5.6-terra
custom:guacano http://ollama:11000/v1 46 Guanaco LLM proxy/router on CT205 (Ollama stack). See systems/guanaco
custom:omni (removed) — No longer configured. It once served auxiliary roles (vision, skills, approval, mcp, web_extract, compression). Two cron jobs still name it as their provider key — see systems/scheduled-tasks

Built-in (non-custom_providers) routes still in use as fallbacks: opencode-go / mimo-v2.5, opencode-zen / deepseek-v4-flash-free, nous / stepfun/step-3.7-flash:free, ollama-cloud / qwen3.5:397b, kilocode / poolside/laguna-m.1:free. charm-hyper (Charmbracelet, hyper.charm.land/v1, free tier 100 credits/mo) is no longer in the live stack.

Default provider: custom → http://100.78.63.20:20128 (OmniRoute). Default model: smart-route.

Provider names that appear in job configs but not in custom_providers do not necessarily fail: the default custom block resolves to the same base URL, so a stale provider key can go unnoticed. Prefer routing aliases (smart-route) over raw upstream model names in pins.

Default Model

  • smart-route — the live default as of 2026-09-30 (provider: custom, base_url: http://100.78.63.20:20128)
  • Anthony has explicitly rejected free-lkgp as a default; stable defaults are smart-route / deepseek-paid-smart (paid, not free)
  • resilient-free was the default after the 2026-08-14 OmniRoute factory reset, but that combo is now in the broken set
  • Previous default auto/best-free was misconfigured — had no combo definition and routed to a paid gpt-5.6-sol

Retired / Unavailable Models (2026-09-25 → 2026-10-01)

Pins that fail at request time, with the exact upstream error. Each is an open cron-repin in systems/scheduled-tasks.

Pin Upstream Error Affected job
qwen3.5:397b ollama-cloud HTTP 410: qwen3.5:397b was retired at 2026-09-25 1a8e90ea2271 (wiki stocktake, streak 7)
gpt-5.6-terra command-code 403 MODEL_NOT_IN_PLAN: GPT-5.6 Terra available in Pro and above plans dca8482e4f76 (Daily Marketing Job Radar)
nvidia/deepseek-ai/deepseek-v4-flash nvidia via custom:omni-101 HTTP 400: Model 'deepseek-ai/deepseek-v4-flash' is not available in the active live catalog for provider 'nvidia' fad5a86979d4 (Facebook birthday sync)

Lesson: a retired or plan-gated raw model pin never self-recovers — it fails identically on every run until someone changes the pin. Pin to a routing alias instead.

Auxiliary Role Routing

  • Vision: live vision.model is stepfun/step-3.7-flash:free on provider nous. Anthony asked on 2026-09-30 that vision calls use the local Ollama on http://anthonys-mac-mini:11434 (gemma4) going forward — as of 2026-10-01 the config still points at nous, so the switch is recorded but not applied.
  • Auxiliary roles (vision, skills, approval, mcp, web_extract, compression) previously routed through custom:omni, which no longer exists.

Notable Model Updates

HyperCharm (Charmbracelet) — Provider Research (2026-08-20)

  • What: Hyper (hyper.charm.land) is Charmbracelet's LLM inference service, purpose-built for agentic coding models
  • OmniRoute integration: Already merged — PR #5961 (2026-07-03) added as charm-hyper provider with passthrough models from hyper.charm.land/v1/models
  • Free plan: 100 Hypercredits/month (~$5 value), no rollover — use it or lose it monthly
  • Credit mechanics: 1 Hypercredit = $0.05; balance queryable via GET /v1/credits (returns {"balance": 100})
  • Catalog pricing (cheap models): deepseek-v4-flash $0.20/$0.40 per 1M in/out, qwen3.7-flash $0.20/$0.80, gemma-4-26b $0.12/$0.42
  • No config changes made — research only; charm-hyper is available but not wired as a Hermes default
  • Status: Candidate for free-tier OmniRoute combo; passthrough model import means zero maintenance

GPT-5.6 Luna — Model Research (2026-08-14)

  • Community consensus: Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper
  • Luna strengths: visual/UI work, final code review, bounded high-stakes tasks, cleaner structured output
  • DeepSeek strengths: automation, persistent executor work, high-volume subtasks, long-context economics
  • Key caveat: Provider route matters more than model name — openai-codex/gpt-5.6-luna, openrouter/openai/gpt-5.6-luna, deepseek/deepseek-v4-flash, and openrouter/deepseek/deepseek-v4-flash are four different tests
  • Pricing: Direct OpenAI $0.20/M input, $1.20/M output; OpenRouter $0.10/M input, $0.60/M output
  • Context: 1.05M tokens, 128K output, function calling, structured outputs, image input, configurable reasoning effort
  • Recommendation: Luna plans and reviews; DeepSeek executes — but test on a clean direct route before judging

Guanaco v0.8.14 (2026-08-14)

  • Updated from v0.8.11 on CT205 — three incremental bug-fix releases focused on DSML/tool-call output hygiene
  • DSML tag leakage fixes: malformed tool-call output no longer leaks raw syntax junk into model context
  • Dashboard performance fix: cached content size, fixed wrong column names in history query
  • No new features, no breaking changes — low-risk update
  • See systems/guanaco

DeepSeek V4 Flash 0731 (2026-07-31)

  • Same model ID: deepseek-v4-flash — no ID change, just a post-training bump
  • Same architecture: 284B total params, 13B active per token, 1M context window
  • Same pricing: $0.14/M input, $0.28/M output
  • Benchmark jump: Terminal Bench 2.1: 82.7 (up from 61.8 preview — 34% relative gain)
  • Now surpasses V4-Pro preview (72.1) on agent/coding benchmarks
  • Stricter tool name validation: Rejects tool names not matching ^[a-zA-Z0-9_-]+$ — may cause issues with OmniRoute-transformed tool names
  • Impact: Your existing cron jobs using deepseek-v4-flash get the upgrade automatically. The empty-stream fallback issue (opencode-go/deepseek-v4-flash returning empty 200s) is a provider plumbing problem, not a model quality problem.

OmniRoute Free-DeepSeek Silent-Stop Mitigation (2026-08-04)

  • Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on deepseek-flash-free and smart-route/good-free across both instances
  • Reworked both combos: strategy priority → fill-first, dropped proven culprits (238d5978/deepseek-v4-flash + -0731, nvidia deepseek), healthy free models promoted, deepseek demoted to last resort
  • See incidents/2026-08-04-omniroute-free-deepseek-silent-stops

Context Enforcement

  • Three layers: model native → combo data.config.context_length → Hermes client cap
  • Effective context is the minimum of the three

Combo Behavior

  • priority retries same provider up to maxRetries, then errors; it does NOT fall through
  • fill-first walks the list on failure
  • auto/lkgp recalculates per request using health scoring
  • Same-provider entries in a combo share rate-limit buckets
  • Session affinity caches provider in session_model_history