Files
agent-estate-wiki/systems/model-providers.md
T

112 lines
8.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Model Providers
type: system
status: active
created: 2026-07-22
updated: 2026-10-01
verified_on: 2026-10-01
confidence: high
tags: [model, provider, inference, routing]
sources: [live-probe, config, web-search]
---
# Model Providers
## Purpose
Inference providers available to Hermes, their routing paths, and notable model updates.
## Current Provider Stack
Verified from live `~/.hermes/config.yaml` → `custom_providers`, 2026-10-01.
| Provider | URL | Models | Role |
|---|---|---|---|
| `custom:omni-101` | `http://100.78.63.20:20128` | 5,384 | **Primary.** OmniRoute on the estate (`omni-101`). 15 of 30 cron jobs reference it |
| `custom:Omniroute:20128` | `http://omniroute:20128/v1` | 5,398 | Secondary OmniRoute host — legacy naming, kept alongside `omni-101` |
| `custom:litellm` | `http://litellm:4000/v1` | 1,728 | Fallback inference (slower path, 6–26s/call) |
| `custom:command-code` | `https://api.commandcode.ai/provider/v1` | 71 | DeepSeek Flash primary via the `deepseek-flash-mix` combo (added 2026-08-19); also the source of the `MODEL_NOT_IN_PLAN` 403 on `gpt-5.6-terra` |
| `custom:guacano` | `http://ollama:11000/v1` | 46 | Guanaco LLM proxy/router on CT205 (Ollama stack). See [[systems/guanaco]] |
| `custom:omni` | *(removed)* | — | **No longer configured.** It once served auxiliary roles (vision, skills, approval, mcp, web_extract, compression). Two cron jobs still name it as their provider key — see [[systems/scheduled-tasks]] |
**Built-in (non-`custom_providers`) routes still in use as fallbacks:** `opencode-go / mimo-v2.5`, `opencode-zen / deepseek-v4-flash-free`, `nous / stepfun/step-3.7-flash:free`, `ollama-cloud / qwen3.5:397b`, `kilocode / poolside/laguna-m.1:free`. `charm-hyper` (Charmbracelet, `hyper.charm.land/v1`, free tier 100 credits/mo) is no longer in the live stack.
**Default provider:** `custom` → `http://100.78.63.20:20128` (OmniRoute). **Default model:** `smart-route`.
Provider names that appear in job configs but not in `custom_providers` do **not** necessarily fail: the default `custom` block resolves to the same base URL, so a stale provider key can go unnoticed. Prefer routing aliases (`smart-route`) over raw upstream model names in pins.
## Default Model
- `smart-route` — the live default as of 2026-09-30 (`provider: custom`, `base_url: http://100.78.63.20:20128`)
- Anthony has explicitly **rejected** `free-lkgp` as a default; stable defaults are `smart-route` / `deepseek-paid-smart` (paid, not free)
- `resilient-free` was the default after the 2026-08-14 OmniRoute factory reset, but that combo is now in the broken set
- Previous default `auto/best-free` was misconfigured — had no combo definition and routed to a paid `gpt-5.6-sol`
## Retired / Unavailable Models (2026-09-25 → 2026-10-01)
Pins that fail at request time, with the exact upstream error. Each is an open cron-repin in [[systems/scheduled-tasks]].
| Pin | Upstream | Error | Affected job |
|---|---|---|---|
| `qwen3.5:397b` | `ollama-cloud` | `HTTP 410: qwen3.5:397b was retired at 2026-09-25` | `1a8e90ea2271` (wiki stocktake, streak 7) |
| `gpt-5.6-terra` | `command-code` | `403 MODEL_NOT_IN_PLAN: GPT-5.6 Terra available in Pro and above plans` | `dca8482e4f76` (Daily Marketing Job Radar) |
| `nvidia/deepseek-ai/deepseek-v4-flash` | `nvidia` via `custom:omni-101` | `HTTP 400: Model 'deepseek-ai/deepseek-v4-flash' is not available in the active live catalog for provider 'nvidia'` | `fad5a86979d4` (Facebook birthday sync) |
Lesson: a retired or plan-gated **raw model pin never self-recovers** — it fails identically on every run until someone changes the pin. Pin to a routing alias instead.
## Auxiliary Role Routing
- **Vision:** live `vision.model` is `stepfun/step-3.7-flash:free` on provider `nous`. Anthony asked on 2026-09-30 that vision calls use the **local** Ollama on `http://anthonys-mac-mini:11434` (gemma4) going forward — as of 2026-10-01 the config still points at `nous`, so the switch is recorded but not applied.
- Auxiliary roles (vision, skills, approval, mcp, web_extract, compression) previously routed through `custom:omni`, which no longer exists.
## Notable Model Updates
### HyperCharm (Charmbracelet) — Provider Research (2026-08-20)
- **What:** Hyper (hyper.charm.land) is Charmbracelet's LLM inference service, purpose-built for agentic coding models
- **OmniRoute integration:** Already merged — PR #5961 (2026-07-03) added as `charm-hyper` provider with passthrough models from `hyper.charm.land/v1/models`
- **Free plan:** 100 Hypercredits/month (~$5 value), no rollover — use it or lose it monthly
- **Credit mechanics:** 1 Hypercredit = $0.05; balance queryable via `GET /v1/credits` (returns `{"balance": 100}`)
- **Catalog pricing (cheap models):** deepseek-v4-flash $0.20/$0.40 per 1M in/out, qwen3.7-flash $0.20/$0.80, gemma-4-26b $0.12/$0.42
- **No config changes made** — research only; `charm-hyper` is available but not wired as a Hermes default
- **Status:** Candidate for free-tier OmniRoute combo; passthrough model import means zero maintenance
### GPT-5.6 Luna — Model Research (2026-08-14)
- **Community consensus:** Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper
- **Luna strengths:** visual/UI work, final code review, bounded high-stakes tasks, cleaner structured output
- **DeepSeek strengths:** automation, persistent executor work, high-volume subtasks, long-context economics
- **Key caveat:** Provider route matters more than model name — `openai-codex/gpt-5.6-luna`, `openrouter/openai/gpt-5.6-luna`, `deepseek/deepseek-v4-flash`, and `openrouter/deepseek/deepseek-v4-flash` are four different tests
- **Pricing:** Direct OpenAI $0.20/M input, $1.20/M output; OpenRouter $0.10/M input, $0.60/M output
- **Context:** 1.05M tokens, 128K output, function calling, structured outputs, image input, configurable reasoning effort
- **Recommendation:** Luna plans and reviews; DeepSeek executes — but test on a clean direct route before judging
### Guanaco v0.8.14 (2026-08-14)
- Updated from v0.8.11 on CT205 — three incremental bug-fix releases focused on DSML/tool-call output hygiene
- DSML tag leakage fixes: malformed tool-call output no longer leaks raw syntax junk into model context
- Dashboard performance fix: cached content size, fixed wrong column names in history query
- No new features, no breaking changes — low-risk update
- See [[systems/guanaco]]
### DeepSeek V4 Flash 0731 (2026-07-31)
- **Same model ID:** `deepseek-v4-flash` — no ID change, just a post-training bump
- **Same architecture:** 284B total params, 13B active per token, 1M context window
- **Same pricing:** $0.14/M input, $0.28/M output
- **Benchmark jump:** Terminal Bench 2.1: 82.7 (up from 61.8 preview — 34% relative gain)
- **Now surpasses V4-Pro preview** (72.1) on agent/coding benchmarks
- **Stricter tool name validation:** Rejects tool names not matching `^[a-zA-Z0-9_-]+$` — may cause issues with OmniRoute-transformed tool names
- **Impact:** Your existing cron jobs using `deepseek-v4-flash` get the upgrade automatically. The empty-stream fallback issue (`opencode-go/deepseek-v4-flash` returning empty 200s) is a provider plumbing problem, not a model quality problem.
### OmniRoute Free-DeepSeek Silent-Stop Mitigation (2026-08-04)
- Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on `deepseek-flash-free` and `smart-route`/`good-free` across both instances
- Reworked both combos: strategy `priority` → `fill-first`, dropped proven culprits (`238d5978/deepseek-v4-flash` + `-0731`, `nvidia` deepseek), healthy free models promoted, deepseek demoted to last resort
- See [[incidents/2026-08-04-omniroute-free-deepseek-silent-stops]]
## Context Enforcement
- Three layers: model native → combo `data.config.context_length` → Hermes client cap
- Effective context is the minimum of the three
## Combo Behavior
- `priority` retries same provider up to `maxRetries`, then errors; it does NOT fall through
- `fill-first` walks the list on failure
- `auto`/`lkgp` recalculates per request using health scoring
- Same-provider entries in a combo share rate-limit buckets
- Session affinity caches provider in `session_model_history`
## Related
- [[concepts/model-routing]]
- [[systems/omniroute]]
- [[systems/opencode-go]]