146 lines
12 KiB
Markdown
146 lines
12 KiB
Markdown
---
|
||
title: Model Providers
|
||
type: system
|
||
status: active
|
||
created: 2026-07-22
|
||
updated: 2026-11-17
|
||
verified_on: 2026-11-17
|
||
confidence: high
|
||
tags: [model, provider, inference, routing]
|
||
sources: [live-probe, config, web-search]
|
||
---
|
||
|
||
# Model Providers
|
||
|
||
## Purpose
|
||
Inference providers available to Hermes, their routing paths, and notable model updates.
|
||
|
||
## Current Provider Stack
|
||
Verified from live `~/.hermes/config.yaml` → `custom_providers`, 2026-10-01.
|
||
|
||
| Provider | URL | Models | Role |
|
||
|---|---|---|---|
|
||
| `custom:omni-101` | `http://100.78.63.20:20128` | 5,384 | **Primary.** OmniRoute on the estate (`omni-101`). 21 of 26 enabled jobs pin it (2026-11-17) |
|
||
| `custom:Omniroute:20128` | `http://omniroute:20128/v1` | 5,398 | Secondary OmniRoute host — legacy naming, kept alongside `omni-101` |
|
||
| `custom:litellm` | `http://litellm:4000/v1` | 1,728 | Fallback inference (slower path, 6–26s/call) |
|
||
| `custom:command-code` | `https://api.commandcode.ai/provider/v1` | 71 | DeepSeek Flash primary via the `deepseek-flash-mix` combo (added 2026-08-19); also the source of the `MODEL_NOT_IN_PLAN` 403 on `gpt-5.6-terra` |
|
||
| `custom:guacano` | `http://ollama:11000/v1` | 46 | Guanaco LLM proxy/router on CT205 (Ollama stack). See [[systems/guanaco]] |
|
||
| `custom:omni` | *(removed)* | — | **No longer configured.** It once served auxiliary roles (vision, skills, approval, mcp, web_extract, compression). Two cron jobs still name it as their provider key — see [[systems/scheduled-tasks]] |
|
||
|
||
**Built-in (non-`custom_providers`) routes still in use as fallbacks:** `opencode-go / mimo-v2.5`, `opencode-zen / deepseek-v4-flash-free`, `nous / stepfun/step-3.7-flash:free`, `ollama-cloud / qwen3.5:397b`, `kilocode / poolside/laguna-m.1:free`. `charm-hyper` (Charmbracelet, `hyper.charm.land/v1`, free tier 100 credits/mo) is no longer in the live stack. **The live `fallback_providers` chain (2026-10-26) is `nous` / `meituan/longcat-2.5-preview:free` → `opencode-zen` / `deepseek-v4.1-flash` → `custom:guacano` / `deepseek-v4.1-flash` → `custom:litellm` / `kilo-reflection-beam-free` → `kilocode` / `poolside/laguna-m.1:free` → `custom` / `or-inclusionai_ling-2_6-1t`.**
|
||
|
||
**Default provider:** **`custom:omni-101`** → `http://100.78.63.20:20128`. **Default model:** **`smart-route`**.
|
||
⚠ **The default moved BACK to OmniRoute on 2026-10-24 19:06:10 AWST** (session switch note: *"model was just
|
||
switched from meituan/longcat-2.5-preview:free to smart-route via omni-101"*). The `nous` direct window ran
|
||
2026-10-20 22:01 → 2026-10-24 19:06 and the 2026-10-22/10-24/10-25 audit entries recorded it as current —
|
||
**stale**. `config.yaml` last written 2026-10-25 19:13.
|
||
→ [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]
|
||
|
||
Provider names that appear in job configs but not in `custom_providers` do **not** necessarily fail: the default `custom` block resolves to the same base URL, so a stale provider key can go unnoticed. Prefer routing aliases (`smart-route`) over raw upstream model names in pins.
|
||
|
||
## Default Model
|
||
- **`smart-route`** on **`custom:omni-101`** (`http://100.78.63.20:20128`, `api_mode: chat_completions`) — the
|
||
live default since **2026-10-24 19:06:10 AWST**, re-verified against live `~/.hermes/config.yaml`
|
||
**2026-10-26 00:1x**.
|
||
- **⚠ A six-provider top-level `fallback_providers` chain is live again** (re-verified 2026-10-26): `nous` /
|
||
`meituan/longcat-2.5-preview:free` → `opencode-zen` / `deepseek-v4.1-flash` → `custom:guacano` /
|
||
`deepseek-v4.1-flash` → `custom:litellm` / `kilo-reflection-beam-free` → `kilocode` /
|
||
`poolside/laguna-m.1:free` → `custom` (`http://litellm:4000/v1`) / `or-inclusionai_ling-2_6-1t`. The
|
||
2026-10-21 claim that **no** fallback chain exists is **stale**; the `model` block itself still carries only
|
||
`base_url`, `default`, `provider`, `api_mode`, `persist_switch_by_default`.
|
||
- ⚠ **The first fallback entry is the dead `nous` credential** (`Invalid refresh token`, since 2026-10-22)
|
||
→ [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]
|
||
- Anthony has explicitly **rejected** `free-lkgp` as a default; stable defaults are `smart-route` / `deepseek-paid-smart` (paid, not free)
|
||
- `resilient-free` was the default after the 2026-08-14 OmniRoute factory reset, but that combo is now in the broken set
|
||
- Previous default `auto/best-free` was misconfigured — had no combo definition and routed to a paid `gpt-5.6-sol`
|
||
- **21 of 26 enabled scheduled jobs pin OmniRoute explicitly** (`smart-route` ×14, `free-stack` ×6, `deepseek-paid-smart` ×1; verified from live `jobs.json` 2026-11-17). The remaining 5 enabled jobs carry no pin (no-agent/script jobs) and follow the global default, which is **also** OmniRoute — the fleet runs a **single upstream path**. The two jobs that still named the removed `custom:omni` key on 2026-10-26 no longer do. → [[systems/scheduled-tasks]]
|
||
|
||
## Auxiliary Role Routing (re-verified live 2026-11-17)
|
||
**All eight `auxiliary:` slots** — `approval`, `compression`, `curator`, `mcp`, `profile_describer`,
|
||
`skills_hub`, `title_generation`, `vision` — are **`no-think/freeapi/auto` on `custom:omni-101`**, each with
|
||
the chain `no-think/freeapi/deepseek-v4-flash` → `no-think/kg/kilo-auto/free` and `free_only: true`.
|
||
**Compression timeout 90 s; vision timeout 180 s.** A ninth slot, **`pulse_memory`**, is
|
||
**`nous` / `anthropic/claude-haiku-4.5`** — i.e. it sits on the **dead `nous` credential**; a pulse-memory call
|
||
that hits it will fail until the slot or the credential is fixed.
|
||
|
||
- Re-placed **2026-10-25 19:13 AWST**, replacing the `nous` / `stepfun/step-3.7-flash:free` and `custom:omni`
|
||
aux paths recorded on 2026-10-02 and 2026-10-25 (17:50).
|
||
- ⚠ The move was made **because the Nous Portal credential is dead** (`Invalid refresh token`), so it **masks
|
||
rather than fixes** that fault → [[incidents/2026-10-22-nous-portal-refresh-token-invalid]]
|
||
- Probe results from that session (17:50–19:13, not re-run 2026-10-26): `freeapi/auto` healthy for text and
|
||
vision; `kg/kilo-auto/free` returns **empty content on image input**; `stepfun/step-3.7-flash:free` → 401
|
||
*no active credentials*; freeapi pinned models in cooldown (429). ⚠ A **vision fallback returning empty
|
||
content** is a silent failure mode — check `freeapi/auto` itself, not the fallback order, if image calls
|
||
go blank.
|
||
|
||
## Retired / Unavailable Models (2026-09-25 → 2026-10-01)
|
||
Pins that fail at request time, with the exact upstream error. Each is an open cron-repin in [[systems/scheduled-tasks]].
|
||
| Pin | Upstream | Error | Affected job |
|
||
|---|---|---|---|
|
||
| `qwen3.5:397b` | `ollama-cloud` | `HTTP 410: qwen3.5:397b was retired at 2026-09-25` | `1a8e90ea2271` (wiki stocktake, streak 7) |
|
||
| `gpt-5.6-terra` | `command-code` | `403 MODEL_NOT_IN_PLAN: GPT-5.6 Terra available in Pro and above plans` | `dca8482e4f76` (Daily Marketing Job Radar) |
|
||
| `nvidia/deepseek-ai/deepseek-v4-flash` | `nvidia` via `custom:omni-101` | `HTTP 400: Model 'deepseek-ai/deepseek-v4-flash' is not available in the active live catalog for provider 'nvidia'` | `fad5a86979d4` (Facebook birthday sync) |
|
||
|
||
Lesson: a retired or plan-gated **raw model pin never self-recovers** — it fails identically on every run until someone changes the pin. Pin to a routing alias instead.
|
||
|
||
## Auxiliary Role Routing
|
||
- **Vision:** live `vision.model` is `stepfun/step-3.7-flash:free` on provider `nous`. Anthony asked on 2026-09-30 that vision calls use the **local** Ollama on `http://anthonys-mac-mini:11434` (gemma4) going forward — as of 2026-10-01 the config still points at `nous`, so the switch is recorded but not applied.
|
||
- Auxiliary roles (vision, skills, approval, mcp, web_extract, compression) previously routed through `custom:omni`, which no longer exists.
|
||
|
||
## Notable Model Updates
|
||
|
||
### HyperCharm (Charmbracelet) — Provider Research (2026-08-20)
|
||
- **What:** Hyper (hyper.charm.land) is Charmbracelet's LLM inference service, purpose-built for agentic coding models
|
||
- **OmniRoute integration:** Already merged — PR #5961 (2026-07-03) added as `charm-hyper` provider with passthrough models from `hyper.charm.land/v1/models`
|
||
- **Free plan:** 100 Hypercredits/month (~$5 value), no rollover — use it or lose it monthly
|
||
- **Credit mechanics:** 1 Hypercredit = $0.05; balance queryable via `GET /v1/credits` (returns `{"balance": 100}`)
|
||
- **Catalog pricing (cheap models):** deepseek-v4-flash $0.20/$0.40 per 1M in/out, qwen3.7-flash $0.20/$0.80, gemma-4-26b $0.12/$0.42
|
||
- **No config changes made** — research only; `charm-hyper` is available but not wired as a Hermes default
|
||
- **Status:** Candidate for free-tier OmniRoute combo; passthrough model import means zero maintenance
|
||
|
||
### GPT-5.6 Luna — Model Research (2026-08-14)
|
||
- **Community consensus:** Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper
|
||
- **Luna strengths:** visual/UI work, final code review, bounded high-stakes tasks, cleaner structured output
|
||
- **DeepSeek strengths:** automation, persistent executor work, high-volume subtasks, long-context economics
|
||
- **Key caveat:** Provider route matters more than model name — `openai-codex/gpt-5.6-luna`, `openrouter/openai/gpt-5.6-luna`, `deepseek/deepseek-v4-flash`, and `openrouter/deepseek/deepseek-v4-flash` are four different tests
|
||
- **Pricing:** Direct OpenAI $0.20/M input, $1.20/M output; OpenRouter $0.10/M input, $0.60/M output
|
||
- **Context:** 1.05M tokens, 128K output, function calling, structured outputs, image input, configurable reasoning effort
|
||
- **Recommendation:** Luna plans and reviews; DeepSeek executes — but test on a clean direct route before judging
|
||
|
||
### Guanaco v0.8.14 (2026-08-14)
|
||
- Updated from v0.8.11 on CT205 — three incremental bug-fix releases focused on DSML/tool-call output hygiene
|
||
- DSML tag leakage fixes: malformed tool-call output no longer leaks raw syntax junk into model context
|
||
- Dashboard performance fix: cached content size, fixed wrong column names in history query
|
||
- No new features, no breaking changes — low-risk update
|
||
- See [[systems/guanaco]]
|
||
|
||
### DeepSeek V4 Flash 0731 (2026-07-31)
|
||
- **Same model ID:** `deepseek-v4-flash` — no ID change, just a post-training bump
|
||
- **Same architecture:** 284B total params, 13B active per token, 1M context window
|
||
- **Same pricing:** $0.14/M input, $0.28/M output
|
||
- **Benchmark jump:** Terminal Bench 2.1: 82.7 (up from 61.8 preview — 34% relative gain)
|
||
- **Now surpasses V4-Pro preview** (72.1) on agent/coding benchmarks
|
||
- **Stricter tool name validation:** Rejects tool names not matching `^[a-zA-Z0-9_-]+$` — may cause issues with OmniRoute-transformed tool names
|
||
- **Impact:** Your existing cron jobs using `deepseek-v4-flash` get the upgrade automatically. The empty-stream fallback issue (`opencode-go/deepseek-v4-flash` returning empty 200s) is a provider plumbing problem, not a model quality problem.
|
||
|
||
### OmniRoute Free-DeepSeek Silent-Stop Mitigation (2026-08-04)
|
||
- Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on `deepseek-flash-free` and `smart-route`/`good-free` across both instances
|
||
- Reworked both combos: strategy `priority` → `fill-first`, dropped proven culprits (`238d5978/deepseek-v4-flash` + `-0731`, `nvidia` deepseek), healthy free models promoted, deepseek demoted to last resort
|
||
- See [[incidents/2026-08-04-omniroute-free-deepseek-silent-stops]]
|
||
|
||
## Context Enforcement
|
||
- Three layers: model native → combo `data.config.context_length` → Hermes client cap
|
||
- Effective context is the minimum of the three
|
||
|
||
## Combo Behavior
|
||
- `priority` retries same provider up to `maxRetries`, then errors; it does NOT fall through
|
||
- `fill-first` walks the list on failure
|
||
- `auto`/`lkgp` recalculates per request using health scoring
|
||
- Same-provider entries in a combo share rate-limit buckets
|
||
- Session affinity caches provider in `session_model_history`
|
||
|
||
## Related
|
||
- [[concepts/model-routing]]
|
||
- [[systems/omniroute]]
|
||
- [[systems/opencode-go]]
|