93 lines
8.6 KiB
Markdown
93 lines
8.6 KiB
Markdown
---
|
||
title: OmniRoute
|
||
type: system
|
||
status: active
|
||
created: 2026-07-22
|
||
updated: 2026-08-14
|
||
verified_on: 2026-08-14
|
||
confidence: high
|
||
tags: [routing, provider, model, inference]
|
||
sources: [raw/configs/hermes-config-sanitized.txt]
|
||
---
|
||
|
||
# OmniRoute
|
||
|
||
## Purpose
|
||
Secondary provider/router for Hermes auxiliary roles and some agent routing.
|
||
|
||
## Current Configuration
|
||
- **Hermes provider name:** `custom:omni`
|
||
- **Base URL:** `http://omni:20128/v1`
|
||
- **Host reachability:** `omni.kangaroo-eel.ts.net` resolves to `100.93.204.4`; CT601 running, direct HTTP reachable
|
||
- **Runs on:** Proxmox CT601 (`omni`) — migrated from CT600 (`omniroute`, now stopped) on 2026-08-13
|
||
- **DB path:** `/opt/omniroute/storage.sqlite` (was `/var/lib/omniroute/storage.sqlite`)
|
||
- **Backup:** pre-migration config saved at `~/.hermes/config.yaml.bak.omniroute-removal`
|
||
- **Miss:** Hermes default provider path does not use OmniRoute for main chat; used for auxiliary roles instead
|
||
- **Auxiliary roles in Hermes config:** `skills_hub`, `approval`, `mcp`
|
||
- **Key prefix visible in config:** `sk-5c67fbdc0...` — full value not stored in wiki
|
||
|
||
## Catalog and Routing Facts
|
||
- Catalog reachable: yes, 47 combos returned at `/api/combos`
|
||
- Combo schema observed: `name`, `strategy`, `models`, `id`; some combos have `config`, `isHidden`, `uuid`
|
||
- Combo-level context can override individual models
|
||
- `auto`/`lkgp` strategies may silently change chosen models
|
||
- Model mappings can substitute models silently
|
||
- Session affinity caches last successful provider in `session_model_history`
|
||
|
||
## Catalog Evidence (2026-07-22)
|
||
- Advertised context for `gemini/gemini-3.6-flash`: `1048576`
|
||
- Related models in catalog: `kc/google/gemini-3.6-flash`, `kilocode/google/gemini-3.6-flash`, `nous/google/gemini-3.6-flash`, `opencode-zen/gemini-3.6-flash`, `vertex/gemini-3.6-flash`
|
||
- No live combo in the returned 47 combos referenced `gemini-3.6-flash`
|
||
|
||
## Known Issues
|
||
- opencode-zen proxy pitfall documented in omniroute-ops skill
|
||
- Web search proxy failures documented in omniroute-ops skill
|
||
- Combo discovery can differ from resolution;
|
||
|
||
## AutoRoute variants (2026-08-12)
|
||
- `auto/best-free` (Auto Cheap) and `auto/coding-free` are **virtual** AutoRoute variants — they do NOT appear in the persisted combo list (`/api/combos` returns 54 combos, none named these).
|
||
- The LKGP engine resolves them dynamically: it scores the operator's **live connected provider accounts** (e.g. `antigravity`, `opencode-zen`, `nvidia`, `openrouter`) rather than a hardcoded candidate list.
|
||
- Scoring signals: health, quota/token-reset windows, latency, consecutive errors, and cost-inverse across all connected accounts; failover to next-best on throttle/error without dropping the request.
|
||
- Example observed: an `auto/best-free` request routed to Antigravity (Google AI Plus) because that account was active, healthy, zero-cost, and highest priority — proving live dynamic scoring, not a static map.
|
||
|
||
## Recent Changes (2026-08-14)
|
||
- **OmniRoute factory reset on CT601.** Database wiped clean: 0 combos, 0 providers, 0 API keys, 115 built-in auto model entries (`auto/best-fast`, `auto/best-coding`, etc.). Old state backed up at `/opt/omniroute/factory-reset-backup-20260814-225524`.
|
||
- **Providers re-added from Notion API vault:** Anthropic, Google Gemini, Groq, Mistral, OpenRouter — all five passed live provider tests. OpenAI keys in the vault were tested but none worked; removed the invalid OpenAI entry.
|
||
- **Hermes config default model changed:** `auto/best-free` → `resilient-free` (via `~/.hermes/config.yaml`). `auto/best-free` was misconfigured — had no combo definition and routed to `gpt-5.6-sol` (paid) instead of free models. Switched to `resilient-free` which is a working free combo.
|
||
- **Multiple free combos broken by upstream model ID changes:**
|
||
- OpenRouter: `poolside/laguna-s-2-1:free` (dash) → `poolside/laguna-s-2.1:free` (dot) — model ID format changed
|
||
- Groq: `meta-llama/llama-4-scout-17b-16e-instruct` — **retired**, only llama-3.3/3.1 remain
|
||
- Affected combos: `resilient-free`, `free-IA`, `free-stack`, `good-free`, `deepseek-flash-free`, `mimo-free`
|
||
- All route through OpenRouter → fail with 400/404 when trying these model IDs
|
||
- **Working free combos as of 2026-08-14:** `free-lkgp` (DeepSeek V4 Flash), `mega-free` (StepFun 3.7 Flash), `smart-route` / `smart-route-ia` (nested combos)
|
||
- **LiteLLM routing issue identified:** Hermes was previously pointing at `http://litellm:4000/v1` with `free-auto` model, causing 6–26s per call. OmniRoute's `auto/best-free` tested at 265ms. Config now points at OmniRoute.
|
||
- **GPT-5.6 Luna research:** Community consensus — Luna is cleaner/more polished; DeepSeek V4 Flash is more persistent/cheaper. Luna better for visual/UI work and final review; DeepSeek better for automation and high-volume executor work. Provider route matters more than model name.
|
||
|
||
## Recent Changes (2026-08-13)
|
||
- **OmniRoute migrated CT600 → CT601.** Provider renamed `custom:omniroute` → `custom:omni`; base URL `http://omniroute:20128/v1` → `http://omni:20128/v1`. Host `omni.kangaroo-eel.ts.net` (100.93.204.4, active) replaces `omniroute.kangaroo-eel.ts.net` (100.88.81.19, offline). CT600 stopped, CT601 running. Database path `/var/lib/omniroute/storage.sqlite` → `/opt/omniroute/storage.sqlite`. API key unchanged. All 14 config + 22 cron-job `custom:omniroute` references updated to `custom:omni`. Pre-migration backup at `~/.hermes/config.yaml.bak.omniroute-removal`. Verified live 2026-08-13 (`pct list`: 600 stopped, 601 running; `grep`: 0 remaining `custom:omniroute` refs).
|
||
|
||
## Recent Changes (2026-07-25)
|
||
- **free-stack combo updated:** Nemotron 3 Ultra 550B model at index 4 now configured with NVIDIA-specific parameters:
|
||
- `max_tokens`: 8192 (was default 65536, exceeding NVIDIA NIM limit of 16384)
|
||
- `reasoning_effort`: medium (was high, reducing long internal monologues)
|
||
- `chat_template_kwargs.force_nonempty_content`: true (prevents empty responses when tool calling + reasoning)
|
||
- **Combo ID:** `23d25f8f-5eb0-4b6f-b966-e260ca3905b3`
|
||
- **Hermes config.yaml updated:** `agent.reasoning_effort: medium`, `agent.max_tokens: 8192`, `agent.chat_template_kwargs.force_nonempty_content: true`
|
||
- **Rationale:** NVIDIA NIM API documents max output of 16384 tokens; medium reasoning uses more efficient chat-template mode; force_nonempty_content resolves empty responses with tool calls combo in picker may still route
|
||
|
||
## Recent Changes (2026-08-07)
|
||
- **Three-layer root cause for slowness identified and fixed:**
|
||
1. **npm package .env override:** OmniRoute loads three .env files (`/var/lib/omniroute/.env`, `/root/.omniroute/.env`, `/usr/lib/node_modules/omniroute/.env`). The npm package's `.env` ships with `ENABLE_SOCKS5_PROXY=true` and `npm update` re-installs it with that default, silently overriding the `/var/lib` config. Fix: flip all three to `false`, then restart. After ANY upgrade, check all three.
|
||
2. **Dead provider connection:** Connection `58debd59` (prefix `nanobot`) points at `clawtest:8900/v1` — nanobot on clawtest is DOWN. Every model sync + credential health check retried it, burning event-loop time. Disabled it.
|
||
3. **PVE host load leaking into CT600:** CT600 has `nesting=1`, so `uptime`/`top` inside the container mirror PVE's host load exactly (verified: identical 10.30/9.23/10.37 on both). Same trap as CT241. Judge CT600 CPU from `pct exec 600 -- ps -o %cpu` or host-side `ps`, never from in-container load average.
|
||
- **First-request-after-cache-expiry stall:** `/v1/models` hangs 10-15s+ on the FIRST hit after the model catalog cache expires (rebuild enumerates all connections, including slow dead ones), then answers in ~0.02s from cache. Don't judge health on one curl — run a burst of 5 and check if later tries are sub-second.
|
||
- **`is_active=0` does NOT stop CredentialHealth probes:** Disabling a dead connection stops it from ModelSync but the health checker still probes it and logs `[ProxyFetch] ECONNREFUSED` noise. That noise is benign once the endpoint is unreachable.
|
||
- **Lessons banked in `omniroute-ops` skill:** `references/omniroute-upgrade-procedure.md` updated with all three pitfalls.
|
||
|
||
## Recent Changes (2026-08-04)
|
||
- **OmniRoute free-DeepSeek silent-stop mitigation:** Free-DeepSeek upstreams returning empty-after-tool responses since Aug 1 → silent turn stops on `deepseek-flash-free` and `smart-route`/`good-free` across both instances. Reworked both combos: strategy `priority` → `fill-first`, dropped proven culprits (`238d5978/deepseek-v4-flash` + `-0731`, `nvidia` deepseek), healthy free models promoted, deepseek demoted to last resort. See [[incidents/2026-08-04-omniroute-free-deepseek-silent-stops]].
|
||
|
||
## Related
|
||
- [[model-providers]]
|
||
- [[model-routing]]
|