Incidents 5->2 (deepseek-v4-flash pair closed 03:05:39, Job Radar timeout 07:33:41); storm page re-tallied 11/12 closed with closure timeline. NEW: Camofox hit its cgroup ceiling inside the 07:01 Job Radar run (2.5G, swap exhausted, 383 tasks) - containment drop-in documented; browser-backend/steel-vs-camoufox/browser-automation corrected to REST :9377 on CT460 (was :9093 on CT450). Mnemosyne 96.7%/99.4% coverage green; 18 cron sessions, 0 interactive; retention floor 2026-10-07.
4.3 KiB
4.3 KiB
title, type, status, created, updated, verified_on, confidence, tags, sources
| title | type | status | created | updated | verified_on | confidence | tags | sources | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Steel Browser vs Camoufox | comparison | active | 2026-07-29 | 2026-10-13 | 2026-10-13 | high |
|
|
Steel Browser vs Camoufox
Purpose
Compare the two browser-automation backends available to Hermes, including the Jul 29–30 experiment and the decision to revert.
At a glance
| Steel Browser (CT201) | Camofox (systemd --user on CT460) | |
|---|---|---|
| In use | Disabled since 2026-07-30 | Yes — primary backend |
| Engine | Chromium 140 (Playwright) | Camoufox (Firefox-based) |
| Connection | CDP (ws:// via port 9223) |
REST API + CDP WebSocket, both localhost:9377 |
| API | REST at :3000 — create/destroy/query sessions |
Camoufox REST + CDP WebSocket |
| Headless | Native headless | Xvfb + VNC (headed, anti-fingerprint) |
| Container | Docker on CT201 | systemd --user unit on the Hermes host (CT460) |
| Stealth | Standard Chromium | Anti-detection fingerprinting |
| Session viewer | Built-in UI at /steel |
VNC at /vnc-camofox |
| Plugin | browser-steel plugin (disabled) |
Native Hermes browser.cloud_provider: camofox |
The Steel Experiment (Jul 29–30)
Why it was tried
- Steel promised full CDP access for richer browser control
- Built-in session viewer and management UI
- Docker-based, clean separation from Hermes host
Timeline
- Jul 29: Steel deployed on CT201 via Docker Compose, Chromium 140. Hermes configured with
browser.cloud_provider: steel,browser-steelplugin enabled. - Jul 29 (later): CDP connectivity issues surfaced — Hermes was reaching
0.0.0.0from the plugin's internal CDP URL. Required_normalize_cdp_url()patch in the Steel plugin to remap to100.96.244.39. - Jul 30: DeepSeek V4 Flash 0731 went live and rejected tool names that didn't match
^[a-zA-Z0-9_-]+$. The Steel plugin's tools (steel_scrape,steel_session_options) were fine, but the route through OmniRoute → Steel → CDP was unreliable with empty-stream errors. - Jul 30 (evening): Decision to revert — Camoufox re-enabled, browser-steel plugin disabled,
CAMOFOX_URLandCAMOFOX_API_KEYuncommented in.env. Steel left running on CT201 but Hermes no longer routes through it.
What broke
- CDP URL normalization required a plugin patch to handle Docker's internal networking
- Empty-stream errors from
opencode-go/deepseek-v4-flashprovider path — the combo strategy does NOT iterate on empty responses, they pass to Hermes's own fallback chain (the #1 cause of "combo works but Hermes falls back") - Tool name validation on DeepSeek 0731 was stricter — this affected all providers, not just Steel
Lesson learned
Steel is a capable backend but adds a network hop and another plugin layer. Camofox runs locally on the
Hermes host (CT460) with fewer failure modes. The browser-steel plugin still exists (disabled) if needed in future.
Detail
Camofox (current primary)
- Deployed: Persistent
systemd --userservice on the Hermes host (CT460),camofox-browser.service - Connection: REST API and CDP WebSocket at
localhost:9377, profileanthony, sessionKeyanthony - Limits: drop-in
MemoryHigh=2G/MemoryMax=3G/MemorySwapMax=1G— a 2026-10-12 browser-heavy cron run hit2.5Gwith swap fully consumed and had to be restarted (see systems/browser-backend) - Key advantage: Runs on the same host as Hermes — no network hop, no container networking to debug
- Auth: Already authenticated into services (Uber Eats, LinkedIn, etc.) — no cookie files needed
- Limitation: Headed via Xvfb, consumes display resources
Steel Browser (disabled but available)
- Deployed: 2026-07-29 on CT201, Docker, custom
steel-browser-api:fixedimage - Status: Still running on CT201, Hermes not connected
- CDP endpoint:
http://100.96.244.39:9223 - REST API:
http://100.96.244.39:3000 - UI:
https://runtipi.kangaroo-eel.ts.net/steel - Plugin:
browser-steelat/home/hermes/.hermes/plugins/browser-steel/— disabled,plugin_steeltoolset flaggedrequires_health=steel