--- title: Browser Automation type: concept status: active created: 2026-07-22 updated: 2026-10-29 verified_on: 2026-10-29 confidence: high tags: [concept, browser, camofox, steel] sources: [live-probe, session-history] --- # Browser Automation ## Idea Giving an agent a real, controllable web browser — as opposed to just an HTTP-fetch tool — so it can handle JS-heavy pages, logins, and visual verification. ## How this install implements it - **Engine:** Camoufox (Firefox-based, fingerprint-resistant) — see [[comparisons/camoufox-vs-chrome]] for why this over stock Chrome. - **Connection:** REST API **and** CDP WebSocket on the same service at `localhost:9377` (profile `anthony`). *Corrected 2026-10-13 — earlier revisions split this across `:9093` (REST) and `:9377` (WS); nothing listens on `:9093`.* - **Display:** headed, not truly headless — runs under `Xvfb :99` (1600×1200×24) with `fluxbox` as window manager, viewable live over VNC (see [[systems/headless-server]]). - **Config:** `cloud_provider: camofox`, `camera_provider: camofox` in `~/.hermes/config.yaml` - **Auth:** Persistent profile `anthony` — authenticated into Uber Eats, LinkedIn, etc. No cookie files needed. ## Why headed-under-Xvfb instead of headless Running a real X display (rather than a headless browser flag) lets a human watch/intervene via VNC and tends to reduce automation-detection fingerprints compared to headless-mode flags — consistent with using Camoufox in the first place. The tradeoff, flagged in [[systems/headless-server]], is that the VNC exposing this display currently has no password. ## The Steel Experiment (Jul 29–30) Steel Browser (Chromium 140 on CT201) was briefly the primary backend. It offered full CDP access and a built-in session viewer, but was reverted after: - CDP URL normalization issues (Docker internal networking required a plugin patch) - Empty-stream errors from the `opencode-go/deepseek-v4-flash` provider path - DeepSeek 0731's stricter tool name validation affected all providers **Lesson:** A local browser backend (same host, no network hop) has fewer failure modes than a remote Docker container. The `browser-steel` plugin **was deleted on 2026-10-29** and the Steel deployments on CT201 and CT333 were torn down, so there is no Steel fallback left to fall back to — Camofox is the only backend. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]]. ## Related - [[systems/browser-backend]] - [[comparisons/steel-vs-camoufox]] - [[comparisons/camoufox-vs-chrome]] - [[systems/headless-server]] - [[concepts/tool-calling]]