Root cause of the long-standing cloud_provider: steel residue: the 2026-07-30 rollback disabled the browser-steel plugin and set browser.engine: camofox but left browser.cloud_provider: steel in config.yaml. The split config was inert-but-wrong, so it survived ~3 months of audits unnoticed. Corrects pages that still described Steel as present, running or configured: - systems/browser-backend.md (incl. gateway-restart env-var note) - comparisons/steel-vs-camoufox.md, comparisons/camoufox-vs-chrome.md - decisions/rollback-steel-to-camofox.md, decisions/index.md - infrastructure/docker-services.md, infrastructure/tailscale.md, infrastructure/hosts.md - systems/homepage-dashboard.md, systems/scheduled-tasks.md - concepts/browser-automation.md, runbooks/recover-docker-service.md - current-state.md, index.md, agents/nanobot.md - log.md (dated entry; supersedes the 00:3x audit's 'endpoint dark' wording) Also restores the Ignis compose row in infrastructure/docker-services.md, which the 2026-10-11 revision dropped (verified: /home/hermes/docker/ignis/docker-compose.yml exists and the ignis container is Up on CT460). Co-Authored-By: Hermes <hermes@local>
42 lines
2.6 KiB
Markdown
42 lines
2.6 KiB
Markdown
---
|
||
title: Browser Automation
|
||
type: concept
|
||
status: active
|
||
created: 2026-07-22
|
||
updated: 2026-10-29
|
||
verified_on: 2026-10-29
|
||
confidence: high
|
||
tags: [concept, browser, camofox, steel]
|
||
sources: [live-probe, session-history]
|
||
---
|
||
|
||
# Browser Automation
|
||
|
||
## Idea
|
||
Giving an agent a real, controllable web browser — as opposed to just an HTTP-fetch tool — so it can handle JS-heavy pages, logins, and visual verification.
|
||
|
||
## How this install implements it
|
||
- **Engine:** Camoufox (Firefox-based, fingerprint-resistant) — see [[comparisons/camoufox-vs-chrome]] for why this over stock Chrome.
|
||
- **Connection:** REST API **and** CDP WebSocket on the same service at `localhost:9377` (profile `anthony`). *Corrected 2026-10-13 — earlier revisions split this across `:9093` (REST) and `:9377` (WS); nothing listens on `:9093`.*
|
||
- **Display:** headed, not truly headless — runs under `Xvfb :99` (1600×1200×24) with `fluxbox` as window manager, viewable live over VNC (see [[systems/headless-server]]).
|
||
- **Config:** `cloud_provider: camofox`, `camera_provider: camofox` in `~/.hermes/config.yaml`
|
||
- **Auth:** Persistent profile `anthony` — authenticated into Uber Eats, LinkedIn, etc. No cookie files needed.
|
||
|
||
## Why headed-under-Xvfb instead of headless
|
||
Running a real X display (rather than a headless browser flag) lets a human watch/intervene via VNC and tends to reduce automation-detection fingerprints compared to headless-mode flags — consistent with using Camoufox in the first place. The tradeoff, flagged in [[systems/headless-server]], is that the VNC exposing this display currently has no password.
|
||
|
||
## The Steel Experiment (Jul 29–30)
|
||
Steel Browser (Chromium 140 on CT201) was briefly the primary backend. It offered full CDP access and a built-in session viewer, but was reverted after:
|
||
- CDP URL normalization issues (Docker internal networking required a plugin patch)
|
||
- Empty-stream errors from the `opencode-go/deepseek-v4-flash` provider path
|
||
- DeepSeek 0731's stricter tool name validation affected all providers
|
||
|
||
**Lesson:** A local browser backend (same host, no network hop) has fewer failure modes than a remote Docker container. The `browser-steel` plugin **was deleted on 2026-10-29** and the Steel deployments on CT201 and CT333 were torn down, so there is no Steel fallback left to fall back to — Camofox is the only backend. See [[comparisons/steel-vs-camoufox]] and [[decisions/rollback-steel-to-camofox]].
|
||
|
||
## Related
|
||
- [[systems/browser-backend]]
|
||
- [[comparisons/steel-vs-camoufox]]
|
||
- [[comparisons/camoufox-vs-chrome]]
|
||
- [[systems/headless-server]]
|
||
- [[concepts/tool-calling]]
|