Files
agent-estate-wiki/systems/browser-backend.md
T

12 KiB
Raw Blame History

title, type, status, created, updated, verified_on, confidence, tags, sources
title type status created updated verified_on confidence tags sources
Browser Backend system active 2026-07-22 2026-11-08 2026-11-08 high
browser
camofox
automation
live-probe
config
session-history

Browser Backend

Purpose

Browser automation for Hermes — web search, form filling, screenshot, CDP control, authenticated scraping.

Current Configuration (re-verified 2026-10-13)

2026-11-07/08 — the browser backend now ships as a plugin, not a core feature. The Hermes 0.22.5 → 0.23.0 update installed camofox v1.1.0 as an enabled plugin at ~/.hermes/plugins/camofox/ (plugin.yaml, camofox_tools.py, camofox_policy.py, camofox_vault.py, camofox_cli.py), confirmed live by hermes plugins list on 2026-11-07. No browser.* key was changed in this window — the sections below still describe the running configuration (systems/hermes-agent).

  • Primary backend: Camofox (Firefox-based, anti-fingerprint) — systemd --user unit camofox-browser.service on the Hermes host itself (the unit path is /home/hermes/.config/systemd/user/…, and the VNC route is named after CT460). The older CT450 attribution below was not reproducible on 2026-10-13 — see the correction note.
  • Provider config: browser.cloud_provider: camofox, camera_provider: camofox
  • REST API: :9377 — live-verified 2026-10-13 (node server.js, CAMOFOX_PORT=9377; LISTEN *:9377). The :9093 figure in earlier revisions of this page is stale — nothing was listening on 9093 on 2026-10-13.
  • VNC viewer: NovNC websockify on 127.0.0.1:6080 → 127.0.0.1:5900, published as https://ct460.kangaroo-eel.ts.net/vnc-camofox
  • Steel Browser (CT201): Fully removed on 2026-10-29 — plugin deleted, CT201 deployment torn down, Serve routes deleted. Camofox is the sole backend. See the 2026-10-29 correction below.

Camofox (local systemd --user service)

⚠ Earlier revisions titled this section "Camoufox (CT450)". Live-verified 2026-10-13: the service Hermes talks to is a systemd --user unit on the Hermes host (/home/hermes/.config/systemd/user/camofox-browser.service), reached at localhost:9377, with its VNC route published as …/vnc-camofox. Treat any "runs on CT450 / CT201" claim as unverified until re-probed.

  • Host: the Hermes host itself (CT460) — systemd --user unit resolved on hostname = hermes, no network hop
  • Deployment: Systemd service camofox-browser.service
  • Engine: Camoufox (custom Firefox build with anti-detection fingerprinting)
  • Display: Headed via Xvfb :99 (1600×1200×24) + fluxbox window manager
  • Auth: Persistent profile anthony — authenticated into Uber Eats, LinkedIn, etc.
  • Connection: REST API and WebSocket bridge both at localhost:9377 (see the port correction above)
  • Last verified: Jul 30 — browserConnected: true, sessions: 1, tabs: 1, active 200s on scroll/navigate

Access

Endpoint URL Used by
REST API http://localhost:9377 Hermes browser tools + direct REST (/tabs, /start, /sessions)
VNC (NovNC) https://ct460.kangaroo-eel.ts.net/vnc-camofox Visual session viewer (websockify 127.0.0.1:6080 → 127.0.0.1:5900)

⚠ Port correction (2026-10-13): earlier revisions listed the REST API on :9093 and a separate CDP WebSocket on :9377. Live probe: the only Camofox listener is node server.js on *:9377 (with CAMOFOX_URL=http://localhost:9377 in the unit). :9093 and :9378 were both closed. :9377 is the REST API and the WebSocket bridge — there is no separate 9093 service.

Config

  • cloud_provider: camofox in ~/.hermes/config.yaml
  • CAMOFOX_URL=http://localhost:9377 and CAMOFOX_API_KEY in ~/.hermes/.env
  • Unit environment (~/.config/systemd/user/camofox-browser.service): CAMOFOX_PORT=9377, CAMOFOX_PROFILE_DIR=/home/hermes/.camofox/profiles, ENABLE_VNC=1, HANDLER_TIMEOUT_MS=90000, NAVIGATE_TIMEOUT_MS=60000, SESSION_TIMEOUT_MS=600000, TAB_INACTIVITY_MS=300000, BROWSER_IDLE_TIMEOUT_MS=300000, Restart=always, RestartSec=5. (The unit also carries an API key value — deliberately not reproduced here.)
  • Profile: anthony (502913bf) — NOT persist-anthony. A clean public profile jobradar is used for job-board scraping.

Resource containment (live-verified 2026-10-13)

The unit carries a drop-in ~/.config/systemd/user/camofox-browser.service.d/resource-containment.conf:

[Service]
MemoryHigh=2G
MemoryMax=3G
MemorySwapMax=1G
OOMPolicy=kill

Observed ceiling event — 2026-10-12 07:01–07:23 (Daily Marketing Job Radar). A long browser run drove the service to the top of that containment window:

  • Mid-run: Memory: 2.5G (high: 2G, max: 3G, swap max: 1G, available: 0B, peak: 2.5G, swap: 1023.9M, swap peak: 1G), 383 tasks, uptime 9 days. Swap fully consumed, no reclaimable headroom.
  • Symptom: the REST /tabs endpoint and page evals began hanging (they look like ordinary tool timeouts).
  • Recovery: restart camofox-browser.service, then work one tab at a time, closing each before opening the next. Post-restart the host sat at ~2.1 GB of 14 GB.

Diagnosis: the limits held and OOMPolicy=kill never had to fire — the workload exceeded the cgroup, not a leak. A green cron row therefore does not mean the browser layer was healthy; check systemctl --user status camofox-browser.service (peak/swap/tasks) whenever a browser-heavy job runs long.

Restart (systemctl --user restart camofox-browser.service) is measurable roughly 16 h later as a healthy baseline: Memory: 16.5M cgroup, peak: 1.4G, swap peak: 234.5M, 16 tasks, browserConnected: false until the next session starts.

Steel Browser (CT201) — Removed (2026-10-29)

  • Status: Fully removed. Nothing Steel-related survives on CT201 or CT333 — containers, images, network, project dir and Serve routes are all gone. Camofox is the only browser backend.
  • Plugin: browser-steel — deleted (~/.hermes/plugins/browser-steel/ no longer exists; no browser-steel key in plugins.disabled or plugins.entries)
  • CDP / REST / UI: retired — the former CDP :9223, REST :3000 and /steel UI addresses no longer serve anything
  • Reversibility: the compose file and Dockerfile are archived on CT201 under /root/_archived-steel/, and the images remain re-pullable from ghcr.io
  • See: comparisons/steel-vs-camoufox for the full experiment story

History

  • 2026-07-22: Camoufox primary, cua-driver bridge

  • 2026-07-29: Steel Browser deployed on CT201, became primary

  • 2026-07-30: Steel reverted — CDP issues + DeepSeek 0731 tool name validation problems. Camoufox restored as primary.

  • 2026-08-02: Camoufox confirmed working with authenticated sessions (Uber Eats report via Camofox REST API)

  • 2026-10-12 — containment ceiling hit under a browser-heavy cron run. Daily Marketing Job Radar drove camofox-browser.service to 2.5G with swap fully consumed (available: 0B, 383 tasks, 9-day uptime); /tabs and evals hung until the service was restarted at 07:23:50 and the job finished one tab at a time. Workload problem, not a leak — limits (MemoryHigh=2G/MemoryMax=3G/MemorySwapMax=1G) held.

  • 2026-10-13 — page corrected against a live probe. REST port :9093 → :9377; :9378 (claimed "Camoufox UI") does not exist; CT450 host attribution replaced with the local systemd --user unit on the Hermes host (CT460). Containment drop-in documented. Same day, the 07:01 Job Radar session was audited end-to-end and its memory-pressure evidence added above.

  • 2026-10-29 — Steel fully removed. Config back to cloud_provider: camofox / camera_provider: camofox, the browser-steel plugin deleted, STEEL_BASE_URL dropped from ~/.hermes/.env, and the Steel deployments on CT201 (containers, images, network, project dir, Serve routes) and CT333 (container + image) torn down. Camofox is the sole backend.

  • 2026-11-02 — the HTTP server wedged at the start of a browser-heavy cron run; a restart cleared it. The 07:00 Job Radar Stage 1 (dca8482e4f76) found the REST server accepting nothing — requests timing out with no lines reaching the service log — before its first tab was opened. systemctl --user restart camofox-browser.service cleared it and the run went on to work four boards. Live re-probe 2026-11-03 00:16:14 AWST: unit active, MainPID 953299, ActiveEnterTimestamp Mon 2026-11-02 18:11:04, MemoryCurrent 37.5 MB; /health → 200 {"ok":true,"engine":"camoufox","browserConnected":false,"browserRunning":false,"activeTabs":0,"activeSessions":0,"consecutiveFailures":0}; /tabs → {"running":true,"tabs":[]}. Idle-clean and far inside the containment drop-in. ⚠ This is a different fault from the 2026-10-12 ceiling: that one was memory exhaustion under many open tabs (2.5G, swap fully consumed); this one was a hung HTTP server at 31 MB RSS, so the containment limits are not the control that applies. A health probe inside browser-heavy scripts should therefore test responsiveness, not just /health status code.

Correction (2026-10-29) — Steel is out of the stack, not just disabled

Verified against live state at 2026-10-29. The earlier 00:3x probe said only "not reachable"; the truth is stronger — Steel is fully removed, not merely down:

  • Hermes is back on Camofox, and Camofox is the sole backend. The browser: block in ~/.hermes/config.yaml reads exactly cloud_provider: camofox, camera_provider: camofox, engine: auto, cdp_url: "".
  • The browser-steel plugin is deleted — ~/.hermes/plugins/browser-steel/ no longer exists, and plugins.disabled / plugins.entries carry no browser-steel key, so the plugin_steel toolset cannot be selected. Hermes core source contains zero Steel references, so there is no Steel fallback path in the tree. Backups: config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z, config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z.
  • CT201 is clean. The containers steel-browser-api-1 and steel-browser-ui-1, the steel-network Docker network, the images steel-browser-api:fixed / ghcr.io/steel-dev/steel-browser-api:latest / ghcr.io/steel-dev/steel-browser-ui:latest, and the project dir /root/steel-browser were all removed (~6 GB reclaimed; disk 70G used → 64G used). The Tailscale Serve routes /steel and /steel-api were deleted with tailscale serve --set-path <path> off; CT201's remaining routes are only / → 127.0.0.1:8085 and /mgr → 127.0.0.1:8084, on both :443 and :8443. No listeners remain on 3000, 5173 or 9223.
  • CT333's second, separate Steel copy is gone too. The orphaned steel-browser container (image ghcr.io/steel-dev/steel-browser:latest, 5.53 GB, created 2026-05-30, publishing port 3000) and its image were removed; no Steel containers, images or listeners remain on CT333.
  • Reversibility: the compose file and Dockerfile are archived on CT201 under /root/_archived-steel/, plus /root/_archived-steel/steel-cache-archive-20261029.tgz; the images remain re-pullable from ghcr.io.
  • Camofox is healthy and working — REST http://100.118.5.51:9377 (and http://localhost:9377); /health returns 200, /tabs returns {"running":true,"tabs":[]}.
  • Stale environment variable — now cleared from the gateway. STEEL_BASE_URL is no longer in ~/.hermes/.env. It survived in the environment of already-running processes (inherited at spawn), so the gateway was restarted on 2026-10-29 01:21 AWST to clear it: the new gateway tree (PID 3913841) reads STEEL_BASE_URL=0. A few older, unrelated services still carry the variable in their inherited environment (the dashboard unit and agentmail-webhook.service) — no core Hermes code reads STEEL_BASE_URL, so it is inert either way, and it is a runtime-only leftover, not a config value.

⚠ Superseded (the 2026-10-29 00:3x probe): an earlier version of this correction described Steel as merely "not reachable — endpoint dark, still running on CT201". That is outdated: Steel is now removed, not down.