Root cause of the long-standing cloud_provider: steel residue: the 2026-07-30 rollback disabled the browser-steel plugin and set browser.engine: camofox but left browser.cloud_provider: steel in config.yaml. The split config was inert-but-wrong, so it survived ~3 months of audits unnoticed. Corrects pages that still described Steel as present, running or configured: - systems/browser-backend.md (incl. gateway-restart env-var note) - comparisons/steel-vs-camoufox.md, comparisons/camoufox-vs-chrome.md - decisions/rollback-steel-to-camofox.md, decisions/index.md - infrastructure/docker-services.md, infrastructure/tailscale.md, infrastructure/hosts.md - systems/homepage-dashboard.md, systems/scheduled-tasks.md - concepts/browser-automation.md, runbooks/recover-docker-service.md - current-state.md, index.md, agents/nanobot.md - log.md (dated entry; supersedes the 00:3x audit's 'endpoint dark' wording) Also restores the Ignis compose row in infrastructure/docker-services.md, which the 2026-10-11 revision dropped (verified: /home/hermes/docker/ignis/docker-compose.yml exists and the ignis container is Up on CT460). Co-Authored-By: Hermes <hermes@local>
110 lines
7.3 KiB
Markdown
110 lines
7.3 KiB
Markdown
---
|
||
title: Rollback Steel to Camoufox
|
||
type: decision
|
||
status: active
|
||
created: 2026-08-02
|
||
updated: 2026-10-29
|
||
verified_on: 2026-10-29
|
||
confidence: high
|
||
tags: [decision, browser, steel, camofox, rollback]
|
||
sources: [session-history, live-config]
|
||
---
|
||
|
||
# Decision: Rollback Steel Browser → Camoufox
|
||
|
||
## Date
|
||
2026-07-30
|
||
|
||
## Context
|
||
Steel Browser was deployed on CT201 on 2026-07-29 as the primary browser automation backend, replacing Camoufox. It ran for approximately 24 hours before being reverted.
|
||
|
||
## Options Considered
|
||
| Option | Pros | Cons |
|
||
|---|---|---|
|
||
| **Keep Steel** | Full CDP access, session viewer UI, Docker isolation | CDP URL normalization issues, network hop, empty-stream errors |
|
||
| **Revert to Camoufox** | Local (no network hop), already authenticated, fewer failure modes | Headed display consumes resources, no built-in session viewer |
|
||
| **Both (Steel primary, Camoufox fallback)** | Redundancy | Complexity, two backends to maintain, unclear which one Hermes uses per-request |
|
||
|
||
## Decision
|
||
**Revert to Camoufox as primary.** Steel plugin disabled but left running on CT201.
|
||
|
||
> ⚠ **Superseded 2026-10-29:** the "left running on CT201" half of this decision no longer holds — Steel is
|
||
> now **fully removed** from the estate. See the Update below.
|
||
|
||
> **Status check 2026-10-20 — the decision is *partially* undone in configuration.** *(superseded 2026-10-29)*
|
||
> The live `browser:` block in `~/.hermes/config.yaml` read
|
||
> `cloud_provider: steel` with `engine: camofox`. The **engine** was Camoufox (decision honoured),
|
||
> but the **cloud_provider** key said `steel`, and `camera_provider: camofox`.
|
||
> This is the same split seen in [[comparisons/camoufox-vs-chrome]].
|
||
>
|
||
> Meanwhile the Steel service itself was **not reachable**: `http://steel.martinwa.org` returned
|
||
> **000** (connection failed) on 2026-10-20. Combined with the degraded CT201 stack recorded in
|
||
> [[systems/homepage-dashboard]] (main container removed, asset proxy 502), it looked like the
|
||
> "left running on CT201" part of this decision was no longer true.
|
||
>
|
||
> **Recommendation, not actioned here:** the `cloud_provider: steel` value appeared to be residue
|
||
> from the reverted deployment. Confirm against the live Steel service and either restore it or
|
||
> set `cloud_provider: camofox` so the config matches the decision this page records.
|
||
>
|
||
> **Resolved 2026-10-29:** `cloud_provider` is `camofox` again and Steel has been removed entirely — no
|
||
> restoration was needed or wanted.
|
||
|
||
## Rationale
|
||
1. **Reliability:** Camoufox runs on the same host as Hermes (CT450) — no Docker networking, no CDP URL normalization patches, no container-to-host routing issues.
|
||
2. **Auth persistence:** Camoufox's persistent profile `anthony` is already authenticated into key services (Uber Eats, LinkedIn). Steel required cookie re-exports.
|
||
3. **Failure mode:** The `opencode-go/deepseek-v4-flash` empty-stream errors were hitting all providers, but Steel added an extra failure surface (CDP connection, plugin layer) on top of the same problem.
|
||
4. **DeepSeek 0731 compatibility:** The new model's stricter tool name validation (`^[a-zA-Z0-9_-]+$`) affected all providers — not a Steel-specific issue, but another reason to minimise the toolchain.
|
||
|
||
## Consequences
|
||
- **Positive:** Browser automation is stable again. Uber Eats report ran successfully via Camofox on 2026-08-01.
|
||
- **Negative:** Steel's session viewer UI and full CDP access are no longer available to Hermes.
|
||
- **Neutral (2026-07-30 — now superseded):** Steel still ran on CT201 and could be re-enabled by uncommenting `.env` lines and enabling the plugin. **As of 2026-10-29 that is no longer true** — Steel is removed, so re-enabling means redeploying it (compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`; images remain re-pullable from ghcr.io).
|
||
|
||
## Evidence
|
||
- Camoufox verified working Jul 30: `browserConnected: true`, `sessions: 1`, `tabs: 1`, active 200s
|
||
- Uber Eats report (Aug 1) successfully scraped via Camofox REST API — 11 orders, $264.51
|
||
- Steel plugin `browser-steel` was present at `/home/hermes/.hermes/plugins/browser-steel/` — disabled *(2026-07-30 state; the plugin has since been deleted — see the Update below)*
|
||
|
||
## Update (2026-10-29) — Steel was fully removed, not just darkened
|
||
|
||
The 2026-07-30 rollback disabled the Steel plugin but left it on disk and left Steel running. Both of those
|
||
were closed on **2026-10-28 22:00 – 2026-10-29 00:15**: `browser.cloud_provider` / `camera_provider` were
|
||
returned to **`camofox`** (backups `config.yaml.bak-pre-cloudprovider-fix-20261028T220057Z`,
|
||
`config.yaml.bak-pre-steel-plugin-removal-20261028T161324Z`), and **`~/.hermes/plugins/browser-steel/` was
|
||
removed**, so the `plugin_steel` toolset is no longer selectable.
|
||
|
||
**Steel is now FULLY REMOVED — it is not merely "down" or "not reachable":**
|
||
|
||
- **Hermes config:** the `browser:` block reads exactly `cloud_provider: camofox`, `camera_provider: camofox`,
|
||
`engine: auto`, `cdp_url: ""`. No `browser-steel` key remains in `plugins.disabled` or `plugins.entries`, and
|
||
`STEEL_BASE_URL` is gone from `~/.hermes/.env` (it survived only in the inherited environment of processes
|
||
already running at the cleanup; the **gateway was restarted on 2026-10-29 01:21 AWST** to clear it, while a
|
||
couple of older, unrelated services still carry it — **no core Hermes code reads it**, so it is a
|
||
runtime-only leftover, not a config value).
|
||
- **Hermes plugins:** `~/.hermes/plugins/browser-steel/` is deleted. Hermes core source contains **zero**
|
||
Steel references, so there is **no Steel fallback path in the tree** — returning to Steel means redeploying
|
||
the plugin, not flipping a flag.
|
||
- **CT201 (runtipi):** containers `steel-browser-api-1` and `steel-browser-ui-1`, the `steel-network` Docker
|
||
network, the images `steel-browser-api:fixed` / `ghcr.io/steel-dev/steel-browser-api:latest` /
|
||
`ghcr.io/steel-dev/steel-browser-ui:latest`, and the project dir `/root/steel-browser` were all removed
|
||
(~6 GB reclaimed; disk 70G used → 64G used). The Tailscale Serve routes `/steel` and `/steel-api` were
|
||
deleted with `tailscale serve --set-path <path> off`; CT201's remaining routes are only `/` → `127.0.0.1:8085`
|
||
and `/mgr` → `127.0.0.1:8084` on both `:443` and `:8443`. Nothing listens on 3000, 5173 or 9223.
|
||
- **CT333 (clawtest):** a second, separate and orphaned Steel install — the `steel-browser` container
|
||
(`ghcr.io/steel-dev/steel-browser:latest`, 5.53 GB, created 2026-05-30, publishing port 3000) and its image
|
||
were removed as well. It had no Tailscale Serve route, no entry in `~/.zeroclaw/config.toml` (which uses
|
||
`backend = "agent_browser"`), and no reference from Nanobot — its last CDP navigation was 2026-10-20 and it
|
||
received no requests after 2026-10-21.
|
||
- **Reversibility:** the compose file and Dockerfile are archived on CT201 under `/root/_archived-steel/`, plus
|
||
`/root/_archived-steel/steel-cache-archive-20261029.tgz`; the images remain re-pullable from ghcr.io.
|
||
- **Camofox is the sole backend** and healthy — REST `http://100.118.5.51:9377` (also `http://localhost:9377`),
|
||
`/health` → **200**, `/tabs` → `{"running":true,"tabs":[]}`.
|
||
|
||
The decision itself is unchanged — Camofox is the backend — but the "Steel is still there if you need it"
|
||
assumption is dead. Detail: [[systems/browser-backend]], [[comparisons/steel-vs-camoufox]].
|
||
|
||
## Related
|
||
- [[comparisons/steel-vs-camoufox]]
|
||
- [[systems/browser-backend]]
|
||
- [[concepts/browser-automation]]
|