Files
agent-estate-wiki/systems/push-notifications.md

5.6 KiB

title, type, status, created, updated, verified_on, review_after, confidence, tags, sources
title type status created updated verified_on review_after confidence tags sources
Push Notifications (Cadu / Vory bridge) system degraded 2026-10-19 2026-11-10 2026-11-10 2026-11-24 medium
push
notifications
plugins
gateway
integrations
live-probe
live-config
live-push-dir
live-logs

Push Notifications (Cadu / Vory bridge)

Re-verified 2026-11-10 (nightly audit). Bridge file present and unchanged: hermes_push.py 91,765 B, sha256 prefix 59cabfcc…. Plugins hermes-push, vory-push, cadu-device, cadu-mail, cadu-rich-cards, cadu-secrets-vault all present under ~/.hermes/plugins/. :9119 listening (dashboard, pid 3628580) and tailnet :8443 listening. vory-push started 2026-11-10 01:00:34. ⚠ Pitfall for future audits: the relay runs embedded in another Hermes process — its log reads "vory-push relay running in another Hermes process; waiting for it to stop" — so a pgrep on hermes_push/vory script names returns nothing while the relay is healthy. Do not read that empty pgrep as "bridge down". Status left degraded (unchanged; no fix verified this window).

Purpose

Delivers Hermes activity to Anthony's phone (APNs push) and carries remote answer/approval traffic back to the gateway. Two plugin layers plus a standalone bridge script.

Components

Piece Where Notes
hermes-push plugin (enabled) Cadu push layer — device delivery
cadu-device, cadu-mail, cadu-rich-cards, cadu-secrets-vault plugins (enabled) Cadu device/mail/card/vault surfaces
vory-push plugin (enabled) runs the push bridge alongside the gateway
hermes_push.py ~/.hermes/push/hermes_push.py (91 KB, v1.0.37, sha256 59cabfcc…) the bridge itself; connects to the gateway's /api/ws as an ordinary client
install.sh ~/.hermes/push/install.sh installer (Vory)
hermes-push.conf ~/.hermes/push/hermes-push.conf written by Vory, read by the bridge via HERMES_PUSH_CONFIG

hermes-push.conf holds HERMES_PUSH_GATEWAY_URL, HERMES_PUSH_PUBLIC_URL, HERMES_PUSH_GATEWAY_BEARER, HERMES_PUSH_GATEWAY_REFRESH_TOKEN and HERMES_PUSH_DEVICES_DIR. The two tokens are JWT-shaped secrets and are deliberately not recorded here. State dirs: devices/, goals/, origins/, plus status.json and relay.lock.

Config (live config.yaml): plugins.enabled lists cadu-device, cadu-mail, cadu-rich-cards, cadu-secrets-vault, hermes-push, vory-push (alongside platforms/mattermost, homeassistant, hermes-browser-stream, a2a-platform); under platforms:, push.gateway_restart_notification: false and relay.enabled: true.

Status file — how to read it

~/.hermes/push/status.json is rewritten every retry cycle. Fields seen live: version, script_sha256, pid, updated_at, profile, connected, gateway, transport, error.

Current state (live, 2026-10-19 00:18 AWST) — BROKEN, and it has been for a while

{"version": "1.0.37", "pid": 2496376, "profile": "default", "connected": false,
 "gateway": "https://hermes.kangaroo-eel.ts.net", "transport": "",
 "error": "https://hermes.kangaroo-eel.ts.net: connection refused (nothing listens there on this machine)"}
  • Cause (verified): hermes.kangaroo-eel.ts.net resolves on this host to 127.0.1.1 (getent hosts → 127.0.1.1 hermes.kangaroo-eel.ts.net hermes, from the host's /etc/hosts convention), so the bridge dials loopback port 443 — where nothing listens. /etc/hosts wins over the tailnet DNS name.
  • The gateway and the Serve route are both healthy — this is a resolution/dial-target problem, not a gateway outage: the gateway listens on 0.0.0.0:9119 (curl http://127.0.0.1:9119/ → 302), tailscale serve status still shows https://hermes.kangaroo-eel.ts.net (tailnet only) with / → http://127.0.0.1:9119, and the public name is served on Funnel :8443.
  • Chronic, not a blip: the connection refused (nothing listens there on this machine) string appears 23,469 times across the retained log set (agent.log, agent.log.1-3, errors.log, errors.log.1-2). relay.lock was last touched 2026-10-19 00:03.
  • Impact: phone push delivery and any remote answer/approve path through the bridge are down. Nothing else observably depends on it — no cron job's delivery uses it.

Fix options (diagnosed, NOT applied)

  1. Point HERMES_PUSH_GATEWAY_URL at an address that actually answers from this host — http://127.0.0.1:9119 (the gateway's own listener) or the tailnet IP (100.118.5.51:443, where tailscaled serves the name) — instead of the FQDN that resolves to 127.0.1.1.
  2. Or give the FQDN a resolvable answer on this host: remove/qualify the /etc/hosts line that maps hermes/hermes.kangaroo-eel.ts.net to 127.0.1.1, letting tailnet DNS resolve it.
  3. Either way, confirm WSS/WS scheme matches what the bridge expects before changing the value.

Not yet traced in this pass: which process launches the bridge and under which unit/service, and whether HERMES_PUSH_PUBLIC_URL also embeds the broken FQDN.

Relay contention (benign, recurring)

relay.lock serialises bridge instances. The log line

vory-push relay running in another Hermes process; waiting for it to stop

appears when a second process contends for the relay (last recorded 2026-10-15; relay.lock touched 2026-10-19 00:03). It resolves without intervention.

No secrets recorded; key names, hashes, ports, status codes and paths only.