Files
agent-estate-wiki/systems/push-notifications.md
T

97 lines
4.8 KiB
Markdown

---
title: Push Notifications (Cadu / Vory bridge)
type: system
status: degraded
created: 2026-10-19
updated: 2026-10-19
verified_on: 2026-10-19
review_after: 2026-10-26
confidence: medium
tags: [push, notifications, plugins, gateway, integrations]
sources: [live-probe, live-config, live-push-dir, live-logs]
---
# Push Notifications (Cadu / Vory bridge)
## Purpose
Delivers Hermes activity to Anthony's phone (APNs push) and carries remote
answer/approval traffic back to the gateway. Two plugin layers plus a standalone bridge script.
## Components
| Piece | Where | Notes |
|---|---|---|
| `hermes-push` | plugin (enabled) | Cadu push layer — device delivery |
| `cadu-device`, `cadu-mail`, `cadu-rich-cards`, `cadu-secrets-vault` | plugins (enabled) | Cadu device/mail/card/vault surfaces |
| `vory-push` | plugin (enabled) | runs the push bridge alongside the gateway |
| `hermes_push.py` | `~/.hermes/push/hermes_push.py` (91 KB, **v1.0.37**, sha256 `59cabfcc…`) | the bridge itself; connects to the gateway's `/api/ws` as an ordinary client |
| `install.sh` | `~/.hermes/push/install.sh` | installer (Vory) |
| `hermes-push.conf` | `~/.hermes/push/hermes-push.conf` | written by Vory, read by the bridge via `HERMES_PUSH_CONFIG` |
`hermes-push.conf` holds `HERMES_PUSH_GATEWAY_URL`, `HERMES_PUSH_PUBLIC_URL`,
`HERMES_PUSH_GATEWAY_BEARER`, `HERMES_PUSH_GATEWAY_REFRESH_TOKEN` and `HERMES_PUSH_DEVICES_DIR`.
**The two tokens are JWT-shaped secrets and are deliberately not recorded here.** State dirs:
`devices/`, `goals/`, `origins/`, plus `status.json` and `relay.lock`.
Config (live `config.yaml`): `plugins.enabled` lists `cadu-device`, `cadu-mail`, `cadu-rich-cards`, `cadu-secrets-vault`, `hermes-push`, `vory-push` (alongside `platforms/mattermost`, `homeassistant`, `hermes-browser-stream`, `a2a-platform`); under `platforms:`, `push.gateway_restart_notification: false` and `relay.enabled: true`.
## Status file — how to read it
`~/.hermes/push/status.json` is rewritten every retry cycle. Fields seen live:
`version`, `script_sha256`, `pid`, `updated_at`, `profile`, `connected`, `gateway`, `transport`, `error`.
## Current state (live, 2026-10-19 00:18 AWST) — BROKEN, and it has been for a while
```json
{"version": "1.0.37", "pid": 2496376, "profile": "default", "connected": false,
"gateway": "https://hermes.kangaroo-eel.ts.net", "transport": "",
"error": "https://hermes.kangaroo-eel.ts.net: connection refused (nothing listens there on this machine)"}
```
- **Cause (verified):** `hermes.kangaroo-eel.ts.net` resolves **on this host to `127.0.1.1`**
(`getent hosts` → `127.0.1.1 hermes.kangaroo-eel.ts.net hermes`, from the host's `/etc/hosts`
convention), so the bridge dials loopback port 443 — where nothing listens. `/etc/hosts` wins over
the tailnet DNS name.
- **The gateway and the Serve route are both healthy** — this is a resolution/dial-target problem,
not a gateway outage: the gateway listens on `0.0.0.0:9119` (`curl http://127.0.0.1:9119/` → **302**),
`tailscale serve status` still shows `https://hermes.kangaroo-eel.ts.net (tailnet only)` with
`/ → http://127.0.0.1:9119`, and the public name is served on Funnel `:8443`.
- **Chronic, not a blip:** the `connection refused (nothing listens there on this machine)` string
appears **23,469** times across the retained log set (`agent.log`, `agent.log.1-3`,
`errors.log`, `errors.log.1-2`). `relay.lock` was last touched 2026-10-19 00:03.
- **Impact:** phone push delivery and any remote answer/approve path through the bridge are down.
Nothing else observably depends on it — no cron job's delivery uses it.
### Fix options (diagnosed, NOT applied)
1. Point `HERMES_PUSH_GATEWAY_URL` at an address that actually answers from this host —
`http://127.0.0.1:9119` (the gateway's own listener) or the tailnet IP
(`100.118.5.51:443`, where `tailscaled` serves the name) — instead of the FQDN that resolves to
`127.0.1.1`.
2. Or give the FQDN a resolvable answer on this host: remove/qualify the `/etc/hosts` line that maps
`hermes`/`hermes.kangaroo-eel.ts.net` to `127.0.1.1`, letting tailnet DNS resolve it.
3. Either way, confirm WSS/WS scheme matches what the bridge expects before changing the value.
**Not yet traced in this pass:** which process launches the bridge and under which unit/service, and
whether `HERMES_PUSH_PUBLIC_URL` also embeds the broken FQDN.
## Relay contention (benign, recurring)
`relay.lock` serialises bridge instances. The log line
```
vory-push relay running in another Hermes process; waiting for it to stop
```
appears when a second process contends for the relay (last recorded 2026-10-15; `relay.lock`
touched 2026-10-19 00:03). It resolves without intervention.
## Related
- [[systems/messaging-integrations]]
- [[infrastructure/tailscale]]
- [[systems/hermes-agent]]
- [[current-state]]
No secrets recorded; key names, hashes, ports, status codes and paths only.