wiki(audit): 2026-10-29 window — Conduit voice fault documented and repaired; radar two-stage split resolved both incidents (20 pages, 2 letters, Notion 455->475); cron 176/0

This commit is contained in:
Tony0410
2026-10-30 00:21:21 +08:00
parent cc95dcc54d
commit 1d335ca8e6
10 changed files with 404 additions and 82 deletions
+95 -50
View File
@@ -3,12 +3,12 @@ title: Conduit
type: system
status: active
created: 2026-10-24
updated: 2026-10-28
verified_on: 2026-10-28
updated: 2026-10-30
verified_on: 2026-10-30
review_after: 2026-11-08
confidence: high
tags: [conduit, plugin, phone, iphone, push, voice, memory, tool-calling]
sources: [live-probe, plugin-tree, live-sqlite-conduit, live-sqlite-cron, live-log, session-20261024_125329]
tags: [conduit, plugin, phone, iphone, push, voice, memory, tool-calling, session-ttl]
sources: [live-probe, plugin-tree, live-sqlite-conduit, live-sqlite-cron, live-log, session-20261024_125329, session-20260908_031541_49c051]
---
# Conduit
@@ -21,11 +21,18 @@ be started from the phone where the agent can speak and listen (`conduit-voice`
Related: [[systems/mnemosyne]] (memory backend the voice session reads), [[systems/hermes-ui]]
(the browser desktop), [[systems/push-notifications]] (the older `hermes-push` bridge).
## Runtime checks (2026-10-26 00:1x AWST)
- **Working tree:** M dashboard/plugin_api.py — the uncommitted memory-timeout patch is **still present**.
- **`conduit-calls.json` → `history` entries:** **0** — the call-request path remains unused
→ [[incidents/2026-10-24-conduit-call-request-refused]].
- **Tool use in-window:** `conduit_show` ×1 — the plugin works for cards; the open incident concerns *placing calls*.
## Runtime checks (2026-10-30 00:0x AWST)
- **`conduit-calls.json` → `history` entries: 2** — the first real calls ever recorded
(2026-10-29 08:38:04 and 08:49:36). The path is no longer unused; the earlier
*"0 calls"* reading is superseded. See [[incidents/2026-10-29-conduit-voice-silence]].
- **Calls still unproven end-to-end:** `last_request_at` remained `null` at the end of the
2026-10-29 window — no call is yet known to have run a real Hermes turn.
- **`ends` vs `history`:** `ends` records *any turn-end that fires a watch* (including cron
sessions) — never read it as a call log.
- **Device:** iPhone app version now **2.2.0** (was 1.10.1 on 2026-10-28), **19 settings keys**,
**no `voice.local.*` keys** → Local Live cannot be configured host-side any more.
- **Voice providers were both dead and are now fixed** — `tts.provider: ollama_local`,
`stt.provider: local` (2026-10-29). Details below and in the 10-29 incident.
## Where it lives
- **Plugin tree:** `/home/hermes/.hermes/plugins/conduit_push`
@@ -46,36 +53,42 @@ Related: [[systems/mnemosyne]] (memory backend the voice session reads), [[syste
- **Data stores** (`~/.hermes/`): `conduit-health.sqlite3`, `conduit-location.sqlite3`,
`conduit-phone.sqlite3`, plus JSON state — `conduit-push.json` (credential / gateway id /
installation id / relay url / e2e — **no values recorded here**), `conduit-calls.json`,
`conduit-settings.json`, `conduit-present.json`, `conduit-desktop-views.json`.
`conduit-settings.json`, `conduit-present.json`, `conduit-desktop-views.json`,
`conduit-link-status.json`, `conduit-local-live.json` (**holds an api key — never copy its
contents into the wiki**).
## Current state (verified 2026-10-25 00:2x–00:5x AWST)
- **Dashboard gateway (re-verified 2026-10-26):** listening on `0.0.0.0:9119`; process started **2026-10-25
18:42:33** — **restarted since the 2026-10-24 entry, so the upstream commits are now live** (the recorded
*"not restarted since / not live"* note is superseded, not wrong at the time) — so it is still the process that picked up the memory-timeout patch, and
the `0.64.0` → `0.69.0` upstream commits are **not** live.
- **Device:** one registered — **iPhone** (`id 70aa63f3…`, `kid c24e8215…`), app version `2.0.0`,
features `chat-cards`, `info-cards`, `live-cards`, `maps-navigate`, `media-cards`, `show-cards`;
tz `Australia/Perth`; sharing window 2026-10-16 → 2027-04-22. Paired **2026-10-22 12:54:39**.
`last_seen` / `last_sent` / `last_contact` all read **2026-10-23 19:10:30 AWST** (~29 h stale at
this audit) while health rows kept arriving until **2026-10-24 09:59:59** and window rollups were
recomputed at **13:56:35** — so those three fields track *phone-status* contact, not data flow.
(The 10-24 page recorded 19:02:16 from the same store; the epoch reads 19:10:35.)
## Current state (re-verified 2026-10-30 00:0x AWST)
- **Dashboard gateway:** listening on `0.0.0.0:9119`; **pid 4042003, started 2026-10-29 08:03:26**
— the restart that put the 30-day session TTL live (see below). The `0.64.0` → `0.69.0` upstream
commits remain **not** live (the process predates nothing relevant; the tree still only carries the
memory-timeout patch).
- **Device:** one registered — **iPhone**, app version **`2.2.0`** (self-updated from `1.10.1` on
2026-10-29), `seen_at` **2026-10-29 18:56:44 AWST**, `mode: small`, **19 settings keys**.
`last_seen` / `last_sent` / `last_contact` are `null` in the current store — those three fields
track *phone-status* contact, not data flow (health rows and link pings keep arriving regardless:
`conduit-link-status.json` `app_seen_at` **2026-10-29 23:53:15**, `push_ok_at` **18:50:00**).
Paired **2026-10-22 12:54:39**; sharing window 2026-10-16 → 2027-04-22.
- **Dashboard session TTL (raised 2026-10-29):** `dashboard.basic_auth.session_ttl_seconds` =
**2592000 (30 days)**, up from the **43200 (12 h)** default. A lapsed session was making Conduit's
call socket answer **WS 4401 "Dashboard sign-in required"** — silently. ⚠ `dashboard.session_ttl_seconds`
is a **separate key and still 43200**; check both.
- **Call settings:** `enabled`, `when_asked`, `decides`, `alerts` all true; rate limits
`min_gap_s 120` / `per_hour 6` / `per_day 20`; `avoid_unknown_place` false.
- **Recent calls — none.** `conduit-calls.json → history` is **empty**: no call has been recorded
since the memory-timeout fix landed, so that fix still has **no positive evidence** behind it. The
“two call sessions closed cleanly 00:03:23 / 00:06:57” claim previously on this page was a **false
positive** — those `ends` entries are the two **cron** sessions of that night, and `ends` records
any turn-end that fires a watch, not calls →
[[incidents/2026-10-24-conduit-call-request-refused]].
- **Data volume (2026-10-25 00:2x):** `conduit-health.sqlite3` **569.9 MB** (+8.2 MB in 25.7 h),
holding **1,520,229 `events`** (1,510,929 `sample` / 6,070 `sleep` / 3,230 `workout`),
**3,028 `buckets`** and **124 `windows`** for its single device; newest row **2026-10-24 09:59:59**,
newest `buckets.date` 2026-10-24; `conduit-location.sqlite3` 80 KB; `conduit-phone.sqlite3` 284 KB.
- **Recent calls — two, both 2026-10-29.** `conduit-calls.json → history` now reads **2**
(08:38:04, 08:49:36). The 08:38:04 call produced no Mac connection and no Hermes turn, and the
iPhone re-authenticated at 08:39:05 — the expired-session fault. The 08:49:36 call was placed after
the `classic` mode switch and the STT/TTS fixes, but the session ended one second earlier, so its
outcome is **unrecorded**. The memory-timeout patch therefore still has no *confirmed* call behind
it → [[incidents/2026-10-29-conduit-voice-silence]].
- **Data volume (2026-10-30 00:0x):** `conduit-health.sqlite3` **930.0 MB** (+360 MB in 5 days;
+8.2 MB/25.7 h as measured on 10-25 — the rate has not held, so treat growth as **ingestion-paced,
unbounded**), holding **2,601,565 `events`** (2,592,189 `sample` / 6,146 `sleep` / 3,230 `workout`),
**4,105 `buckets`** and **411 `windows`**; newest health row **2026-10-29 23:12:01 AWST**.
`conduit-location.sqlite3` 80 KB; `conduit-phone.sqlite3` 284 KB (last measured 10-25).
⚠ The 10-24 page recorded **308 `buckets` / 62 `windows`**; a direct `COUNT(*)` reads
**3,028 / 124**, so the earlier pair was a narrower grouping — quote these. **No retention or
compaction has been observed** — the store grows from ingestion alone and is worth watching against
the CT460 disk budget.
**4,105 / 411**, so the earlier pair was a narrower grouping — quote these. **No retention or
compaction has been observed**, and the store is now closing on 1 GB — watch it against the CT460
disk budget.
## Calls from Hermes (the `conduit_call_user` tool)
The plugin registers a **`conduit_call_user`** tool (toolset `conduit`, 📞) for every session and
@@ -88,20 +101,36 @@ the call rings when the asking turn ends and plays the reason first. Limits: `mi
⚠ **Known failure, 2026-10-24:** three requests to be phoned were refused by the agent, which
asserted the capability did not exist **without calling the tool or loading the skill** →
[[incidents/2026-10-24-conduit-call-request-refused]].
[[incidents/2026-10-24-conduit-call-request-refused]]. **Positive counter-example 2026-10-29:** in
that window the agent did look the tool up and did place calls (two entries in `history`), so the
failures were behavioural, not a missing capability.
## Voice pipeline (probed 2026-10-28)
## Voice pipeline (verified 2026-10-30)
Phone voice sessions ride the Mac mini speech server, which is also the Hermes TTS target:
Phone voice sessions ride the Mac mini speech server, which is also the Hermes TTS/STT target.
**All three settings below were wrong or dead as of 2026-10-28 and were fixed on 2026-10-29** —
see [[incidents/2026-10-29-conduit-voice-silence]].
- `voice.speech_server.url` = `wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/` — live-voice transport,
`wake_word.enabled: true`.
- `tts.provider: openai` → `https://anthonys-mac-mini.kangaroo-eel.ts.net/v1/speech` (model `phonon`,
voice `alloy`) — see [[systems/hermes-agent]].
- **Health check:** `GET https://anthonys-mac-mini.kangaroo-eel.ts.net/v1/pool` returned
`{"size": 2, "in_use": 0}` with both units `idle` on 2026-10-28 00:2x — a **2-unit pool**. `/v1/models` is
**not implemented** (`{"detail":"Not Found"}`); probe `/v1/pool`, not `/v1/models`.
| Setting | Value (now) | Was | Failure when wrong |
|---|---|---|---|
| `voice.speech_server.url` | `wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/realtime` | `…:8443/v1/` | **403** — the plugin dials the value **verbatim**, it does not append `/realtime` |
| `tts.provider` | `ollama_local` (`~/.hermes/scripts/ollama-tts.sh`, model `phonon`) | `openai` → Mac `/v1/speech` | **502** |
| `stt.provider` | `local` (`faster_whisper`, model `base`) | `groq` | **403 / Cloudflare error 1010** — a *block*, not a bad key (fails identically with a bogus key) |
- `voice.mode` = **`classic`** (switched from `local` 2026-10-29). Classic relays everything through
the plugin and needs **no phone-side server address** — which is why both host providers above had
to work. A Local Live path cannot be configured from the host at all: app **2.2.0**'s schema exposes
**19 keys, none of them `voice.local.*`**.
- `voice.auto_tts: false` — assistant replies are not auto-spoken; audio appears only when requested.
- `wake_word.enabled: true`.
- **Health check:** `GET https://anthonys-mac-mini.kangaroo-eel.ts.net/v1/pool` returned
`{"size": 2, "in_use": 0}` with both units `idle` on 2026-10-28 00:2x — a **2-unit pool**. `/v1/models`
is **not implemented** (`{"detail":"Not Found"}`); probe `/v1/pool`, not `/v1/models`.
- **End-to-end proof (2026-10-29):** driving the real call path produced `session.created` →
transcription of live speech → `response.done`, with the Mac counters moving **0 → 10 connections
/ 1 completed response / 2.98 s audio / 0 errors** and the plugin handshaking in **41 ms**.
- ⚠ **Fragile URL:** the Conduit settings tool writes Hermes config (it had flipped `tts.provider` to
`openai`). If it or an app update ever writes the bare `/v1/` back, every call silently 403s again.
## Memory integration
The voice session (`conduit-voice`) reaches Mnemosyne through the plugin's own memory layer, not
@@ -118,15 +147,27 @@ See [[incidents/2026-10-22-conduit-memory-start-timeout]] and [[systems/mnemosyn
git -C ~/.hermes/plugins/conduit_push log --oneline -1 && \
git -C ~/.hermes/plugins/conduit_push status --short
# is the gateway up, and since when?
# is the gateway up, and since when? (the restart that applies a session-TTL change)
ss -tlnp | grep 9119
# session TTLs — BOTH keys, they are different consumers
python3 -c "import yaml;c=yaml.safe_load(open('/home/hermes/.hermes/config.yaml'));\
print('basic_auth',c['dashboard']['basic_auth'].get('session_ttl_seconds'),\
'| dashboard',c['dashboard'].get('session_ttl_seconds'))"
# voice transport + providers
python3 -c "import yaml;c=yaml.safe_load(open('/home/hermes/.hermes/config.yaml'));\
print(c['voice']['speech_server']['url']);print(c['tts']['provider'],c['stt']['provider'])"
# did a voice call lose its memory context?
grep -h "Conduit" ~/.hermes/logs/agent.log | grep -i "memory" | tail
# did a real call happen? (history = calls; ends = turn-ends, NOT calls)
python3 -c "import json,os;d=json.load(open(os.path.expanduser('~/.hermes/conduit-calls.json')));print(len(d['history']),'calls;',len(d['ends']),'turn-ends')"
# is the phone actually reaching the relay?
cat ~/.hermes/conduit-link-status.json
# state files (no secrets in the wiki)
ls -la ~/.hermes/conduit-*.json ~/.hermes/conduit-*.sqlite3
```
@@ -134,16 +175,20 @@ ls -la ~/.hermes/conduit-*.json ~/.hermes/conduit-*.sqlite3
## Open items
- Commit or upstream the `plugin_api.py` timeout patch so a plugin update cannot revert it — it is
now five releases behind HEAD.
- **Verify a call end-to-end.** No call has been placed since the fix, so `MEMORY_TIMEOUT_S = 60` is
still unexercised by a real call, and the 10-24 refusal means the chat route has never worked →
[[incidents/2026-10-24-conduit-call-request-refused]].
- No retention policy on `conduit-health.sqlite3` (569.9 MB and growing).
- **Verify a call end-to-end from the phone.** The 2026-10-29 08:49:36 call has no recorded outcome,
so `MEMORY_TIMEOUT_S = 60` is still unexercised by a *confirmed* call, and `last_request_at` was
still `null` → [[incidents/2026-10-29-conduit-voice-silence]].
- Re-check `voice.speech_server.url` after any Conduit/app update or settings sync — a bare `/v1/`
silently 403s every call.
- No retention policy on `conduit-health.sqlite3` (**930 MB** and growing unbounded).
- The Mnemosyne cold start that motivated the patch is still slow (10–45 s); only the *budget* was
raised. → [[systems/mnemosyne]]
## Related
- [[incidents/2026-10-29-conduit-voice-silence]]
- [[incidents/2026-10-24-conduit-call-request-refused]]
- [[incidents/2026-10-22-conduit-memory-start-timeout]]
- [[systems/hermes-agent]]
- [[systems/mnemosyne]]
- [[systems/scheduled-tasks]]
- [[current-state]]