wiki(audit): 2026-10-29 window — Conduit voice fault documented and repaired; radar two-stage split resolved both incidents (20 pages, 2 letters, Notion 455->475); cron 176/0

This commit is contained in:
Tony0410
2026-10-30 00:21:21 +08:00
parent cc95dcc54d
commit 1d335ca8e6
10 changed files with 404 additions and 82 deletions
@@ -0,0 +1,106 @@
---
title: Conduit Voice Calls Silently Died — Truncated Speech-Server URL, 12-Hour Session TTL, Dead STT/TTS Providers
type: incident
status: resolved
created: 2026-10-30
updated: 2026-10-30
verified_on: 2026-10-30
confidence: high
tags: [conduit, voice, tts, stt, config, session-ttl, incident]
sources:
- live-probe-2026-10-30
- live-config-2026-10-30
- live-log-2026-10-30
- session-20260908_031541_49c051
- plugin-tree-conduit_push
---
# Conduit Voice Calls Silently Died — Truncated Speech-Server URL, 12-Hour Session TTL, Dead STT/TTS Providers
**Window:** 2026-10-29 07:54 – 09:0x AWST · **Fixed:** 2026-10-29 08:02–08:4x AWST (dashboard restarted 08:03:26)
**Symptom:** Anthony asked *"why do you never answer when I call you on Conduit"* — every Conduit voice call connected and then died silently. No Mac connection, no Hermes turn, no error surfaced on the phone.
Three independent faults sat in series. Fixing any one alone would not have made a call work; each one masked the next.
## Evidence trail (why it was invisible)
- The Mac Mini speech server recorded **zero connections all day.** `last_request_at` on the plugin was `null` — meaning no call had *ever* reached the Hermes turn logic.
- `conduit-calls.json → history` was empty: no successful call had ever been recorded.
- The only traces were in the plugin log: a wrong-looking reason string per attempt, which is what made each fault look like the *next* fault.
## Fault A — Truncated speech-server URL (403)
`voice.speech_server.url` was set to the bare host path:
```
wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/ ⛔ (truncated)
wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/realtime ✅
```
`plugin_api.py` (~line 2646) dials the configured value **verbatim** — it does not append `/realtime`. The bare `/v1/` returns HTTP **403**, the socket closes, and the plugin reports it as *"the speech server refused the key."* The key was fine; the path was wrong.
**Fix:** set `voice.speech_server.url` to `…:8443/v1/realtime`. Backup: `~/.hermes/config.yaml.bak.url-081525` (08:15:25).
## Fault B — Dashboard session TTL of 12 hours (WS 4401)
`dashboard.basic_auth.session_ttl_seconds` held the **12-hour default (43 200 s)**. The dashboard issues the phone its session, and when that session lapses the Conduit call socket answers **WS 4401 "Dashboard sign-in required"** — with no Hermes turn and no user-visible error.
Audit-log corroboration: the iPhone **re-authenticated at 08:00:59**, *minutes after* the silent call — i.e. it was walking around with an expired session and only logging back in when the dashboard happened to prompt.
**Fix:** raised to **30 days (2 592 000 s)** — matched to the refresh-token life so the phone's session no longer expires under it. Dashboard restarted **2026-10-29 08:03:26** (pid 4042003). Backup: `~/.hermes/config.yaml.bak.ttl-20261029080258`.
> ⚠ Two keys exist and only one was changed: `dashboard.basic_auth.session_ttl_seconds` = **2592000** (raised) and `dashboard.session_ttl_seconds` = **43200** (still 12 h, different key, different consumer). Check *both* when diagnosing a 4401 again.
## Fault C — Host STT and TTS providers both dead
Both halves of the voice loop were broken on the host, so even a correctly-signed call would have had no ears and no mouth.
| Setting | Was | Failed with | Now | Verified |
|---|---|---|---|---|
| `tts.provider` | `openai` | Mac `/v1/speech` → **502** | `ollama_local` | 6.94 s of real 64 kbps speech produced |
| `stt.provider` | `groq` | **403 / Cloudflare error 1010** | `local` | Hermes' own transcription path returned `success: True, provider: local` |
- Groq was **blocked, not mis-keyed** — it returns 403/1010 identically with a bogus key, so the failure was Cloudflare bot-blocking, not credentials.
- `tts.provider` had been flipped to `openai` by the **Conduit settings tool** earlier (that tool writes Hermes config). The app holds no `tts.provider` setting, so this fix will not be silently overwritten.
- `ollama_local` runs `~/.hermes/scripts/ollama-tts.sh` (model `phonon`); `local` STT runs `faster_whisper` (`base`).
- Backup: `~/.hermes/config.yaml.bak.stt-084536` (08:45:36); the TTS flip landed between the 08:15 and 08:45 backups.
## Verification (end-to-end, not just config)
Rather than trusting three config edits, the whole call path was driven for real: minted a valid HMAC dashboard session → got a WS ticket → let the plugin dial the Mac. Result:
- auth gate passed → `session.created` / `session.updated`
- speech in transcribed correctly ("Hello Conduit, can you hear me right now?")
- turn committed → audio streamed back → `response.done`
- **Mac counters 0 → 10 connections, 1 completed response, 2.98 s audio, 0 errors**
- plugin `connected: true`, handshake **41 ms**
- the iPhone minted its own WS tickets at **08:07** and **08:19** — i.e. against the new 30-day session
## App update to 2.2.0 changed the settings schema
The iPhone app self-updated to **2.2.0** (from 1.10.1) mid-window, and its settings schema changed with it: the phone now exposes **19 keys, none of them `voice.local.*`**. There is **no phone-side Local Live server address**, so the Local Live path cannot be configured from the host side at all — a prior assumption that it could is now obsolete.
Consequence: `voice.mode` was moved **`local` → `classic`** (request queued 08:47, approved in the app; `conduit-settings.json` now reads `classic`). Classic needs no phone-side server config because the plugin relays the whole call — which is precisely why Fault C (both host providers dead) had to be fixed for it to work at all.
## Call record
`conduit-calls.json → history` now holds **2 entries — the first real calls ever recorded**:
| Time (AWST) | Outcome |
|---|---|
| 2026-10-29 08:38:04 | No Mac connection, no Hermes turn. iPhone had to re-login at 08:39:05, a minute later. |
| 2026-10-29 08:49:36 | Placed *after* the Classic switch and the STT/TTS fixes. Session ended 08:49:35 — Anthony's verdict is unrecorded. |
## Open items
- The 08:49:36 Classic call has **no recorded outcome**. Nothing in the window proves a phone-originated call ran a real Hermes turn.
- `last_request_at` was still `null` at the end of the window: the call→Hermes-turn bridge remains **unproven** even though the transport is now healthy. Classic mode is the recommended path.
- If an app update or the settings sync ever writes the bare `/v1/` back into `voice.speech_server.url`, every call silently 403s again. Re-check it after any Conduit update.
- Watch for the Conduit settings tool re-flipping `tts.provider` should a `tts.provider` key ever appear in the app's schema.
## Related
- [Conduit](../systems/conduit.md) — service page (voice pipeline, call path, pairing)
- [Hermes Agent](../systems/hermes-agent.md) — TTS/STT provider block
- [Conduit call request refused (2026-10-24)](2026-10-24-conduit-call-request-refused.md) — the earlier, behavioural half of the "calls don't arrive" problem
- [Conduit memory-start timeout (2026-10-22)](2026-10-22-conduit-memory-start-timeout.md)