wiki(audit): 2026-10-29 window — Conduit voice fault documented and repaired; radar two-stage split resolved both incidents (20 pages, 2 letters, Notion 455->475); cron 176/0
This commit is contained in:
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: Conduit Voice Calls Silently Died — Truncated Speech-Server URL, 12-Hour Session TTL, Dead STT/TTS Providers
|
||||
type: incident
|
||||
status: resolved
|
||||
created: 2026-10-30
|
||||
updated: 2026-10-30
|
||||
verified_on: 2026-10-30
|
||||
confidence: high
|
||||
tags: [conduit, voice, tts, stt, config, session-ttl, incident]
|
||||
sources:
|
||||
- live-probe-2026-10-30
|
||||
- live-config-2026-10-30
|
||||
- live-log-2026-10-30
|
||||
- session-20260908_031541_49c051
|
||||
- plugin-tree-conduit_push
|
||||
---
|
||||
|
||||
# Conduit Voice Calls Silently Died — Truncated Speech-Server URL, 12-Hour Session TTL, Dead STT/TTS Providers
|
||||
|
||||
**Window:** 2026-10-29 07:54 – 09:0x AWST · **Fixed:** 2026-10-29 08:02–08:4x AWST (dashboard restarted 08:03:26)
|
||||
**Symptom:** Anthony asked *"why do you never answer when I call you on Conduit"* — every Conduit voice call connected and then died silently. No Mac connection, no Hermes turn, no error surfaced on the phone.
|
||||
|
||||
Three independent faults sat in series. Fixing any one alone would not have made a call work; each one masked the next.
|
||||
|
||||
## Evidence trail (why it was invisible)
|
||||
|
||||
- The Mac Mini speech server recorded **zero connections all day.** `last_request_at` on the plugin was `null` — meaning no call had *ever* reached the Hermes turn logic.
|
||||
- `conduit-calls.json → history` was empty: no successful call had ever been recorded.
|
||||
- The only traces were in the plugin log: a wrong-looking reason string per attempt, which is what made each fault look like the *next* fault.
|
||||
|
||||
## Fault A — Truncated speech-server URL (403)
|
||||
|
||||
`voice.speech_server.url` was set to the bare host path:
|
||||
|
||||
```
|
||||
wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/ ⛔ (truncated)
|
||||
wss://anthonys-mac-mini.kangaroo-eel.ts.net:8443/v1/realtime ✅
|
||||
```
|
||||
|
||||
`plugin_api.py` (~line 2646) dials the configured value **verbatim** — it does not append `/realtime`. The bare `/v1/` returns HTTP **403**, the socket closes, and the plugin reports it as *"the speech server refused the key."* The key was fine; the path was wrong.
|
||||
|
||||
**Fix:** set `voice.speech_server.url` to `…:8443/v1/realtime`. Backup: `~/.hermes/config.yaml.bak.url-081525` (08:15:25).
|
||||
|
||||
## Fault B — Dashboard session TTL of 12 hours (WS 4401)
|
||||
|
||||
`dashboard.basic_auth.session_ttl_seconds` held the **12-hour default (43 200 s)**. The dashboard issues the phone its session, and when that session lapses the Conduit call socket answers **WS 4401 "Dashboard sign-in required"** — with no Hermes turn and no user-visible error.
|
||||
|
||||
Audit-log corroboration: the iPhone **re-authenticated at 08:00:59**, *minutes after* the silent call — i.e. it was walking around with an expired session and only logging back in when the dashboard happened to prompt.
|
||||
|
||||
**Fix:** raised to **30 days (2 592 000 s)** — matched to the refresh-token life so the phone's session no longer expires under it. Dashboard restarted **2026-10-29 08:03:26** (pid 4042003). Backup: `~/.hermes/config.yaml.bak.ttl-20261029080258`.
|
||||
|
||||
> ⚠ Two keys exist and only one was changed: `dashboard.basic_auth.session_ttl_seconds` = **2592000** (raised) and `dashboard.session_ttl_seconds` = **43200** (still 12 h, different key, different consumer). Check *both* when diagnosing a 4401 again.
|
||||
|
||||
## Fault C — Host STT and TTS providers both dead
|
||||
|
||||
Both halves of the voice loop were broken on the host, so even a correctly-signed call would have had no ears and no mouth.
|
||||
|
||||
| Setting | Was | Failed with | Now | Verified |
|
||||
|---|---|---|---|---|
|
||||
| `tts.provider` | `openai` | Mac `/v1/speech` → **502** | `ollama_local` | 6.94 s of real 64 kbps speech produced |
|
||||
| `stt.provider` | `groq` | **403 / Cloudflare error 1010** | `local` | Hermes' own transcription path returned `success: True, provider: local` |
|
||||
|
||||
- Groq was **blocked, not mis-keyed** — it returns 403/1010 identically with a bogus key, so the failure was Cloudflare bot-blocking, not credentials.
|
||||
- `tts.provider` had been flipped to `openai` by the **Conduit settings tool** earlier (that tool writes Hermes config). The app holds no `tts.provider` setting, so this fix will not be silently overwritten.
|
||||
- `ollama_local` runs `~/.hermes/scripts/ollama-tts.sh` (model `phonon`); `local` STT runs `faster_whisper` (`base`).
|
||||
- Backup: `~/.hermes/config.yaml.bak.stt-084536` (08:45:36); the TTS flip landed between the 08:15 and 08:45 backups.
|
||||
|
||||
## Verification (end-to-end, not just config)
|
||||
|
||||
Rather than trusting three config edits, the whole call path was driven for real: minted a valid HMAC dashboard session → got a WS ticket → let the plugin dial the Mac. Result:
|
||||
|
||||
- auth gate passed → `session.created` / `session.updated`
|
||||
- speech in transcribed correctly ("Hello Conduit, can you hear me right now?")
|
||||
- turn committed → audio streamed back → `response.done`
|
||||
- **Mac counters 0 → 10 connections, 1 completed response, 2.98 s audio, 0 errors**
|
||||
- plugin `connected: true`, handshake **41 ms**
|
||||
- the iPhone minted its own WS tickets at **08:07** and **08:19** — i.e. against the new 30-day session
|
||||
|
||||
## App update to 2.2.0 changed the settings schema
|
||||
|
||||
The iPhone app self-updated to **2.2.0** (from 1.10.1) mid-window, and its settings schema changed with it: the phone now exposes **19 keys, none of them `voice.local.*`**. There is **no phone-side Local Live server address**, so the Local Live path cannot be configured from the host side at all — a prior assumption that it could is now obsolete.
|
||||
|
||||
Consequence: `voice.mode` was moved **`local` → `classic`** (request queued 08:47, approved in the app; `conduit-settings.json` now reads `classic`). Classic needs no phone-side server config because the plugin relays the whole call — which is precisely why Fault C (both host providers dead) had to be fixed for it to work at all.
|
||||
|
||||
## Call record
|
||||
|
||||
`conduit-calls.json → history` now holds **2 entries — the first real calls ever recorded**:
|
||||
|
||||
| Time (AWST) | Outcome |
|
||||
|---|---|
|
||||
| 2026-10-29 08:38:04 | No Mac connection, no Hermes turn. iPhone had to re-login at 08:39:05, a minute later. |
|
||||
| 2026-10-29 08:49:36 | Placed *after* the Classic switch and the STT/TTS fixes. Session ended 08:49:35 — Anthony's verdict is unrecorded. |
|
||||
|
||||
## Open items
|
||||
|
||||
- The 08:49:36 Classic call has **no recorded outcome**. Nothing in the window proves a phone-originated call ran a real Hermes turn.
|
||||
- `last_request_at` was still `null` at the end of the window: the call→Hermes-turn bridge remains **unproven** even though the transport is now healthy. Classic mode is the recommended path.
|
||||
- If an app update or the settings sync ever writes the bare `/v1/` back into `voice.speech_server.url`, every call silently 403s again. Re-check it after any Conduit update.
|
||||
- Watch for the Conduit settings tool re-flipping `tts.provider` should a `tts.provider` key ever appear in the app's schema.
|
||||
|
||||
## Related
|
||||
|
||||
- [Conduit](../systems/conduit.md) — service page (voice pipeline, call path, pairing)
|
||||
- [Hermes Agent](../systems/hermes-agent.md) — TTS/STT provider block
|
||||
- [Conduit call request refused (2026-10-24)](2026-10-24-conduit-call-request-refused.md) — the earlier, behavioural half of the "calls don't arrive" problem
|
||||
- [Conduit memory-start timeout (2026-10-22)](2026-10-22-conduit-memory-start-timeout.md)
|
||||
Reference in New Issue
Block a user