AI Newsletter Digest improvements: fixed QP soft line break decoding, URL extraction, and content cleaning

memory/2026-03-03-gateway-instability.md

@@ -0,0 +1,79 @@
+# Session: 2026-03-03 08:37:20 UTC
+
+- **Session Key**: agent:main:main
+- **Session ID**: 92fb6be0-9712-4fe4-91cd-4bbe97fc0916
+- **Source**: telegram
+
+## Conversation Summary
+
+assistant: Let me check if the morning briefing ran today and if there are any current issues:
+assistant: Everything looks good from my checks:
+
+- ✅ Birthday tracker ran successfully this morning (sent reminder for Pineapples)
+- ✅ No stuck processes or failing cron jobs visible
+- ✅ No urgent calendar events flagged
+- ✅ Recent sessions show normal operation
+
+HEARTBEAT_OK
+user: System: [2026-03-03 05:11:46 UTC] Node: Anthony’s iMac (192.168.178.88) · app 2026.2.21 (13056) · mode remote · reason launch
+
+Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 6:00 AM (UTC)
+user: System: [2026-03-03 06:35:50 UTC] Node: Anthony’s iMac (192.168.178.88) · app 2026.2.21 (13056) · mode remote
+
+Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 6:42 AM (UTC)
+assistant:  
+assistant:  
+assistant: ⚠️ **Attention Required**
+
+**Gateway Instability Detected:**
+- Multiple health check failures and auto-restarts since 5:04 AM UTC (approx every 10-20 minutes)
+- Latest restart at 6:31 AM UTC - gateway recovered but pattern continues
+- Watchdog log shows repeated failures: 5:04, 5:07, 5:11, 6:00, 6:11, 6:26, 6:30
+
+**Current Status:**
+- Gateway appears operational now (watchdog running)
+- Morning briefing ran successfully at 11:50 AM Perth time (March 3)
+- Memory checkpoint completed (with rate limit fallback to StepFun model)
+- Backup verification passed at 2:02 AM
+
+**Recommendation:** Monitor for continued instability - may need gateway log investigation if restarts continue.
+user: Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 7:12 AM (UTC)
+assistant: ⚠️ **Gateway Instability Continuing**
+
+**Status Update:**
+- **07:11 UTC** - Another health check failure, config reverted to last-known-good snapshot
+- Pattern continues: gateway restarts approximately every 10-20 minutes since 5:04 AM UTC
+- Watchdog is actively recovering the gateway each time
+
+**Current Action:** Watchdog restarting gateway now (07:11:59 UTC)
+user: Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 7:21 AM (UTC)
+user: Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 7:21 AM (UTC)
+assistant: Let me check the latest status since the last heartbeat (9 minutes ago at 7:12 AM UTC).
+
+assistant: The gateway appears to have stabilized after the config revert. 
+
+**Current Status:**
+- **Last recovery:** 07:12:34 UTC (9 minutes ago) - stable since then
+- **Total restarts today:** 8+ health check failures with gateway restarts
+- **Config revert:** Applied at 07:11:59 UTC - appears to have resolved the issue
+
+The gateway is currently operational and stable. The config revert seems to have worked.
+
+HEARTBEAT_OK
+user: System: [2026-03-03 07:52:30 UTC] Node: Anthony’s iMac (192.168.178.88) · app 2026.3.2 (2026030290) · mode remote
+
+Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. Do not infer or repeat old tasks from prior chats. If nothing needs attention, reply HEARTBEAT_OK.
+Current time: Tuesday, March 3rd, 2026 — 7:55 AM (UTC)
+assistant: ⚠️ **Critical: Gateway Instability Ongoing**
+
+**Status Update (since last check ~34 min ago):**
+- **07:33-07:38 UTC** - 15 consecutive health check failures, gateway finally recovered at 07:38:37
+- **07:49:51 UTC** - Another health check failure, recovered at 07:50:26
+- **Current status:** Gateway recovered (as of ~5 min ago)
+
+**Pattern:** Continuous instability since 5:04 AM UTC - 20+ restarts today, with periods of multiple consecutive failures. This is an ongoing critical issue.