Files
agent-estate-wiki/incidents/2026-08-29-gmail-read-state.md
T
Hermes 341dda95dd stocktake 2026-11-11: 36 stale pages re-verified (0 stale); fallback chain rewritten 4/6 slots; CT333 'clawtest' naming claim corrected; nous 404 credits signature replaces refresh-token 401; estate shrank (VM 310 + CT601 gone)
- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13)
- skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299)
- runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated
- comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest
- incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised
- incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused)
- 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists)
- log.md addendum + index.md date bump; no page deleted, no secrets written
2026-11-11 03:52:48 +08:00

123 lines
6.4 KiB
Markdown

---
title: "Gmail Read-State Incident (2026-08-29)"
type: incident
status: active
created: 2026-08-29
updated: 2026-11-11
verified_on: 2026-11-11
confidence: high
tags: [email, gmail, cron, himalaya, incident]
sources: [session:default/20260829_204103_9e1087]
---
# Gmail Read-State Incident
**Date discovered:** 2026-08-29 20:41 AWST
**Status:** Active — mitigation in place, root cause identified but not fully fixed
## Summary
Anthony's Gmail inbox messages were being automatically marked as read without his consent. The issue was traced to Hermes cron jobs using Himalaya's `message read` command, which applies the "seen" flag even with `--preview`.
## Timeline
- **2026-08-29 20:41** — Anthony reports: "What is marking my Gmail email as read it is not meant to be"
- **Investigation** — Loaded skills: `gmail-imap-email-monitor`, `hermes-cron-operations`, `email-inbox-triage`
- **Root cause identified** — `fetch_newsletters.py` (line 49) runs:
```bash
himalaya message read --preview --folder INBOX <message-id>
```
Himalaya's `message read` command **automatically applies the "seen" flag** unless `--preview` is used for preview-only access. However, even `--preview` may still mark messages read in some configurations.
- **Other scripts checked** — `gmail_readonly_imap.py` uses `SELECT readonly` and `BODY.PEEK[]` (safe, does not mark read)
- **Mitigation applied:**
- Stopped `goimapnotify.service` (IMAP watcher)
- Paused "Important-mail monitor" cron job (if present)
- Newsletter digest already patched to use `--preview`
## Root Cause
The `fetch_newsletters.py` script at `/home/hermes/.hermes/scripts/fetch_newsletters.py` line 49:
```python
r2 = subprocess.run(
["himalaya", "message", "read", "--preview", "--folder", "INBOX", m["id"]],
capture_output=True, text=True, timeout=60
)
```
Despite using `--preview`, Himalaya may still mark messages as read depending on configuration. The proper fix is to use readonly IMAP with `BODY.PEEK[]` instead of the `himalaya message read` command.
## Impact
- Newsletter emails processed by the Tech & AI Newsletter Digest cron job were marked as read
- Potential for other Gmail automation scripts to have similar behavior
- User trust issue — unexpected state changes in personal email
## Resolution Status
**Mitigation:** Partially complete
- `goimapnotify` stopped
- Important-mail monitor paused (if it existed)
> **Re-verified 2026-10-20 — the mitigation has partially REGRESSED, and the root cause is still
> live.**
>
> | Item | State on 2026-08-29 | State on 2026-10-20 |
> |---|---|---|
> | `goimapnotify.service` | stopped | **ACTIVE again** — `systemctl --user is-active goimapnotify.service` → `active`, PID 383 `/home/hermes/go/bin/goimapnotify -conf /home/hermes/.config/goimapnotify/goimapnotify.yaml` |
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False` in `jobs.json`, `last_status: ok` |
> | `fetch_newsletters.py` line 49 | patched to `--preview` | **still `himalaya message read --preview --folder INBOX`** — the unsafe call is still in the file, unfixed |
>
> **So the real fix — rewriting `fetch_newsletters.py` to use readonly IMAP (`SELECT readonly` /
> `BODY.PEEK[]`) — has still not been done 52 days later.** The two mitigations that were applied
> are one of them gone (goimapnotify restarted) and one still holding (the important-mail monitor
> remains disabled).
>
> **Other email crons currently in `error` state (2026-10-20 read of `jobs.json`):**
> `85edf6772bca` Tech & AI Newsletter Digest, `08eb86f02d37` Gmail Action Digest,
> `06a0a1c8704f` AI Newsletter Curation for Patrick. Their most recent failures were
> provider-side (OmniRoute 503 resource pressure on 2026-10-19; a connection error on 2026-10-15),
> **not** read-state errors — but the read-state exposure persists regardless, because the
> newsletter fetch path still issues `message read`.
>
> **Re-verified 2026-11-11 — the cron error state has cleared; the read-state root cause has not.**
>
> | Item | State on 2026-10-20 | State on 2026-11-11 |
> |---|---|---|
> | `goimapnotify.service` | active (PID 383) | **still active**, now PID **441**, same binary/conf path |
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False`, `last_status: ok` |
> | `fetch_newsletters.py` body-fetch call | `himalaya message read --preview …` | **identical, still unfixed** — the unsafe call is still in the file |
> | `85edf6772bca` Tech & AI Newsletter Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 07:05 |
> | `08eb86f02d37` Gmail Action Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 09:18 |
> | `06a0a1c8704f` AI Newsletter Curation for Patrick | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-05 10:41 |
>
> So 74 days after the incident: **both** original mitigations that were applied on 2026-08-29 are
> now **inert** — the monitor is still paused, but `goimapnotify` is back up, so IMAP IDLE is live
> again — and the root cause (`himalaya message read` on Anthony's INBOX) is untouched. The three
> email cron jobs that were in `error` on 2026-10-20 are now green, but that says nothing about
> the read-state exposure: a job that marks emails read and *succeeds* still pollutes the mailbox.
> Per the skill's own rule (a delivered/successful run is not evidence of correct behaviour), the
> exposure is best judged from the code path, not the cron status — and the code path is unchanged.
>
> **Recommended action (not taken here — this is a wiki job, and the fix touches Anthony's mail):**
> rewrite `fetch_newsletters.py` to use readonly IMAP, and decide whether `goimapnotify` should be
> running. See [[systems/scheduled-tasks]] and the Gmail IMAP lessons in
> [[runbooks/provider-health]].
**Fix required:**
- Rewrite `fetch_newsletters.py` to use readonly IMAP (`BODY.PEEK[]`) instead of `himalaya message read`
- Audit all Gmail-related scripts for similar patterns
- Verify `gmail_action_digest_v3.py` and other email scripts use readonly access
## Related Pages
- [[systems/scheduled-tasks]] — Tech & AI Newsletter Digest cron job
- [[systems/messaging-integrations]] — Email integration patterns
- [[runbooks/job-radar]] — Email monitoring patterns (for reference)
## Prevention
- All Gmail IMAP scripts should use `SELECT readonly` and `BODY.PEEK[]` patterns
- Add a pre-commit check or linting rule for `himalaya message read` usage
- Document readonly email access pattern in a runbook