- freshness: 36 stale / 93 fresh / 3 no-date -> 0 stale / 129 fresh / 3 no-date of 132 (same-day report regenerated after the corrections, per pattern 13) - skills: 299 skills / 67 categories (count unchanged; generator arithmetic re-verified by hand against the live tree: 300 SKILL.md files - 1 nested duplicate = 299) - runbooks/test-openrouter: live fallback chain rewritten (primary back to smart-route/custom:omni-101; mimo-v2.5, stepfun, qwen3.5/ollama-cloud out; guacano + kilo-reflection-beam-free in); ollama.com dropped from the URL test list; guacano/litellm noted as internal + key-gated - comparisons/hermes-vs-nanobot: the 2026-10-20 'CT333 is zeroclaw in Proxmox naming' claim was itself wrong - pct list and hostname both return clawtest - incidents/2026-10-22-nous: refresh-token failure gone from logs; new signature 404 insufficient_credits_for_paid_model from auxiliary.pulse_memory (ninth slot, back on nous, paid model) - open item revised - incidents/2026-08-29-gmail: email crons now ok but root cause unfixed 74 days (himalaya message read still in fetch_newsletters.py; goimapnotify back up PID 441; monitor still paused) - 30 further pages re-verified live and de-flagged with dated probe notes (ct207 ingress, CT205 guanaco v0.8.15, CT270 nextcloud 33.0.2.2, headless stack, hermes-ui both hosts, systemd units + cgroup limits, terminal/tool_search/mnemosyne/custom_providers config blocks, ops profile still deleted, hosts/pve guest lists) - log.md addendum + index.md date bump; no page deleted, no secrets written
123 lines
6.4 KiB
Markdown
123 lines
6.4 KiB
Markdown
---
|
|
title: "Gmail Read-State Incident (2026-08-29)"
|
|
type: incident
|
|
status: active
|
|
created: 2026-08-29
|
|
updated: 2026-11-11
|
|
verified_on: 2026-11-11
|
|
confidence: high
|
|
tags: [email, gmail, cron, himalaya, incident]
|
|
sources: [session:default/20260829_204103_9e1087]
|
|
---
|
|
|
|
# Gmail Read-State Incident
|
|
|
|
**Date discovered:** 2026-08-29 20:41 AWST
|
|
**Status:** Active — mitigation in place, root cause identified but not fully fixed
|
|
|
|
## Summary
|
|
|
|
Anthony's Gmail inbox messages were being automatically marked as read without his consent. The issue was traced to Hermes cron jobs using Himalaya's `message read` command, which applies the "seen" flag even with `--preview`.
|
|
|
|
## Timeline
|
|
|
|
- **2026-08-29 20:41** — Anthony reports: "What is marking my Gmail email as read it is not meant to be"
|
|
- **Investigation** — Loaded skills: `gmail-imap-email-monitor`, `hermes-cron-operations`, `email-inbox-triage`
|
|
- **Root cause identified** — `fetch_newsletters.py` (line 49) runs:
|
|
```bash
|
|
himalaya message read --preview --folder INBOX <message-id>
|
|
```
|
|
Himalaya's `message read` command **automatically applies the "seen" flag** unless `--preview` is used for preview-only access. However, even `--preview` may still mark messages read in some configurations.
|
|
- **Other scripts checked** — `gmail_readonly_imap.py` uses `SELECT readonly` and `BODY.PEEK[]` (safe, does not mark read)
|
|
- **Mitigation applied:**
|
|
- Stopped `goimapnotify.service` (IMAP watcher)
|
|
- Paused "Important-mail monitor" cron job (if present)
|
|
- Newsletter digest already patched to use `--preview`
|
|
|
|
## Root Cause
|
|
|
|
The `fetch_newsletters.py` script at `/home/hermes/.hermes/scripts/fetch_newsletters.py` line 49:
|
|
|
|
```python
|
|
r2 = subprocess.run(
|
|
["himalaya", "message", "read", "--preview", "--folder", "INBOX", m["id"]],
|
|
capture_output=True, text=True, timeout=60
|
|
)
|
|
```
|
|
|
|
Despite using `--preview`, Himalaya may still mark messages as read depending on configuration. The proper fix is to use readonly IMAP with `BODY.PEEK[]` instead of the `himalaya message read` command.
|
|
|
|
## Impact
|
|
|
|
- Newsletter emails processed by the Tech & AI Newsletter Digest cron job were marked as read
|
|
- Potential for other Gmail automation scripts to have similar behavior
|
|
- User trust issue — unexpected state changes in personal email
|
|
|
|
## Resolution Status
|
|
|
|
**Mitigation:** Partially complete
|
|
- `goimapnotify` stopped
|
|
- Important-mail monitor paused (if it existed)
|
|
|
|
> **Re-verified 2026-10-20 — the mitigation has partially REGRESSED, and the root cause is still
|
|
> live.**
|
|
>
|
|
> | Item | State on 2026-08-29 | State on 2026-10-20 |
|
|
> |---|---|---|
|
|
> | `goimapnotify.service` | stopped | **ACTIVE again** — `systemctl --user is-active goimapnotify.service` → `active`, PID 383 `/home/hermes/go/bin/goimapnotify -conf /home/hermes/.config/goimapnotify/goimapnotify.yaml` |
|
|
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False` in `jobs.json`, `last_status: ok` |
|
|
> | `fetch_newsletters.py` line 49 | patched to `--preview` | **still `himalaya message read --preview --folder INBOX`** — the unsafe call is still in the file, unfixed |
|
|
>
|
|
> **So the real fix — rewriting `fetch_newsletters.py` to use readonly IMAP (`SELECT readonly` /
|
|
> `BODY.PEEK[]`) — has still not been done 52 days later.** The two mitigations that were applied
|
|
> are one of them gone (goimapnotify restarted) and one still holding (the important-mail monitor
|
|
> remains disabled).
|
|
>
|
|
> **Other email crons currently in `error` state (2026-10-20 read of `jobs.json`):**
|
|
> `85edf6772bca` Tech & AI Newsletter Digest, `08eb86f02d37` Gmail Action Digest,
|
|
> `06a0a1c8704f` AI Newsletter Curation for Patrick. Their most recent failures were
|
|
> provider-side (OmniRoute 503 resource pressure on 2026-10-19; a connection error on 2026-10-15),
|
|
> **not** read-state errors — but the read-state exposure persists regardless, because the
|
|
> newsletter fetch path still issues `message read`.
|
|
>
|
|
> **Re-verified 2026-11-11 — the cron error state has cleared; the read-state root cause has not.**
|
|
>
|
|
> | Item | State on 2026-10-20 | State on 2026-11-11 |
|
|
> |---|---|---|
|
|
> | `goimapnotify.service` | active (PID 383) | **still active**, now PID **441**, same binary/conf path |
|
|
> | Important-mail monitor (`d40085da631c`) | paused | **still paused** — `enabled: False`, `last_status: ok` |
|
|
> | `fetch_newsletters.py` body-fetch call | `himalaya message read --preview …` | **identical, still unfixed** — the unsafe call is still in the file |
|
|
> | `85edf6772bca` Tech & AI Newsletter Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 07:05 |
|
|
> | `08eb86f02d37` Gmail Action Digest | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-10 09:18 |
|
|
> | `06a0a1c8704f` AI Newsletter Curation for Patrick | `error` | **`ok`**, `failure_streak: 0`, last run 2026-11-05 10:41 |
|
|
>
|
|
> So 74 days after the incident: **both** original mitigations that were applied on 2026-08-29 are
|
|
> now **inert** — the monitor is still paused, but `goimapnotify` is back up, so IMAP IDLE is live
|
|
> again — and the root cause (`himalaya message read` on Anthony's INBOX) is untouched. The three
|
|
> email cron jobs that were in `error` on 2026-10-20 are now green, but that says nothing about
|
|
> the read-state exposure: a job that marks emails read and *succeeds* still pollutes the mailbox.
|
|
> Per the skill's own rule (a delivered/successful run is not evidence of correct behaviour), the
|
|
> exposure is best judged from the code path, not the cron status — and the code path is unchanged.
|
|
>
|
|
> **Recommended action (not taken here — this is a wiki job, and the fix touches Anthony's mail):**
|
|
> rewrite `fetch_newsletters.py` to use readonly IMAP, and decide whether `goimapnotify` should be
|
|
> running. See [[systems/scheduled-tasks]] and the Gmail IMAP lessons in
|
|
> [[runbooks/provider-health]].
|
|
|
|
**Fix required:**
|
|
- Rewrite `fetch_newsletters.py` to use readonly IMAP (`BODY.PEEK[]`) instead of `himalaya message read`
|
|
- Audit all Gmail-related scripts for similar patterns
|
|
- Verify `gmail_action_digest_v3.py` and other email scripts use readonly access
|
|
|
|
## Related Pages
|
|
|
|
- [[systems/scheduled-tasks]] — Tech & AI Newsletter Digest cron job
|
|
- [[systems/messaging-integrations]] — Email integration patterns
|
|
- [[runbooks/job-radar]] — Email monitoring patterns (for reference)
|
|
|
|
## Prevention
|
|
|
|
- All Gmail IMAP scripts should use `SELECT readonly` and `BODY.PEEK[]` patterns
|
|
- Add a pre-commit check or linting rule for `himalaya message read` usage
|
|
- Document readonly email access pattern in a runbook
|