249 lines
12 KiB
Markdown
249 lines
12 KiB
Markdown
---
|
|
title: Daily Marketing Job Radar Operations and Recovery
|
|
type: runbook
|
|
status: active
|
|
created: 2026-07-27
|
|
updated: 2026-07-28
|
|
review_after: 2026-10-25
|
|
verified_on: 2026-07-28
|
|
version_applies_to: Hermes Agent cron dca8482e4f76
|
|
confidence: high
|
|
tags: [automation, cron, jobs, notion, dashboard, runbook]
|
|
sources: [live-cron-output, live-notion-api, live-dashboard-api, hermes-docs]
|
|
---
|
|
|
|
# Daily Marketing Job Radar Operations and Recovery
|
|
|
|
## Purpose
|
|
|
|
Run a verified Western Australian marketing-job search, deduplicate against the complete Notion board, create only fresh direct adverts, score new roles against Anthony's resume, automatically produce cover letters for strong matches, and expose the results through the Jobs Radar dashboard.
|
|
|
|
## Production configuration
|
|
|
|
| Setting | Value |
|
|
|---|---|
|
|
| Cron job | `dca8482e4f76` — Daily Marketing Job Radar |
|
|
| Schedule | `0 7 * * 1-5` (07:00 AWST weekdays) |
|
|
| Delivery | Telegram group `-1004321904721`, topic `1030` |
|
|
| Skill | `job-search-automation` |
|
|
| Primary model | `openai-codex / gpt-5.6-terra` |
|
|
| First fallback | `opencode-go / mimo-v2.5` |
|
|
| Dashboard | `https://hermes.kangaroo-eel.ts.net/jobs/` |
|
|
| Dashboard backend | `127.0.0.1:9099` |
|
|
| Notion database | `0f90ba2b-8b10-4d02-a62c-6f692d5b1168` |
|
|
|
|
Cron jobs inherit the profile-level `fallback_providers` chain; Hermes does not currently store a separate fallback chain in each cron record. MiMo v2.5 is first in the global chain, so Terra fails over to it before the older fallbacks.
|
|
|
|
## Cover letters are non-negotiable
|
|
|
|
A successful radar run includes the cover-letter stage.
|
|
|
|
- Audit **every Strong Match added today**, including pages created by an interrupted earlier run.
|
|
- A valid result requires a canonical rendered `.html` URL in Notion, not plain text and not the dashboard root.
|
|
- Production template files:
|
|
- `/home/hermes/.hermes/dashboard/templates/cover-letter.html`
|
|
- `/home/hermes/.hermes/dashboard/templates/Anthony_Martin_Cover_Letter_Template.docx`
|
|
- The old upload-cache location is not authoritative and may be evicted.
|
|
- Required nonempty artifacts: `.txt`, `.json`, `.html`, `.docx`, `.pdf`.
|
|
- The HTML must contain zero unresolved `{{...}}` tokens and return HTTP 200.
|
|
- Retry generation once when an artifact, Notion link or HTTP check fails.
|
|
- A clean run requires `MISSING_COVER_LETTER_COUNT=0`. Any remaining missing letter must be reported prominently with its Notion page ID; it must never be silently omitted.
|
|
|
|
### Live audit — 27 July 2026
|
|
|
|
- Same-day strong matches: **1**
|
|
- Verified canonical HTML cover letters: **1**
|
|
- Missing cover letters: **0**
|
|
- The Market Creations Agency strong match has all five artifact formats, zero unresolved template tokens and a live HTTP 200 HTML link.
|
|
|
|
## Required production pipeline
|
|
|
|
1. Obtain the live AWST date and calculate a seven-day freshness cutoff.
|
|
2. Search SEEK, LinkedIn, Indeed, WA Government Jobs and direct employer sites.
|
|
3. Open and verify every individual advert. Search snippets are discovery evidence only.
|
|
4. Reject stale, expired, cached, interstate, non-marketing, salary-guide, search/category and aggregator pages.
|
|
5. Reject **all Pacific Energy roles**. Anthony was made redundant there in July 2026; old Pacific Energy adverts must never be presented as opportunities.
|
|
6. Accept only direct individual advert URLs:
|
|
- SEEK `/job/<numeric-id>`
|
|
- LinkedIn `/jobs/view/...<numeric-id>`
|
|
- Indeed `/viewjob?jk=<real-id>`
|
|
- direct employer career pages
|
|
- individual WA Government role pages
|
|
7. A WA Government slug page is valid without `AdvertID` in its URL when the page proves the exact role and agency, WA location, unique job/pool reference, future closing date and active **Apply Now** function.
|
|
8. Query the complete Notion database with live pagination. Deduplicate by exact URL, normalized company plus role, and materially similar company-role combinations.
|
|
9. Create only verified nonduplicates and require returned Notion page IDs.
|
|
10. Refresh `POST http://127.0.0.1:9099/api/refresh`, then confirm every page and direct URL through `/api/jobs`.
|
|
11. Score newly created roles against `/home/hermes/.hermes/radar-ref/resumes/current_resume_from_website.txt` using the documented 100-point rubric. Strong Match starts at 70.
|
|
12. For each new strong match, run:
|
|
|
|
```bash
|
|
/home/hermes/.hermes/hermes-agent/venv/bin/python \
|
|
/home/hermes/.hermes/scripts/regenerate_cover_letter.py \
|
|
<NOTION_PAGE_ID> \
|
|
"Generate the initial tailored cover letter using the verified job description. Preserve Anthony's factual employment history and use the canonical teal Montserrat template."
|
|
```
|
|
|
|
13. Verify nonempty `.txt`, `.json`, `.html`, `.docx` and `.pdf` artifacts; no unresolved `{{...}}` tokens; and a live `.html` Notion Cover Letter URL.
|
|
14. Report only facts produced by tools in that run. Never estimate counts or fabricate Notion writes.
|
|
|
|
## Dashboard behaviour
|
|
|
|
Source: `/home/hermes/.hermes/dashboard/index.html`
|
|
Backend: `/home/hermes/.hermes/scripts/job_dashboard_server.py`
|
|
|
|
### Layout (top to bottom)
|
|
1. **Hero header** — RADAR logo, name, dateline
|
|
2. **Market Intel panel** — expandable salary/location/source stats from `/api/market-intel`
|
|
3. **KPI strip** — clickable filter cards: Total Jobs, Backlog, Applied, Phone Screen, Interviews, Offers, Closing Soon
|
|
4. **Filter bar** — text search, view toggles (table/cards), Refresh button
|
|
5. **Stats strip** — This Week, Stale 14d+, Strong Match, Past Close, Applied Rate
|
|
6. **Pipeline Funnel** — clickable per-status breakdown with bar chart
|
|
7. **Job table/cards** — sortable columns, inline status pills, cover letter links, edit actions
|
|
|
|
### Column sorting
|
|
Click any column header (Status, Company, Role, Match, Salary, Age, Closes, Updated) to sort ascending; click again for descending. Arrow indicator shows current direction.
|
|
|
|
### Status management
|
|
- **Bulk update**: select jobs via checkboxes → pick status from dropdown → Apply. Sends `{ids, updates: {status}}` to `POST /api/jobs/bulk-update`.
|
|
- **Single edit**: click ✎ → modal with Status, Priority, Role, Salary, Closing Date, URL, Notes, Cover Letter, Attachments. Saves to Notion via `POST /api/jobs/<id>/update`.
|
|
- **Funnel/status filters**: clicking a status in the funnel or KPI strip filters the table to that status.
|
|
|
|
### Rich text notes
|
|
- Write/Preview editor in the edit modal (monospace textarea + markdown preview)
|
|
- Supports: bold `**`, italic `*`, links `[text](url)`, bullet lists `- item`
|
|
- Syncs to Notion `Notes / JD Summary` rich_text property on every save
|
|
|
|
### Document attachments
|
|
- Attach any URL (documents, links, files) to a job entry
|
|
- Stored as **bookmark blocks** on the Notion page body (not a property)
|
|
- Lazy-loaded when edit modal opens: `GET /api/jobs/<id>/attachments`
|
|
- Add: `POST /api/jobs/<id>/attachments` with `{name, url}`
|
|
- Delete: `POST /api/jobs/<id>/attachments/delete` with `{block_id}`
|
|
- Visible in Notion when you open the page
|
|
|
|
### Cover letter integration
|
|
- Regenerate button (🔄) on each row calls `POST /api/jobs/<id>/cover-letter/regenerate`
|
|
- CL URL stored in Notion `Cover Letter` property and rendered as clickable link in table
|
|
- All 5 artifact formats: TXT, JSON, HTML, DOCX, PDF
|
|
|
|
### Market Intel
|
|
- Expandable panel at top of page
|
|
- Data from `GET /api/market-intel`: total tracked, salary stats (avg/median/min/max), counts
|
|
- Lazy-loaded on expand; also refreshes on ↺ Refresh
|
|
|
|
### API endpoints
|
|
| Endpoint | Method | Purpose |
|
|
|---|---|---|
|
|
| `/api/jobs` | GET | All jobs (cached, paginated from Notion) |
|
|
| `/api/stats` | GET | Aggregate statistics |
|
|
| `/api/refresh` | POST | Force Notion cache refresh |
|
|
| `/api/market-intel` | GET | Salary/location/source analytics |
|
|
| `/api/jobs/<id>/update` | POST | Update single job properties |
|
|
| `/api/jobs/<id>/attachments` | GET | Fetch page block attachments |
|
|
| `/api/jobs/<id>/attachments` | POST | Add bookmark attachment to page |
|
|
| `/api/jobs/<id>/attachments/delete` | POST | Remove block from page |
|
|
| `/api/jobs/<id>/cover-letter/regenerate` | POST | Regenerate cover letter |
|
|
| `/api/jobs/<id>/match` | GET | Match analysis for a job |
|
|
| `/api/jobs/bulk-update` | POST | Update multiple jobs at once |
|
|
| `/api/jobs/create` | POST | Create new job in Notion |
|
|
|
|
- Company names and job titles with a valid `URL` render as blue external links and open the direct advert in a new tab.
|
|
- URL values are protocol-validated; only `http:` and `https:` are rendered as links.
|
|
- KPI/stat cards are one-click filters with an active highlight:
|
|
- Total Jobs
|
|
- Backlog
|
|
- Applied
|
|
- Interviews
|
|
- Offers
|
|
- Closing Soon
|
|
- This Week
|
|
- Stale
|
|
- Strong Matches
|
|
- Past Closing
|
|
- Application Rate
|
|
- Funnel status rows are also one-click exact-status filters.
|
|
- Existing status editing remains through the job edit modal and bulk-action controls, which write back to Notion through `/api/update`.
|
|
- The frontend filter/link test harness is `/tmp/radar_dashboard_test.js` (temporary; recreate if absent).
|
|
|
|
## 27 July 2026 incident
|
|
|
|
### Symptoms
|
|
|
|
- The morning cron returned old and irrelevant listings, including the stale Pacific Energy Marketing Specialist advert associated with Anthony's redundancy.
|
|
- Aggregator mirrors, generic SEEK salary pages and category pages were treated as job adverts.
|
|
- Counts and quality signals were presented without a demonstrated live Notion query/write trail.
|
|
- The dashboard had valid URLs for almost every record, but the frontend never rendered `job.url`; users could not open the actual adverts.
|
|
- The earlier prompt explicitly narrowed the task so far that it accidentally prohibited the documented strong-match cover-letter stage.
|
|
|
|
### Recovery
|
|
|
|
- Archived all **16** records created by the contaminated morning run: 16 successful, zero failures.
|
|
- Replaced the cron prompt with deterministic freshness, source, verification, Notion, scoring, exclusion and cover-letter gates.
|
|
- Pinned the cron primary to `gpt-5.6-terra`.
|
|
- Added and directly probed `opencode-go / mimo-v2.5`; it returned `MIMO_FALLBACK_OK`, then was inserted first in the inherited fallback chain.
|
|
- Repaired `regenerate_cover_letter.py`:
|
|
- absolute Hermes paths rather than `~` expansion
|
|
- one correct command dispatcher rather than two conflicting `__main__` blocks
|
|
- Repaired dashboard direct links and one-click quick filters.
|
|
- The corrected Terra run created four verified fresh direct LinkedIn roles. One scored 75 and produced all five cover-letter artifact formats.
|
|
- A follow-up run queried 166 live Notion records across two pages, verified three current direct adverts, and correctly skipped all three as duplicates. No Pacific Energy records were added.
|
|
|
|
### Final verification
|
|
|
|
- Dashboard API: 166 records
|
|
- Records created on 27 July after cleanup: 4
|
|
- Pacific Energy records created on 27 July: 0
|
|
- Records with direct URLs: 164 of 166
|
|
- Strong-match cover letter artifact sizes were nonzero for all five formats
|
|
- Unresolved template tokens: 0
|
|
- Dashboard, API and generated cover-letter URLs returned HTTP 200
|
|
- Synthetic dashboard tests passed all quick-filter categories, unsafe-URL rejection and rendered job-link checks
|
|
|
|
## Tailscale route recovery found during verification
|
|
|
|
The dashboard's local service was healthy while the tailnet URL failed because the CT460 `:443` Serve table had been cleared. Running the original atomic helper restored Serve but also erased the public `:8443` Funnel route.
|
|
|
|
`/home/hermes/.hermes/scripts/tailscale-serve-apply.sh` now atomically restores both:
|
|
|
|
- tailnet-only `:443`: `/`, `/jobs`, `/desktop`, `/vnc-camofox`, `/evening-grid.html`
|
|
- public `:8443`: `/webhook` → `http://127.0.0.1:8085/webhook`
|
|
|
|
Important: `tailscale serve reset` clears Funnel state too. Never restore only the 443 routes and assume 8443 survived.
|
|
|
|
## Verification commands
|
|
|
|
```bash
|
|
# Cron configuration
|
|
hermes cron list
|
|
|
|
# Fallback order
|
|
hermes fallback list
|
|
|
|
# Dashboard/API
|
|
curl -I https://hermes.kangaroo-eel.ts.net/jobs/
|
|
curl -sS https://hermes.kangaroo-eel.ts.net/jobs/api/jobs | jq '.jobs | length'
|
|
|
|
# Local service
|
|
ss -ltnp 'sport = :9099'
|
|
curl -sS http://127.0.0.1:9099/api/jobs | jq '.jobs | length'
|
|
|
|
# Route table
|
|
tailscale serve status
|
|
|
|
# Atomic route rebuild
|
|
/home/hermes/.hermes/scripts/tailscale-serve-apply.sh --dry-run
|
|
/home/hermes/.hermes/scripts/tailscale-serve-apply.sh
|
|
```
|
|
|
|
## Backups from the incident
|
|
|
|
- `/home/hermes/workspace/index.html.radar-links-backup-20260727`
|
|
- `/home/hermes/workspace/jobs.json.radar-prompt-backup-20260727`
|
|
- `/home/hermes/workspace/config.yaml.pre-mimo-fallback-20260727`
|
|
|
|
## Related
|
|
|
|
- [[systems/scheduled-tasks]]
|
|
- [[infrastructure/tailscale]]
|
|
- [[systems/model-providers]]
|