- Wire host-scoped Gitea credential (git credential approve); origin was HTTPS with no userinfo - Strip 1.76GB checkpoints/ from the unpushed range via git filter-repo --partial (fast-forward preserved) - Untrack checkpoints/ at the tip; push set ~2GB -> 264.8MB - Rewrite backup.sh: exit non-zero on failure, stamp .last_backup_ok only after remote verification, decide push by remote-vs-local, guard ls-remote with || true, GitHub orphan snapshot via commit-tree, honest AgentMail/Telegram delivery reporting - Verified live: exit 0, origin/main == local HEAD, 0 unpushed, Gitea + GitHub APIs confirmed - Incident page -> status: resolved; backups.md -> RESOLVED; scheduled-tasks job row updated - Also commits the uncommitted stocktake edits (the 03:00 stocktake cron failed with 'lost its durable fire claim ownership' and left the tree dirty)
17 KiB
title, type, status, created, updated, verified_on, confidence, tags
| title | type | status | created | updated | verified_on | confidence | tags | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Proxmox VE | infrastructure | active | 2026-07-26 | 2026-11-07 | 2026-11-07 | high |
|
Proxmox VE
Purpose
Hypervisor hosting all VMs and LXC containers for the homelab.
Details
- Host:
pve.kangaroo-eel.ts.net(Tailscale:100.96.100.82) - Node name:
pve - Version: Proxmox VE 9.2.10 (
pve-manager/9.2.10/43df2e01f27a1a19, running kernel7.0.14-5-pve) — corrected 2026-10-31; this page previously said "Proxmox VE 8.x", which was wrong.m93pruns the samepve-manager 9.2.10build, so the two hosts are no longer version-skewed (see them93psection). - Primary user:
Anthony@pam(full admin) - Additional users:
root@pam,glance@pve,prom-exporter@pve,homepage@pve
Live LXC Inventory (verified 2026-11-07 via pct list)
Re-verified 2026-11-07 03:2x AWST (stocktake): live pct list on pve still returns exactly the 17 LXCs in the table below (CT255 mattermost running, CT209 cloudreve and CT555 elementsynapse stopped, CT600 omniroute running, and no CT601 on either hypervisor), and qm list on pve still returns exactly the 4 VMs (100 homeassistant, 101 guest-vps, 110 omarchy, 282 ubuntu-server-xfce — all running; no 310, no 403, no 401). Both pve-manager packages are still 9.2.10 on both hosts. The one correction was on the m93p side: see the m93p section below — CT107 (mem0) was missing, so the m93p guest count was one short.
Re-verified 2026-11-01 03:5x AWST (stocktake): pct list on pve and qm list on pve both still match the tables below exactly — 17 LXCs and 4 VMs on pve, with CT255 mattermost still running, CT209 cloudreve and CT555 elementsynapse still stopped, and no CT601 on either hypervisor (/etc/pve/lxc/601.conf absent). m93p re-confirmed: pve-manager/9.2.10 (same build as pve), qm list returns VM 103 cachyos-m93p stopped and VM 106 openmausbot-vm running, and pct exec 1001 -- docker ps returns sshwifty (CT1001). Storage percentages were refreshed against live pvesm status on both hosts (see the Storage section). Note: m93p SSH over the tailnet timed out on the first two attempts and succeeded on the third — a flaky path, not a host outage (tailscale ping → pong throughout).
Two Proxmox hosts exist — always qualify a guest with its host.
pvecm nodesis empty on both, so they are independent and unclustered. Guest IDs also collide — CT221 (gitea) and CT245 (openweb) carry the same VMID on both hosts — so an unqualified "CT221" is ambiguous.m93p(100.104.51.98, PVE 9.2.10) was discovered 2026-10-14 and is documented in its own section below. It hosts several services the wiki had placed onpve.
| VMID | Name | Status | Purpose |
|---|---|---|---|
| 103 | cliproxyapi | running | Cliproxy API |
| 200 | docker | running | Main Docker host — 60+ containers (CT460's docker.sock is NOT accessible to the agent user; use CT200 for fleet inspection) |
| 201 | runtipi | running | Runtipi — Homepage, ntfy, etc. |
| 205 | ollama | running | Ollama LLM inference |
| 207 | webserver | running | Web server |
| 209 | cloudreve | stopped | Cloudreve (deprecated) |
| 215 | n8n | running | N8n automation |
| 255 | mattermost | running | Mattermost — back online 2026-10-30/31 after a ~40-day outage; systemctl is-active mattermost → active, listening on :8065, gateway reconnected 2026-10-30 16:36:17 |
| 270 | nextcloud | running | Nextcloud (apache2 active) |
| 299 | sftpgo | running | SFTPGo file server |
| 333 | clawtest | running | ClawTest — Nanobot, Zeroclaw |
| 450 | hermesagent | running | Hermes Agent CT450 (Chaos Lab) |
| 460 | hermes | running | Hermes CT460 (production) |
| 465 | qwenpaw | running | QwenPaw + Paseo daemon (active) |
| 475 | agentzero | running | AgentZero (100.106.54.94) — not previously recorded |
| 555 | elementsynapse | stopped | Element Synapse (deprecated) — also retired from Tailscale (offline 102d) |
| 600 | omniroute | running | OmniRoute — live server on :20128 (see systems/omniroute) |
CT601 (omni) no longer exists. The 2026-10-14 inventory listed it as **stopped**; live
ls /etc/pve/lxc/601.conf on both hypervisors returns no such file, and the orphaned omni
Tailscale peer (100.93.204.4) has expired out of the tailnet entirely — the live
tailscale status --json contains no peer named omni and no peer at that IP. The guest was
deleted, not merely halted. The only OmniRoute nodes left are CT600 (omniroute,
100.88.81.19) and CT101 (omniroute-1, 100.78.63.20, on m93p) — both verified running with
omniroute listening on 0.0.0.0:20128. → systems/omniroute, infrastructure/hosts
⚠ Four entries moved off this host (corrected 2026-10-14)
The 2026-10-11 inventory listed CT221 (gitea), CT241 (ntfy), CT245 (openweb) and CT260
(freshrss) as guests of pve. They are absent from live pct list on pve — yet all four
services are still up, because they live on m93p. /etc/pve/lxc/ on pve holds only
260.conf.bak (no active config) and no vzdump task log references CT 221/241/245/260, so these
were never deleted here: they were never on this host. The 2026-10-11 audit carried the older
error forward.
| Wiki entry (removed) | Real host | Real VMID |
|---|---|---|
m93p |
CT221 — same ID, different host (coincidence) | |
m93p |
CT1241 | |
m93p |
CT245 — same ID, different host | |
m93p |
CT1260 |
Live QEMU Inventory (verified 2026-10-31 via qm list)
| VMID | Name | Status | Notes |
|---|---|---|---|
| 100 | homeassistant | running | Home Assistant (5100 MB RAM — the page previously said 13 GB) |
| 101 | guest-vps | running | Guest VPS (6144 MB RAM, 48 GB disk) — was missing from this table entirely |
| 110 | omarchy | running | Omarchy (13000 MB RAM, 106 GB disk) |
| 282 | ubuntu-server-xfce | running | Ubuntu desktop (5000 MB RAM, 133 GB disk) — recorded stopped until 2026-10-14; live qm list shows running |
| 310 | cachyos | absent | CachyOS — no longer present in qm list on pve; the only CachyOS guest is cachyos-m93p (VM 103) on m93p, which is stopped |
| 403 | open-claw | — | OpenClaw agent; not present in qm list — it runs elsewhere (Open-Claw VM, see infrastructure/hosts) |
| 401 | hermes-old | — | Legacy Hermes VM; not present in qm list — appears decommissioned/removed |
Not in either list: VM 403 (open-claw) and VM 401 (hermes-old) are referenced by the wiki but no longer appear in qm list. They have either been migrated off this host or deleted. Status unconfirmed — verify before relying on either entry.
Re-verified 2026-10-31: live qm list on pve returns exactly 4 VMs — 100 homeassistant, 101
guest-vps, 110 omarchy, 282 ubuntu-server-xfce, all running. Two corrections to the 2026-10-14 table:
VM 101 (guest-vps) was missing despite running, and VM 310 (cachyos) no longer exists on this
host (it had been recorded as stopped; the live qm list has no 310 at all). Home Assistant's memory was
also corrected from "13GB" to the live 5100 MB.
Host m93p (100.104.51.98) — second Proxmox hypervisor, discovered 2026-10-14
Identified because a tailnet-wide scan for :8006 returned only m93p and pve. It had been
sitting in the Tailscale node list as an unremarkable online peer, but no wiki page recorded it
as a hypervisor.
- Hostname:
m93p· Tailscale:100.104.51.98 - PVE version: 9.2.10 (
pve-manager/9.2.10/43df2e01f27a1a19, kernel7.0.14-12-pve) — corrected 2026-10-31. The previous entry read "9.2.0" and claimedm93pwas "materially newer than pve's 8.x". Both hosts now run the identicalpve-manager 9.2.10build; only the kernel differs (7.0.14-12-pvevs7.0.14-5-pve). The hosts are version-identical, not version-skewed. - Uptime: up 1 day, 7:18 at 2026-11-07 04:38 AWST (live
uptime -s→ boot 2026-11-05 21:19:52;uptime→up 1 day, 7:18, load average 0.76 / 1.08 / 1.24 — normal). ⚠ Corrected 2026-11-07: the previous entry read "up 1 day, 12:02 at 2026-11-01 03:56 AWST — the host rebooted ~2026-10-30 16:0x". The reboot itself is still real (pvebooted 2026-10-30 16:24:00 peruptime -s, and the CT1264 openmausbot outage below follows from it), but the uptime figure and the implied boot date were wrong — the host has since rebooted again on 2026-11-05 21:19:52, so it was up only 1 day, 7:18 when probed, not 12 h. Thepvehost's own boot time is 2026-10-30 16:24:00 (up 7 days, 12:14 at probe time) - Not clustered with
pve:pvecm nodesempty, no/etc/pve/corosync.conf
LXC Inventory (verified 2026-10-31 via pct list on m93p)
| VMID | Name | Status | Notes |
|---|---|---|---|
| 100 | obsidian-livesync | running | Obsidian LiveSync — ID does not collide; pve has no CT100 |
| 101 | omniroute | running | Second OmniRoute node, 100.78.63.20 (omniroute-1 Tailscale peer), omniroute listening on :20128 — identity/role now confirmed, see systems/omniroute |
| 102 | vaultwarden | running | Vaultwarden password manager |
| 104 | rclone | running | Rclone |
| 105 | conduit | running | Conduit — iPhone calendar / health / location / contacts bridge (see systems/messaging-integrations). Added 2026-10-31 — this guest was missing from the 2026-10-14 table despite running |
| 107 | mem0 | running | mem0 MCP server (mem0-mcp.service → :8890), Tailscale 100.73.94.53 (mem0 peer) — added 2026-11-07, this guest was missing from the table despite running |
| 221 | gitea | running | This is the host behind the wiki's gitea node — the Gitea remote lives here, not on pve |
| 245 | openweb | running | Open WebUI — pve has no live CT245; the guest this page previously recorded on pve is this one |
| 500 | litellm | running | LiteLLM proxy |
| 998 | nightbridge | stopped | Nightbridge |
| 1001 | docker-01 | running | Secondary Docker host |
| 1006 | freellmapi | running | FreeLLM API |
| 1007 | homebridge | running | Homebridge |
| 1008 | rustdesk | running | RustDesk remote access |
| 1241 | ntfy | running | ntfy push notifications — wiki's CT241 |
| 1260 | freshrss | running | FreshRSS feed reader — wiki's CT260 |
| 1261 | paseo | running | Paseo coding-agent daemon |
| 1262 | mailflow | running | MailFlow |
| 1263 | social-dashboard | running | Social dashboard |
| 1264 | openmausbot | stopped | OpenMausBot — stopped, which is why the openmausbot Tailscale peer is offline 9d. Added 2026-10-31 — was missing from the 2026-10-14 table. The companion VM106 openmausbot-vm is running and holds the separate openmausbot-vm peer (100.77.107.30) |
Re-verified 2026-11-07 03:2x AWST (stocktake): live pct list on m93p returns 20 LXCs — 18 running,
2 stopped (CT998 nightbridge and CT1264 openmausbot). ⚠ The count was one short: the previous
figure of 19 omitted CT107 (mem0), a live running guest confirmed from inside the container
(pct exec 107 -- tailscale ip -4 → 100.73.94.53, systemctl is-active tailscaled → active,
mem0-mcp.service → active). CT107 is now in the table above.
Re-verified 2026-10-31: the earlier reading of 19 LXCs — 18 running, 1 stopped
(CT998 nightbridge) plus CT1264 openmausbot stopped was itself already one short for the same reason
(CT107 mem0 was not counted). The 2026-10-14 table listed only 17, omitting
two live guests: CT105 (conduit) and CT1264 (openmausbot). Note that VMIDs 103, 220 and 555
do not exist on m93p (/etc/pve/lxc/{103,220,555}.conf all absent) — CT103 (cliproxyapi),
CT220 (caddy) and CT555 (elementsynapse) are on pve only, and pve's CT555 is stopped while
m93p has no 555 at all. Guest IDs still collide across hosts, so always qualify with the host.
QEMU Inventory (verified 2026-10-31 via qm list on m93p)
| VMID | Name | Status | Notes |
|---|---|---|---|
| 103 | cachyos-m93p | stopped | CachyOS (6144 MB RAM, 80 GB disk) |
| 106 | openmausbot-vm | running | OpenMausBot VM (8192 MB RAM, 30 GB disk) — Added 2026-10-31; was missing from the 2026-10-14 table. Holds the openmausbot-vm Tailscale peer (100.77.107.30) |
Reachability note: guest service ports on m93p were not reachable from CT460 during this
pass (they bind to their own interfaces and are not Tailscale-Served). Only hypervisor :8006 on
100.104.51.98 is open. Reach them via the host or a Tailscale Serve route before probing services.
Re-confirmed 2026-10-31: ports 3000, 4000, 5432, 8766, 8080 and 9000 were all closed from CT460.
✅ Follow-up closed 2026-10-31: m93p's omniroute (CT101) is a second live OmniRoute node,
not a duplicate of CT600/CT601. Live pct config 101 on m93p confirms hostname: omniroute, and
omniroute is listening on 0.0.0.0:20128 inside it; the omniroute-1 Tailscale peer
(100.78.63.20) resolves to it. CT601 no longer exists on either host — /etc/pve/lxc/601.conf is
absent from pve and m93p, and the orphaned omni peer (100.93.204.4) has expired out of the
tailnet. → systems/omniroute, infrastructure/hosts
Storage (live, 2026-11-01 03:5x AWST — pvesm status on both hosts)
Host pve (100.96.100.82)
| Datastore | Type | Status | Total | Used | Used % |
|---|---|---|---|---|---|
ssd0 (/mnt/ssd0, /dev/sdb) |
dir | active | 916 GiB | 717 GiB | 78.29 % (pvesm) |
SSD1 |
lvmthin | active | 931 GiB | 610 GiB | 65.46 % |
fred-qnap |
nfs | active | 13.4 TiB | 11.3 TiB | 84.21 % |
local |
dir | active | 67.7 GiB | 55.3 GiB | 81.62 % |
local-lvm |
lvmthin | active | 141 GiB | 20.1 GiB | 14.25 % |
ssd0 has improved further: 88.17 % (2026-10-14) → 76.43 % (2026-11-01) → 78.29 % (2026-11-07), ~152 GiB free. fred-qnap is now
84.21 % and its type changed from cifs to nfs (13.4 TiB total, 11.3 TiB used) — the share was
remounted over NFS rather than SMB.
Storage re-verified 2026-11-07 03:2x AWST against live pvesm status on both hosts — every
percentage on both tables below was refreshed.
iMacHDD no longer appears in pvesm status on pve — the 2026-10-14 table listed it as an
inactive cifs entry. The iMac shares now live on m93p as cifs-anthonyimac and cifs-imachdd
(both active, 76.21 %). Storage was reassigned from pve to m93p rather than the mount failing.
Host m93p (100.104.51.98)
| Datastore | Type | Status | Total | Used | Used % |
|---|---|---|---|---|---|
cifs-anthonyimac |
dir | active | 93.9 GiB | 71.6 GiB | 76.21 % |
cifs-imachdd |
dir | active | 93.9 GiB | 71.6 GiB | 76.21 % |
local |
dir | active | 93.9 GiB | 71.6 GiB | 76.21 % |
local-hdd |
dir | active | 93.9 GiB | 71.6 GiB | 76.21 % |
nfs-proxmox-31 |
dir | active | 13.4 TiB | 11.3 TiB | 84.21 % |
nfs-proxmox-54 |
dir | active | 13.4 TiB | 11.3 TiB | 84.21 % |
nfs-ssd0-39 |
dir | active | 916 GiB | 717 GiB | 78.29 % |
omb-internal |
dir | active | 93.9 GiB | 60.7 GiB | 64.66 % |
m93p storage was not documented before 2026-10-31. Note that m93p exposes the same backing
volumes as pve under different names — nfs-proxmox-31/nfs-proxmox-54 mirror pve's fred-qnap
(both 84.21 %), and nfs-ssd0-39 mirrors pve's ssd0 (both 78.29 %). The iMac shares that were
iMacHDD on pve are cifs-anthonyimac / cifs-imachdd on m93p.
Mount points on the host (ls -d /mnt/*): /mnt/ssd0, /mnt/pve, /mnt/vm, /mnt/rootfs, plus per-guest mounts /mnt/ct200, /mnt/vm110-esp-ro, /mnt/vm282, /mnt/vm283, /mnt/vm400, /mnt/vm402, /mnt/esp110, /mnt/test-nfs, /mnt/tmp and retired ones (/mnt/old-openclaw-ro, /mnt/openclaw-old, /mnt/vm400-backup, /mnt/vm400-export, /mnt/vm402-fix).
Use the pvesm number when comparing across hosts — it is the only one that means the same thing on
both hypervisors.
API Tokens
homepage@pve!homepage— PVEAuditor role, privsep=0 (used by Homepage dashboard)
Key Paths
/etc/pve/— PVE config directory/var/lib/vz/— Local storage/etc/pve/priv/token/— API token storage
Related
- systems/homepage-dashboard — Dashboard shows PVE stats
- infrastructure/hosts — Full host inventory
- infrastructure/network-map — Network topology