Observability for a homelab without the Prometheus sprawl
Metrics, logs, uptime checks and alerts for a four-node Proxmox cluster, UniFi network and NAS boxes, running on one small VM. What I picked, what I skipped, and why alerts go to Telegram.
- Published
- Length
- 3 min, 2U
- Topics
- Homelab, Observability, Proxmox
For years, “monitoring” in my lab meant noticing something was broken when I tried to use it. Plex wouldn’t load, or a backup had quietly failed for a week. This month I finally built proper observability, with three goals:
- Know something is down before I go looking for it.
- Keep a searchable history of what happened, and when.
- Don’t build a second hobby. Whatever I run has to be smaller than the thing it watches.
What it runs on
Everything lives on one Debian VM with 4 vCPUs, 8 GB of RAM and a modest disk, managed with a single Docker Compose file in Git. A deploy script pulls credentials from my secrets vault at deploy time, so nothing sensitive sits in the repository.
| Job | Tool |
|---|---|
| Metrics storage | VictoriaMetrics |
| Log storage | VictoriaLogs |
| Dashboards and alerts | Grafana |
| UniFi network and cameras | unpoller |
| Uptime and endpoint checks | Gatus |
| The VM’s own health | node_exporter |
| “Is the monitoring itself alive?” | A heartbeat to healthchecks.io |
Why VictoriaMetrics instead of Prometheus
I like Prometheus. But my biggest metric sources don’t speak it natively. Proxmox VE and Proxmox Backup Server both have a built-in “metric server” setting that pushes InfluxDB line protocol. VictoriaMetrics accepts that directly, alongside normal Prometheus scraping, so the Proxmox side needed one setting per node and no extra exporters.
It also stores the same data in noticeably less disk and memory, which matters when the goal is “smaller than the thing it watches.”
Logs without Elasticsearch
VictoriaLogs takes syslog straight from the UniFi gateway and other devices and keeps 30 days. Grafana queries it with the same interface as everything else. I don’t need full-text search across terabytes; I need to answer “what did the router say at 2:14 this morning?” quickly. It does that well on a few hundred megabytes of RAM.
Checks that look like a user
Metrics tell you a process is running. Gatus tells you whether the thing actually answers. It probes each Proxmox node’s web interface, the backup server, Home Assistant, Plex, the NAS boxes, the gateway’s DNS and outside internet DNS, every minute. If Plex returns an error page, the check fails even though the container is up.
Alerts I’ll actually read
I wrote thirteen alert rules as code: a node stops reporting, a check fails, the internet drops, a UniFi device or camera goes offline, a disk crosses a threshold, a backup datastore fills up, or a collector stops sending data.
They go to a private Telegram group through a bot. I picked Telegram because the notification arrives on my phone and watch reliably, it’s free, and a group lets me add someone else later without rebuilding anything.
Two decisions made the alerts bearable:
- Quiet hours. Between 10 p.m. and 7 a.m., warnings wait until morning. Critical alerts still come through and repeat every 30 minutes until resolved.
- Watching the watcher. If the monitoring VM itself dies, it can’t alert about it. So a tiny container pings healthchecks.io every five minutes, and healthchecks.io messages me if the pings stop. It only reports failure after three consecutive misses, which stopped a restart from paging me.
Getting to it from outside
Grafana is published through a Cloudflare Tunnel with Cloudflare Access in front. Grafana trusts the identity Access passes along, so there’s one sign-in, no Grafana passwords to manage, and nothing exposed on my connection.
What’s next
Phase one covers infrastructure health. Next on the list: sending Proxmox and backup server logs into VictoriaLogs, a morning summary message, and a few security-focused log rules. That’s the first step toward a lightweight SIEM, built on the same storage rather than a second stack.