A Dead Tunnel Is the Outage Nothing Reports

Every other outage announces itself. A server falls over and stops sending metrics; a website goes down and a probe fails. A tunnel is different: when it dies, the machines behind it are still running perfectly and still trying to report. What breaks is the path their reports travel on, so the symptom is an absence rather than an error, and an absence is what a quiet weekend looks like too.

For an MSP that is the worst possible failure mode, because the blast radius is a whole client site rather than one host. Everything behind the tunnel goes dark at once, and nothing in the tool that is supposed to be watching it says anything is wrong.

So the tunnel gets watched as its own thing. The agent reads wg show on the host that terminates the tunnel and reports, per peer, how long ago it last completed a handshake. A peer with keepalive that has stopped handshaking has stopped carrying traffic - that is the signal, and it arrives seconds after it happens rather than whenever someone notices the client has been quiet.

WireGuard is silent by design. It does not keep a session open or send heartbeats of its own; it handshakes when there is data to send, and rekeys roughly every two minutes while traffic flows. Stop the traffic and the handshakes stop too, which means a peer that has been idle since Friday has a handshake age of three days and is in perfect health.

Alerting on handshake age alone therefore pages you for peers that are fine, and no threshold fixes it: any cutoff long enough to spare a genuinely idle peer is far too long to catch a dead one. It is a false positive by construction rather than a number that needs tuning.

The way out is to watch only the peers that have a guaranteed cadence. A peer with PersistentKeepalive set sends a packet every few seconds whether or not it has anything to say, so its handshake age genuinely does track its health - and that is the normal configuration for a site-to-site tunnel anyway, because at least one end usually sits behind NAT and needs keepalive to hold the mapping open. The alert watches those peers by default and leaves the rest alone.

The tradeoff is stated rather than hidden: a peer without keepalive is not watched by default, so set PersistentKeepalive = 25 on the NAT side of each tunnel you want covered. If your tunnels carry constant traffic you can flip the alert to watch every peer instead, and accept that idle ones will fire.

One alert ships as a one-click workflow template: VPN Tunnel Down. It fires per peer, so a hub with thirty tunnels opens one incident for the tunnel that broke rather than one for the host, and each closes on its own when that peer handshakes again. The default threshold is 5 minutes, which is also the floor the form will accept - a rekey fires on the first packet after two minutes and the scrape adds up to another minute, so a healthy keepalive tunnel can legitimately show an age near three minutes and a tighter cutoff would flap.

If wg itself stops working - the binary goes missing, the capability is dropped - the agent keeps reporting everything else, and a naive implementation would see no stale peers and close every open incident as recovered. That would be a dead tunnel reported as fixed. Instead the alert freezes: it stops paging, it stops resolving, and it waits for real data. Resolutions are worded honestly too, so a peer that was removed from the config says it is no longer reported rather than claiming it recovered.

Tailscale has no dedicated trigger - you alert on it with a metric threshold, which is the same mechanism as any other metric here. The recipe worth setting up is key-expiry days below seven, so you are warned while the fix is still a login rather than after the node has already dropped off the tailnet. Node state works as a threshold too, because an off-tailnet daemon reports a real zero rather than a gap. What a threshold cannot see is a daemon that is gone entirely, since there is then no value to compare - for that, watch the tailscaled unit with a systemd trigger.

Every one of the 9 metrics can carry a threshold, with the same sustained-duration rules, escalation gates and maintenance windows as any other alert. Incidents never carry a public key: the identifier attached to one is a hash, and the notification names the peer by its alias.

Alerts go where your team works: email, Slack, Microsoft Teams, Telegram, Discord, Pushover, PagerDuty or webhooks. Which of those you can use depends on your plan - email, Telegram and Discord are on every plan, and the rest unlock as you move up.

WireGuard monitoring is Linux-only, and that is a real boundary rather than a roadmap gap. The collector parses the output of wg show all dump, which is the Linux and BSD userspace tooling, so a Linux host is where peer data comes from. The toggle is hidden on Windows outright; on anything else it is available, and a machine without that tooling simply reports nothing rather than reporting a broken tunnel.

Tailscale monitoring runs on Linux, Windows and macOS, because tailscale status is identical on all three. So a mixed fleet is covered for the Tailscale half wherever it runs, and for the WireGuard half wherever the tunnels actually terminate - which on a site-to-site setup is a Linux box in nearly every case.

Both collectors are off until you turn them on, per host, and enabling one does not enable the other. Both also need an agent new enough to carry them: an older one will let you turn the toggle on and then simply report nothing, so update the agent if a host stays empty after you enable it.

What the agent reads, module by module, is documented in the agent security model, which lists WireGuard and Tailscale alongside every other optional collector.