WAN IP Watcher

Monitors dynamic ISP WAN IPs and automatically updates Traefik, Authelia, and WireGuard configs when they change. Prevents 403 errors for LAN users accessing hub2 via public URLs after an ISP IP reassignment.

Overview

Property	Value
Location	hub2 (OVH Dedicated Server)
Container	charliehub_wan_watcher
Port	8003 (localhost only)
Interval	Every 5 minutes
Version	2.0 (Feb 2026)

Problem Solved

Both UK (Trooli) and FR ISP public IPs are dynamic. They were previously hardcoded in the Traefik lan-only middleware and Authelia access control rules. When the ISP reassigned an IP, LAN users hitting hub2 via public URLs would get 403 Forbidden until someone manually updated the configs.

Architecture

                                    ┌─────────────────────┐
                                    │  wan-ip-service     │
                                    │  (px1:8004)         │
                                    │                     │
                                    │  Queries external   │
                                    │  IP services        │
                                    │  (ipify, etc)       │
                                    └──────────┬──────────┘
                                               │
                                               │ HTTP /ip
                                               ▼
┌──────────────────────────────────────────────────────────────┐
│                  wan-watcher (:8003)                          │
│                                                              │
│  WAN IP Detection (priority order):                          │
│    1. External IP service (if configured) ─ UK uses this     │
│    2. UniFi API geo_info.WAN.address ─ FR uses this          │
│                                                              │
│  REST API:                                                   │
│    GET  /health                                               │
│    GET  /api/wan-ips                                          │
│    GET  /api/trusted-networks                                │
│                                                              │
│  File outputs (on change):                                   │
│    /traefik-config/lan-allowlist.yml                          │
│    /wireguard-config/wg-uk.conf (Endpoint update)            │
│    /wireguard-config/wg-fr.conf (Endpoint update)            │
│    Patches authelia configuration.yml                         │
└──────────┬───────────┬──────────────┬────────────────────────┘
           │           │              │
     ┌─────▼──┐   ┌────▼─────┐   ┌────▼─────┐
     │Traefik │   │ Authelia │   │WireGuard │
     │(watch) │   │(restart) │   │(trigger) │
     └────────┘   └──────────┘   └──────────┘

WAN IP Detection Methods

UK Site - External IP Service

The UK UCG (Cloud Gateway Ultra) sits behind the Technicolor router, so its wan1.ip is a private 192.168.100.x address. The UniFi geo_info.WAN.address is often stale when the ISP changes the IP.

Solution: A lightweight IP detection service runs on px1 that queries external services (ipify, ifconfig.me, etc.) and exposes the result via HTTP.

Property	Value
Location	px1-silverstone (10.44.1.10)
Service	wan-ip-service.service
Port	8004
Script	/opt/wan-ip-service.py
Endpoint	http://10.44.1.10:8004/ip

wan-ip-service API

# Get current public IP (plain text)
curl http://10.44.1.10:8004/ip
# 185.122.194.5

# Get status with timestamps (JSON)
curl http://10.44.1.10:8004/health
# {"ip": "185.122.194.5", "last_check": "2026-02-08T16:27:16+00:00", "last_change": "2026-02-08T16:27:16+00:00"}

Managing wan-ip-service (on px1)

# Check status
systemctl status wan-ip-service

# View logs
journalctl -u wan-ip-service -f

# Restart
systemctl restart wan-ip-service

FR Site - UniFi API

The FR UCG connects directly to the ISP, so the UniFi API geo_info.WAN.address works correctly.

Detection Priority

For each site, wan-watcher tries detection methods in order:

1. External IP URL (if configured via UK_EXTERNAL_IP_URL or FR_EXTERNAL_IP_URL)
2. UniFi API geo_info.WAN.address (true public IP from geo lookup)
3. UniFi API wan1.ip (skipped if private IP detected)
4. UniFi API uplink.ip (fallback)

Trusted Networks

The service maintains a single source of truth for all trusted source CIDRs:

CIDR	Label	Type
10.44.0.0/16	UK homelab LAN	Static
10.35.0.0/16	FR homelab LAN	Static
10.75.0.0/16	Additional LAN	Static
192.168.100.0/24	Technicolor LAN	Static
127.0.0.1/32	Localhost	Static
51.68.235.106/32	hub2 public IP	Static
UK WAN IP/32	UK WAN - Trooli	Dynamic
FR WAN IP/32	FR WAN	Dynamic

Subnet Ranges

UK and FR LANs use /16 to cover all subnets (e.g., 10.44.1.x, 10.44.2.x, etc.). Previously these were incorrectly set to /24 which only covered the .0.x subnet.

API Endpoints

Health Check

curl http://localhost:8003/health

{
  "status": "healthy",
  "last_check": "2026-02-08T16:28:12+00:00",
  "wan_ips": {
    "uk": "185.122.194.5",
    "fr": "78.116.21.175"
  },
  "sources": {
    "uk": "external",
    "fr": "unifi"
  },
  "last_change": "2026-02-08T16:28:12+00:00"
}

WAN IPs

curl http://localhost:8003/api/wan-ips

{
  "uk": {
    "ip": "185.122.194.5",
    "last_seen": "2026-02-08T16:28:12+00:00",
    "source": "external"
  },
  "fr": {
    "ip": "78.116.21.175",
    "last_seen": "2026-02-08T16:28:12+00:00",
    "source": "unifi"
  }
}

Trusted Networks

Returns the full set of trusted source CIDRs (static LAN + dynamic WAN).

curl http://localhost:8003/api/trusted-networks

{
  "networks": [
    {"cidr": "10.44.0.0/16", "label": "UK homelab LAN", "type": "static"},
    {"cidr": "10.35.0.0/16", "label": "FR homelab LAN", "type": "static"},
    {"cidr": "185.122.194.5/32", "label": "UK WAN - Trooli (auto)", "type": "dynamic"},
    {"cidr": "78.116.21.175/32", "label": "FR WAN (auto)", "type": "dynamic"}
  ],
  "updated": "2026-02-08T16:28:12+00:00"
}

DDNS Integration

The service automatically updates DNS records for domains marked with wan_watcher_enabled in the Domain Manager. When a WAN IP changes, wan-watcher queries Domain Manager for eligible domains and updates their A records via the DDNS endpoint.

Enabling DDNS Auto-Update for a Domain

To enable automatic DNS updates when the WAN IP changes, set the domain's wan_watcher_enabled field to the controller (uk or fr):

curl -X PUT http://charliehub-api:8001/api/domains/{domain_id} \
  -H "X-API-Key: {API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.verdegris.eu",
    "wan_watcher_enabled": "uk"
  }'

Field	Value	Meaning
wan_watcher_enabled	"uk"	Update A record when UK WAN IP changes
wan_watcher_enabled	"fr"	Update A record when FR WAN IP changes
wan_watcher_enabled	null	Don't auto-update (manual control)

Example: Domain Manager tracks: - Domain without wan_watcher_enabled → requires manual update

Explicit VPN Domain Binding (Feb 2026)

Instead of discovering domains via the implicit wan_watcher_enabled filter, VPN endpoints can now be explicitly configured via environment variables. This hardens control plane semantics and provides clear operational intent for infrastructure-critical WireGuard endpoints.

VPN Endpoint Binding

The uk-vpn.charliehub.net and fr-vpn.charliehub.net endpoints are explicitly bound and automatically updated when WAN IPs change:

environment:
  - UK_VPN_DOMAIN=uk-vpn.charliehub.net
  - FR_VPN_DOMAIN=fr-vpn.charliehub.net

How It Works

When WAN IP changes:

1. Tier 1 (Explicit): Direct DDNS update via /api/ddns/update/{domain}?ip={new_ip}
2. Fast path (no discovery query needed)

3. Structured logging: DDNS update success: uk-vpn.charliehub.net -> 185.x.x.x

4. Tier 2 (Fallback): Discovery-based update for other domains

5. Original behavior (queries wan_watcher_enabled domains)
6. Maintains backward compatibility
7. Falls back only if explicit domains not configured

Backward Compatibility

If explicit domains are not configured, wan-watcher falls back to the original discovery mechanism:

# If UK_VPN_DOMAIN is empty or not set:
environment:
  - UK_VPN_DOMAIN=        # Falls back to discovery

# Explicit domains take precedence:
environment:
  - UK_VPN_DOMAIN=uk-vpn.charliehub.net  # Uses explicit path

Startup Validation

At startup, wan-watcher logs which domains are bound:

2026-02-13T09:46:15 [INFO] VPN domain binding: UK=uk-vpn.charliehub.net (explicit)
2026-02-13T09:46:15 [INFO] VPN domain binding: FR=fr-vpn.charliehub.net (explicit)

Configuration

Environment variables in docker-compose.yml:

environment:
  - UNIFI_API_URL=http://unifi-api:8002
  - CHECK_INTERVAL=300
  - TRAEFIK_CONFIG_DIR=/traefik-config
  - AUTHELIA_CONFIG_DIR=/authelia-config
  - AUTHELIA_CONTAINER=charliehub_authelia
  - WIREGUARD_CONFIG_DIR=/wireguard-config
  - HUB2_PUBLIC_IP=51.68.235.106
  - DOMAIN_MANAGER_URL=http://domain-manager:8001
  - DOMAIN_MANAGER_API_KEY=${DOMAIN_MANAGER_API_KEY}
  - UK_EXTERNAL_IP_HOSTS=px1,px2,px3
  - FR_EXTERNAL_IP_HOSTS=px5

Variable	Default	Description
UNIFI_API_URL	http://unifi-api:8002	UniFi API base URL
CHECK_INTERVAL	300	Seconds between WAN IP checks
TRAEFIK_CONFIG_DIR	/traefik-config	Path to Traefik dynamic config directory
AUTHELIA_CONFIG_DIR	/authelia-config	Path to Authelia config directory
AUTHELIA_CONTAINER	charliehub_authelia	Container name for Authelia restarts
WIREGUARD_CONFIG_DIR	/wireguard-config	Path to WireGuard config directory
HUB2_PUBLIC_IP	51.68.235.106	hub2 public IP (static)
DOMAIN_MANAGER_URL	http://domain-manager:8001	Domain Manager API URL (for DDNS updates)
DOMAIN_MANAGER_API_KEY	(empty)	API key for Domain Manager DDNS integration
UK_EXTERNAL_IP_HOSTS	px1,px2,px3	SSH hosts to query for UK external IP (comma-separated)
FR_EXTERNAL_IP_HOSTS	px5	SSH hosts to query for FR external IP
LISTEN_PORT	8003	API listen port

Generated Files

Traefik: lan-allowlist.yml

Written to traefik/config/dynamic/lan-allowlist.yml. Traefik picks this up automatically via its file provider watch mode.

# Auto-generated by wan-watcher - DO NOT EDIT
# Last updated: 2026-02-08T16:28:12+00:00
http:
  middlewares:
    lan-only:
      ipAllowList:
        sourceRange:
          - 10.44.0.0/16       # UK homelab LAN
          - 10.35.0.0/16       # FR homelab LAN
          - 10.75.0.0/16       # Additional LAN
          - 192.168.100.0/24   # Technicolor LAN
          - 127.0.0.1/32       # Localhost
          - 51.68.235.106/32   # hub2 public IP
          - 185.122.194.5/32   # UK WAN - Trooli (auto)
          - 78.116.21.175/32   # FR WAN (auto)

Do Not Edit Manually

The lan-only middleware is auto-generated by wan-watcher. Manual edits will be overwritten on the next IP check.

WireGuard: Endpoint Updates

When a WAN IP changes, wan-watcher updates the Endpoint line in the corresponding WireGuard config file and writes a trigger file for the host to detect and restart WireGuard.

/etc/wireguard/wg-uk.conf  →  Endpoint=185.122.194.5:51821
/etc/wireguard/wg-fr.conf  →  Endpoint=78.116.21.175:51820

Authelia: configuration.yml

WAN IP lines in configuration.yml are patched in-place using regex. The script matches lines containing an IP/32 CIDR followed by a comment with "UK ... WAN" or "FR ... WAN" and replaces the IP. After patching, the Authelia container is restarted via Docker API.

Service Management

# View logs
docker logs charliehub_wan_watcher --tail 50

# Follow logs (useful for debugging IP changes and DDNS updates)
docker logs charliehub_wan_watcher -f

# Restart (triggers immediate IP check on startup)
docker restart charliehub_wan_watcher

# Check health
curl -s http://localhost:8003/health | jq .

# Check current WAN IPs with sources
curl -s http://localhost:8003/api/wan-ips | jq .

# Verify generated Traefik config
cat /opt/charliehub/traefik/config/dynamic/lan-allowlist.yml

Troubleshooting

UK WAN IP is wrong or stale

If the UK WAN IP doesn't match reality, check the external IP service on px1:

# From px1: Check wan-ip-service is running
systemctl status wan-ip-service

# Check what IP it's reporting
curl http://10.44.1.10:8004/ip

# Check external services directly (compare with above)
curl https://api.ipify.org

WAN IPs showing as null

The detection sources may not be available:

# Check external IP service (UK)
curl -s http://10.44.1.10:8004/health

# Check unifi-api health
curl -s http://localhost:8002/health | jq .

# Check if gateway devices are returned
curl -s "http://localhost:8002/api/devices?controller=uk" | jq '.data[] | select(.type == "udm") | {name, wan1_ip: .wan1.ip, geo_ip: .geo_info.WAN.address}'
curl -s "http://localhost:8002/api/devices?controller=fr" | jq '.data[] | select(.type == "udm") | {name, wan1_ip: .wan1.ip, geo_ip: .geo_info.WAN.address}'

Traefik not picking up changes

Verify the file was written and Traefik can see it:

# Check file exists and is recent
ls -la /opt/charliehub/traefik/config/dynamic/lan-allowlist.yml

# Check Traefik logs for config reload
docker logs charliehub-traefik --tail 20 | grep -i "config"

403 Forbidden after IP change

If you're still getting 403s after an IP change, verify the full chain:

# 1. Check wan-watcher detected the change
curl -s http://localhost:8003/health | jq '{last_change, sources, wan_ips}'

# 2. Check lan-allowlist.yml has new IP
grep -E "WAN" /opt/charliehub/traefik/config/dynamic/lan-allowlist.yml

# 3. Check Authelia config has new IP
grep -E "WAN" /opt/charliehub/authelia/config/configuration.yml

# 4. If still not updated, restart wan-watcher to trigger immediate check
docker restart charliehub_wan_watcher && docker logs charliehub_wan_watcher -f

wan-ip-service not starting (px1)

# Check service status
systemctl status wan-ip-service

# Check logs
journalctl -u wan-ip-service --no-pager -n 50

# Verify script exists
ls -la /opt/wan-ip-service.py

# Test script manually
python3 /opt/wan-ip-service.py

Related

• UniFi API - Source of WAN IP data for FR site
• Traefik Routing - Reverse proxy consuming the lan-only middleware
• Authelia SSO - SSO service with network-based bypass rules
• WireGuard - VPN tunnels with dynamic endpoint IPs
• hub2 Services - Central services hub