Hub2 Deployment Runbook¶

For: Adding new services/domains to CharlieHub Last Updated: 2026-02-09 Author: DevOps Team

Quick Checklist¶

Before and after deploying a service to Hub2, follow this checklist:

BEFORE DEPLOYMENT
-----------------
[ ] Service container is running (pct, Docker, or terraform)
[ ] Backend service responds to health check (curl http://ip:port/health)
[ ] Service is reachable from hub2 (ping + curl)

DURING DEPLOYMENT
-----------------
[ ] Domain is registered with charliehub-domain
[ ] Changes are deployed (--deploy flag or charliehub-domain deploy)
[ ] DNS resolves correctly
[ ] Public URL is accessible

AFTER DEPLOYMENT
----------------
[ ] charliehub-audit-report shows the change
[ ] Service is in git log
[ ] Team is notified of the change

Step-by-Step: Adding a New Service¶

Scenario: Deploy a service on CT1935 (10.44.1.235:8000) and expose as rodent.microshare.eu¶

Step 1: Verify the service is running

# Verify from hub2:
ssh hub2
ping 10.44.1.235
curl http://10.44.1.235:8000/health

# Should return: 200 OK with health status

Step 2: Register the domain with charliehub-domain

# Option A: Simple command with auto-deploy
charliehub-domain add rodent.microshare.eu 10.44.1.235:8000 --no-auth --deploy

# Option B: Create first, deploy later
charliehub-domain add rodent.microshare.eu 10.44.1.235:8000 --no-auth
charliehub-domain deploy

Step 3: Verify DNS

# Should resolve to hub2's IP (51.68.235.106)
dig rodent.microshare.eu +short

Step 4: Test the public URL

# Should return 200 OK or expected response
curl -I https://rodent.microshare.eu/
curl -I https://rodent.microshare.eu/video/MSD10_1770593933

Step 5: Verify audit trail

# Should show your domain creation
charliehub-audit-report | tail -10

# Or check git
git log --oneline | head -5

Done! The service is now live.

Troubleshooting¶

Service returns 404¶

# Verify domain exists
charliehub-domain list | grep rodent

# If not listed:
charliehub-domain add rodent.microshare.eu 10.44.1.235:8000 --no-auth --deploy

# If listed:
charliehub-domain deploy  # Ensure changes are applied
sleep 2
curl -I https://rodent.microshare.eu/

Service returns 503 (Service Unavailable)¶

# Check backend is actually running
ssh px1  # Or wherever the container is
pct exec 1935 -- curl http://localhost:8000/health

# If backend is down, start it first
# Then redeploy:
charliehub-domain deploy

DNS not resolving¶

# Check domain is registered
charliehub-domain list

# Check DNS propagation (may take 1-2 minutes)
dig rodent.microshare.eu +short
nslookup rodent.microshare.eu

# If still not resolving after 5 minutes:
- Check with OVH DNS API
- See AGENT_TROUBLESHOOTING.md for DNS issues

API returns error¶

# Run the agent help
/opt/charliehub/agent-help

# Or check troubleshooting guide
cat /opt/charliehub/AGENT_TROUBLESHOOTING.md

# Common errors:
# - "Permission denied" → Files are read-only (use API)
# - "401 Unauthorized" → API key not set
# - "422 Unprocessable Entity" → Missing required field
# - "409 Conflict" → Domain already exists

Advanced: Updating an Existing Domain¶

If a service's backend IP or port changes:

# Find domain ID
charliehub-domain list | grep yourdomain

# Update (ID = 27 in this example)
charliehub-domain update 27 10.44.1.250:9000 --deploy

# Verify
charliehub-domain get 27

⚠️ What NOT to Do¶

❌ DO NOT edit Traefik YAML files directly:

sudo nano /opt/charliehub/traefik/config/dynamic/something.yml  # WRONG
sudo tee /opt/charliehub/traefik/config/dynamic/new.yml        # WRONG

✅ DO THIS INSTEAD:

charliehub-domain add rodent.microshare.eu 10.44.1.235:8000 --deploy

Why? - Direct edits bypass validation - No audit trail - Can break other domains - Git doesn't track changes properly - Harder to debug later

Integration with Deployment Pipelines¶

In Terraform¶

# After deploying container:
resource "null_resource" "register_domain" {
  provisioner "local-exec" {
    command = <<-EOT
      charliehub-domain add \
        rodent.microshare.eu \
        ${module.ct1935.container_ip}:8000 \
        --no-auth \
        --deploy
    EOT
    environment = {
      DOMAIN_MANAGER_API_KEY = var.api_key
    }
  }
  depends_on = [
    module.ct1935.container
  ]
}

In Ansible¶

- name: Register domain with charliehub
  shell: |
    export DOMAIN_MANAGER_API_KEY={{ domain_manager_api_key }}
    charliehub-domain add \
      rodent.microshare.eu \
      {{ container_ip }}:8000 \
      --no-auth \
      --deploy
  environment:
    DOMAIN_MANAGER_API_KEY: "{{ domain_manager_api_key }}"
  register: domain_result
  changed_when: "'created' in domain_result.stdout"

In Shell Scripts¶

#!/bin/bash
set -e

export DOMAIN_MANAGER_API_KEY="$API_KEY"

# Deploy container (example)
pct create ...

# Register domain
charliehub-domain add myservice.charliehub.net 10.44.1.100:8000 --deploy

# Verify
sleep 2
curl https://myservice.charliehub.net/ || exit 1

echo "✓ Deployment complete"

Emergency: Rollback a Domain¶

If something goes wrong:

# Find the domain ID
charliehub-domain list

# Delete it
charliehub-domain delete 27

# Or revert to previous state
git log --oneline | head -5
git revert <commit-hash>
charliehub-domain deploy

Reference¶

Command	Purpose
`charliehub-domain add domain ip:port [--no-auth] [--deploy]`	Create domain
`charliehub-domain update id ip:port [--deploy]`	Update domain
`charliehub-domain delete id`	Remove domain
`charliehub-domain list`	Show all domains
`charliehub-domain get id`	Show domain details
`charliehub-domain deploy`	Apply changes to Traefik
`charliehub-domain help`	Show help

Support¶

For API issues: - Run /opt/charliehub/agent-help - Read /opt/charliehub/AGENT_TROUBLESHOOTING.md

For infrastructure issues: - Read /opt/charliehub/CLAUDE-TRAEFIK.md - Ask DevOps team

For general questions: - Read /opt/charliehub/AGENT_START_HERE.md

Last updated: 2026-02-09 Contact: DevOps Team