ping_service/manager/GATEWAY.md

Gateway Mode - External Worker Support

The manager can act as a gateway/proxy for external ping_service instances that cannot directly access your internal input/output services. This simplifies deployment for workers running outside your WireGuard network.

Architecture

External Ping Service (Internet)
        |
        | HTTPS + API Key
        v
   Manager (Public Internet)
        |
        +---> Input Services (Private WireGuard)
        |
        +---> Output Services (Private WireGuard)

Benefits

✅ Simple Deployment: External workers only need manager URL + API key ✅ Single Public Endpoint: Only manager exposed to internet ✅ Load Balancing: Automatic round-robin across healthy backends ✅ Centralized Auth: API key management from dashboard ✅ Monitoring: Track usage per API key ✅ Revocable Access: Instantly disable compromised keys

Enabling Gateway Mode

Start the manager with the --enable-gateway flag:

sudo ./manager --port=443 --domain=example.dy.fi --enable-gateway

API Key Management

1. Generate API Key (Admin)

After logging into the dashboard with TOTP, generate an API key:

curl -X POST https://example.dy.fi/api/apikeys/generate \
  -H "Cookie: auth_session=YOUR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "External Ping Worker #1",
    "worker_type": "ping"
  }'

Response:

{
  "key": "xLmKj9fR3pQ2vH8nY7tW1sZ4bC6dF5gN0aE3uI2oP7kM9jL8hG4fD1qS6rT5yV3w==",
  "name": "External Ping Worker #1",
  "worker_type": "ping",
  "note": "⚠️ Save this key! It won't be shown again."
}

⚠️ IMPORTANT: Save the API key immediately - it won't be displayed again!

2. List API Keys

curl https://example.dy.fi/api/apikeys/list \
  -H "Cookie: auth_session=YOUR_SESSION"

Response:

[
  {
    "key_preview": "xLmKj9fR...yV3w==",
    "name": "External Ping Worker #1",
    "worker_type": "ping",
    "created_at": "2026-01-07 14:23:10",
    "last_used_at": "2026-01-07 15:45:33",
    "request_count": 1523,
    "enabled": true
  }
]

3. Revoke API Key

curl -X DELETE "https://example.dy.fi/api/apikeys/revoke?key=FULL_API_KEY_HERE" \
  -H "Cookie: auth_session=YOUR_SESSION"

Gateway Endpoints

GET /api/gateway/target

Get next IP address to ping (proxies to input service).

Authentication: API Key (Bearer token)

Request:

curl https://example.dy.fi/api/gateway/target \
  -H "Authorization: Bearer YOUR_API_KEY"

Response:

203.0.113.42

POST /api/gateway/result

Submit ping/traceroute result (proxies to output service).

Authentication: API Key (Bearer token)

Request:

curl -X POST https://example.dy.fi/api/gateway/result \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "203.0.113.1",
    "target": "203.0.113.42",
    "ping": {
      "sent": 4,
      "received": 4,
      "loss_percent": 0,
      "min_rtt": 12.3,
      "avg_rtt": 13.1,
      "max_rtt": 14.2,
      "stddev_rtt": 0.8
    },
    "traceroute": {
      "hops": [
        {"hop": 1, "ip": "192.168.1.1", "rtt": 1.2, "timeout": false},
        {"hop": 2, "ip": "10.0.0.1", "rtt": 5.3, "timeout": false},
        {"hop": 3, "ip": "203.0.113.42", "rtt": 12.3, "timeout": false}
      ]
    }
  }'

Response:

{"status": "ok"}

Configuring External Ping Service

For an external ping service to use the gateway, configure it with:

export MANAGER_URL="https://example.dy.fi"
export WORKER_API_KEY="xLmKj9fR3pQ2vH8nY7tW1sZ4bC6dF5gN0aE3uI2oP7kM9jL8hG4fD1qS6rT5yV3w=="
export GATEWAY_MODE="true"

Modified ping service main loop:

// Get target from gateway
req, _ := http.NewRequest("GET", os.Getenv("MANAGER_URL")+"/api/gateway/target", nil)
req.Header.Set("Authorization", "Bearer "+os.Getenv("WORKER_API_KEY"))
resp, err := client.Do(req)
// ... read target IP

// Perform ping/traceroute
result := performPing(target)

// Submit result to gateway
resultJSON, _ := json.Marshal(result)
req, _ = http.NewRequest("POST", os.Getenv("MANAGER_URL")+"/api/gateway/result",
    bytes.NewBuffer(resultJSON))
req.Header.Set("Authorization", "Bearer "+os.Getenv("WORKER_API_KEY"))
req.Header.Set("Content-Type", "application/json")
resp, err = client.Do(req)

Load Balancing

The gateway automatically load balances across healthy backend services:

• Input Services: Round-robin across all healthy input workers
• Output Services: Round-robin across all healthy output workers
• Health Awareness: Only routes to workers marked as healthy by the health poller

If a backend becomes unhealthy, it's automatically removed from the rotation until it recovers.

Security

API Key Security

• 256-bit keys: Cryptographically secure random generation
• Encrypted storage: API keys stored with AES-256-GCM encryption
• Bearer token auth: Standard OAuth 2.0 bearer token format
• Usage tracking: Monitor request count and last used time
• Instant revocation: Disable keys immediately if compromised

Rate Limiting

Gateway endpoints inherit the same rate limiting as other API endpoints:

• 100 requests/minute per IP
• Logs API_KEY_INVALID attempts
• Compatible with fail2ban for IP blocking

Logging

All gateway activity is logged:

API_KEY_AUTH: External Ping Worker #1 (type: ping) from IP 203.0.113.100

Failed authentication attempts:

API_KEY_MISSING: Request from IP 203.0.113.100
API_KEY_INVALID: Failed auth from IP 203.0.113.100

Monitoring

Gateway Statistics

Get current gateway pool statistics (admin only):

curl https://example.dy.fi/api/gateway/stats \
  -H "Cookie: auth_session=YOUR_SESSION"

Response:

{
  "input_backends": 3,
  "output_backends": 2,
  "total_backends": 5
}

Health Checks

The gateway uses the existing worker health poller to track backend availability:

• Polls every 60 seconds
• Only routes to healthy backends
• Automatic failover on backend failure

Deployment Example

1. Start Manager with Gateway

# On your public server
sudo ./manager --port=443 --domain=example.dy.fi --enable-gateway

2. Register Internal Workers

From the dashboard, register your internal services:

• Input Service #1: http://10.0.0.5:8080 (WireGuard)
• Output Service #1: http://10.0.0.10:9090 (WireGuard)

3. Generate API Key

Generate an API key for your external ping worker.

4. Deploy External Ping Service

# On external server (e.g., AWS, DigitalOcean)
export MANAGER_URL="https://example.dy.fi"
export WORKER_API_KEY="your-api-key-here"
export GATEWAY_MODE="true"
./ping_service

The external ping service will:

1. Request targets from the manager gateway
2. Perform pings/traceroutes
3. Submit results back through the gateway
4. Manager forwards requests to internal services

Troubleshooting

"No healthy backends available"

Problem: Gateway returns error when requesting target or submitting results.

Solution:

1. Check if input/output services are registered in the dashboard
2. Verify services are marked as "Healthy" (green dot)
3. Check health poller logs: grep "Health check" /var/log/twostepauth.log
4. Ensure internal services are reachable from manager

"Invalid API key"

Problem: Gateway rejects API key.

Solution:

1. Verify API key hasn't been revoked (check /api/apikeys/list)
2. Check key is enabled ("enabled": true)
3. Ensure key is sent correctly: Authorization: Bearer <key>
4. Check for typos or truncation in environment variable

High Latency

Problem: Gateway adds latency to requests.

Solution:

• Gateway adds minimal overhead (~5-10ms for proxy)
• Most latency comes from: External worker → Manager → Internal service
• Consider deploying manager closer to internal services
• Use WireGuard for lower latency between manager and internal services

Best Practices

1. Key Rotation: Rotate API keys periodically (e.g., every 90 days)
2. One Key Per Worker: Generate separate keys for each external instance
3. Descriptive Names: Use clear names like "AWS-US-East-1-Ping-Worker"
4. Monitor Usage: Review request_count and last_used_at regularly
5. Revoke Unused Keys: Remove keys for decommissioned workers
6. Secure Storage: Store API keys in environment variables, not in code
7. Backup Keys: Keep secure backup of active API keys

Performance

Gateway performance characteristics:

• Latency overhead: ~5-10ms per request
• Throughput: Handles 100+ req/s per backend easily
• Connection pooling: Maintains persistent connections to backends
• Concurrent requests: Go's concurrency handles many simultaneous workers

Future Enhancements

Potential improvements (not yet implemented):

• WebSocket support for persistent connections
• Request caching for frequently accessed targets
• Metrics endpoint (Prometheus format)
• Geographic routing (route to closest backend)
• Custom routing rules (pin worker to specific backend)
• API key scopes (restrict to specific endpoints)

---

Last Updated: 2026-01-07 Version: 1.0