ping_service/manager/GATEWAY.md

# Gateway Mode - External Worker Support

The manager can act as a **gateway/proxy** for external ping_service instances that cannot directly access your internal input/output services. This simplifies deployment for workers running outside your WireGuard network.

## Architecture

```
External Ping Service (Internet)
        |
        | HTTPS + API Key
        v
   Manager (Public Internet)
        |
        +---> Input Services (Private WireGuard)
        |
        +---> Output Services (Private WireGuard)
```

## Benefits

✅ **Simple Deployment**: External workers only need manager URL + API key
✅ **Single Public Endpoint**: Only manager exposed to internet
✅ **Load Balancing**: Automatic round-robin across healthy backends
✅ **Centralized Auth**: API key management from dashboard
✅ **Monitoring**: Track usage per API key
✅ **Revocable Access**: Instantly disable compromised keys

## Enabling Gateway Mode

Start the manager with the `--enable-gateway` flag:

```bash
sudo ./manager --port=443 --domain=example.dy.fi --enable-gateway
```

## API Key Management

### 1. Generate API Key (Admin)

After logging into the dashboard with TOTP, generate an API key:

```bash
curl -X POST https://example.dy.fi/api/apikeys/generate \
  -H "Cookie: auth_session=YOUR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "External Ping Worker #1",
    "worker_type": "ping"
  }'
```

**Response:**
```json
{
  "key": "xLmKj9fR3pQ2vH8nY7tW1sZ4bC6dF5gN0aE3uI2oP7kM9jL8hG4fD1qS6rT5yV3w==",
  "name": "External Ping Worker #1",
  "worker_type": "ping",
  "note": "⚠️ Save this key! It won't be shown again."
}
```

**⚠️ IMPORTANT**: Save the API key immediately - it won't be displayed again!

### 2. List API Keys

```bash
curl https://example.dy.fi/api/apikeys/list \
  -H "Cookie: auth_session=YOUR_SESSION"
```

**Response:**
```json
[
  {
    "key_preview": "xLmKj9fR...yV3w==",
    "name": "External Ping Worker #1",
    "worker_type": "ping",
    "created_at": "2026-01-07 14:23:10",
    "last_used_at": "2026-01-07 15:45:33",
    "request_count": 1523,
    "enabled": true
  }
]
```

### 3. Revoke API Key

```bash
curl -X DELETE "https://example.dy.fi/api/apikeys/revoke?key=FULL_API_KEY_HERE" \
  -H "Cookie: auth_session=YOUR_SESSION"
```

## Gateway Endpoints

### GET /api/gateway/target

Get next IP address to ping (proxies to input service).

**Authentication**: API Key (Bearer token)

**Request:**
```bash
curl https://example.dy.fi/api/gateway/target \
  -H "Authorization: Bearer YOUR_API_KEY"
```

**Response:**
```
203.0.113.42
```

### POST /api/gateway/result

Submit ping/traceroute result (proxies to output service).

**Authentication**: API Key (Bearer token)

**Request:**
```bash
curl -X POST https://example.dy.fi/api/gateway/result \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "203.0.113.1",
    "target": "203.0.113.42",
    "ping": {
      "sent": 4,
      "received": 4,
      "loss_percent": 0,
      "min_rtt": 12.3,
      "avg_rtt": 13.1,
      "max_rtt": 14.2,
      "stddev_rtt": 0.8
    },
    "traceroute": {
      "hops": [
        {"hop": 1, "ip": "192.168.1.1", "rtt": 1.2, "timeout": false},
        {"hop": 2, "ip": "10.0.0.1", "rtt": 5.3, "timeout": false},
        {"hop": 3, "ip": "203.0.113.42", "rtt": 12.3, "timeout": false}
      ]
    }
  }'
```

**Response:**
```json
{"status": "ok"}
```

## Configuring External Ping Service

For an external ping service to use the gateway, configure it with:

```bash
export MANAGER_URL="https://example.dy.fi"
export WORKER_API_KEY="xLmKj9fR3pQ2vH8nY7tW1sZ4bC6dF5gN0aE3uI2oP7kM9jL8hG4fD1qS6rT5yV3w=="
export GATEWAY_MODE="true"
```

**Modified ping service main loop:**
```go
// Get target from gateway
req, _ := http.NewRequest("GET", os.Getenv("MANAGER_URL")+"/api/gateway/target", nil)
req.Header.Set("Authorization", "Bearer "+os.Getenv("WORKER_API_KEY"))
resp, err := client.Do(req)
// ... read target IP

// Perform ping/traceroute
result := performPing(target)

// Submit result to gateway
resultJSON, _ := json.Marshal(result)
req, _ = http.NewRequest("POST", os.Getenv("MANAGER_URL")+"/api/gateway/result",
    bytes.NewBuffer(resultJSON))
req.Header.Set("Authorization", "Bearer "+os.Getenv("WORKER_API_KEY"))
req.Header.Set("Content-Type", "application/json")
resp, err = client.Do(req)
```

## Load Balancing

The gateway automatically load balances across healthy backend services:

- **Input Services**: Round-robin across all healthy input workers
- **Output Services**: Round-robin across all healthy output workers
- **Health Awareness**: Only routes to workers marked as healthy by the health poller

If a backend becomes unhealthy, it's automatically removed from the rotation until it recovers.

## Security

### API Key Security

- **256-bit keys**: Cryptographically secure random generation
- **Encrypted storage**: API keys stored with AES-256-GCM encryption
- **Bearer token auth**: Standard OAuth 2.0 bearer token format
- **Usage tracking**: Monitor request count and last used time
- **Instant revocation**: Disable keys immediately if compromised

### Rate Limiting

Gateway endpoints inherit the same rate limiting as other API endpoints:
- **100 requests/minute per IP**
- Logs `API_KEY_INVALID` attempts
- Compatible with fail2ban for IP blocking

### Logging

All gateway activity is logged:
```
API_KEY_AUTH: External Ping Worker #1 (type: ping) from IP 203.0.113.100
```

Failed authentication attempts:
```
API_KEY_MISSING: Request from IP 203.0.113.100
API_KEY_INVALID: Failed auth from IP 203.0.113.100
```

## Monitoring

### Gateway Statistics

Get current gateway pool statistics (admin only):

```bash
curl https://example.dy.fi/api/gateway/stats \
  -H "Cookie: auth_session=YOUR_SESSION"
```

**Response:**
```json
{
  "input_backends": 3,
  "output_backends": 2,
  "total_backends": 5
}
```

### Health Checks

The gateway uses the existing worker health poller to track backend availability:
- Polls every 60 seconds
- Only routes to healthy backends
- Automatic failover on backend failure

## Deployment Example

### 1. Start Manager with Gateway

```bash
# On your public server
sudo ./manager --port=443 --domain=example.dy.fi --enable-gateway
```

### 2. Register Internal Workers

From the dashboard, register your internal services:
- Input Service #1: `http://10.0.0.5:8080` (WireGuard)
- Output Service #1: `http://10.0.0.10:9090` (WireGuard)

### 3. Generate API Key

Generate an API key for your external ping worker.

### 4. Deploy External Ping Service

```bash
# On external server (e.g., AWS, DigitalOcean)
export MANAGER_URL="https://example.dy.fi"
export WORKER_API_KEY="your-api-key-here"
export GATEWAY_MODE="true"
./ping_service
```

The external ping service will:
1. Request targets from the manager gateway
2. Perform pings/traceroutes
3. Submit results back through the gateway
4. Manager forwards requests to internal services

## Troubleshooting

### "No healthy backends available"

**Problem**: Gateway returns error when requesting target or submitting results.

**Solution**:
1. Check if input/output services are registered in the dashboard
2. Verify services are marked as "Healthy" (green dot)
3. Check health poller logs: `grep "Health check" /var/log/twostepauth.log`
4. Ensure internal services are reachable from manager

### "Invalid API key"

**Problem**: Gateway rejects API key.

**Solution**:
1. Verify API key hasn't been revoked (check `/api/apikeys/list`)
2. Check key is enabled (`"enabled": true`)
3. Ensure key is sent correctly: `Authorization: Bearer <key>`
4. Check for typos or truncation in environment variable

### High Latency

**Problem**: Gateway adds latency to requests.

**Solution**:
- Gateway adds minimal overhead (~5-10ms for proxy)
- Most latency comes from: External worker → Manager → Internal service
- Consider deploying manager closer to internal services
- Use WireGuard for lower latency between manager and internal services

## Best Practices

1. **Key Rotation**: Rotate API keys periodically (e.g., every 90 days)
2. **One Key Per Worker**: Generate separate keys for each external instance
3. **Descriptive Names**: Use clear names like "AWS-US-East-1-Ping-Worker"
4. **Monitor Usage**: Review `request_count` and `last_used_at` regularly
5. **Revoke Unused Keys**: Remove keys for decommissioned workers
6. **Secure Storage**: Store API keys in environment variables, not in code
7. **Backup Keys**: Keep secure backup of active API keys

## Performance

Gateway performance characteristics:

- **Latency overhead**: ~5-10ms per request
- **Throughput**: Handles 100+ req/s per backend easily
- **Connection pooling**: Maintains persistent connections to backends
- **Concurrent requests**: Go's concurrency handles many simultaneous workers

## Future Enhancements

Potential improvements (not yet implemented):

- [ ] WebSocket support for persistent connections
- [ ] Request caching for frequently accessed targets
- [ ] Metrics endpoint (Prometheus format)
- [ ] Geographic routing (route to closest backend)
- [ ] Custom routing rules (pin worker to specific backend)
- [ ] API key scopes (restrict to specific endpoints)

---

**Last Updated**: 2026-01-07
**Version**: 1.0