ping_service/input_service/README.md

HTTP Input Service

A lightweight HTTP server that serves individual IPv4 addresses from cloud provider CIDR ranges and accepts discovered hop IPs from traceroute results to organically grow the target pool.

Purpose

Provides a continuous stream of IPv4 addresses to network scanning tools. Each consumer (identified by IP) receives addresses in highly interleaved order from cloud provider IP ranges, avoiding consecutive IPs from the same subnet. Accepts discovered hop IPs from output_service to expand the target pool.

Requirements

• Go 1.25+
• Cloud provider IP repository cloned at ./cloud-provider-ip-addresses/

Building

go build -ldflags="-s -w" -o http_input_service http_input_service.go

Usage

./http_input_service

Server starts on http://localhost:8080

API Endpoints

GET /

Returns a single IPv4 address per request.

curl http://localhost:8080
# Output: 13.248.118.1

Each consumer (identified by source IP) gets their own independent sequence with interleaved IPs from different subnets.

POST /hops

Accept discovered hop IPs from traceroute results.

Request Body:

{
  "hops": ["10.0.0.1", "172.16.5.3", "8.8.8.8"]
}

Response:

{
  "status": "ok",
  "received": 3,
  "added": 2,
  "duplicates": 1
}

• Validates and filters out private, multicast, loopback IPs
• Global deduplication prevents re-adding seen IPs
• Automatically adds new hops to all consumer pools

GET /status

View current service status and consumer information.

Response:

{
  "total_consumers": 2,
  "consumers": [
    {
      "consumer": "192.168.1.100",
      "remaining_cidrs": 1234,
      "has_active_gen": true,
      "total_cidrs": 5000
    }
  ],
  "state_directory": "progress_state",
  "save_interval": "30s"
}

GET /export

Export all consumer states for backup/migration.

Downloads a JSON file with all consumer progress states.

POST /import

Import previously exported consumer states.

Request: Upload JSON from /export endpoint

Features

• Subnet Interleaving - Maintains 10 active CIDR generators, rotating between them to avoid serving consecutive IPs from the same subnet
• Per-consumer state - Each client gets independent, deterministic sequence
• Deduplication - Both per-consumer and global deduplication to prevent serving duplicate IPs
• Hop Discovery - Accepts discovered traceroute hops via /hops endpoint to grow target pool organically
• Memory efficient - Loads CIDR files lazily (~5-15MB RAM usage)
• Lazy expansion - IPs generated on-demand from CIDR notation
• Persistent state - Progress saved every 30s, survives restarts
• State export/import - Backup and migrate consumer states between instances
• IPv4 only - Filters IPv6, multicast, network/broadcast, private addresses
• Graceful shutdown - Ctrl+C drains connections cleanly

Expected Input Format

Scans ./cloud-provider-ip-addresses/ for .txt files containing IP ranges:

# Comments ignored
13.248.118.0/24
52.94.0.0/16
3.5.140.0/22

How Interleaving Works

To avoid consecutive IPs from the same subnet (e.g., 8.8.8.1, 8.8.8.2, 8.8.8.3), the service:

1. Maintains 10 active CIDR generators concurrently
2. Rotates between them in round-robin fashion
3. Each request pulls from the next generator in sequence

Example output:

9.9.9.1          # From CIDR 9.9.9.0/29
208.67.222.1     # From CIDR 208.67.222.0/29
1.1.1.1          # From CIDR 1.1.1.0/29
8.8.8.1          # From CIDR 8.8.8.0/29
8.8.4.1          # From CIDR 8.8.4.0/29
9.9.9.2          # Back to first CIDR
208.67.222.2     # Second CIDR
...

This ensures diverse network targeting and better coverage.

Integration with Output Service

The /hops endpoint is designed to receive discovered hop IPs from output_service:

# Example from output_service
curl -X POST http://localhost:8080/hops \
  -H "Content-Type: application/json" \
  -d '{"hops": ["10.0.0.1", "172.16.5.3", "8.8.8.8"]}'

• Output service extracts intermediate hops from traceroute results
• POSTs them to input service /hops endpoint
• Input service validates, deduplicates, and adds to target pool
• Future consumers will receive these discovered IPs

This creates a feedback loop where the system organically discovers new targets through network exploration.

Graceful Shutdown

Press Ctrl+C for graceful shutdown with 10s timeout. All consumer states are saved before exit.

Multi-Instance Deployment

Each instance maintains its own consumer state files in progress_state/ directory. For load-balanced deployments:

• Use session affinity (stick consumers to same instance) for optimal state consistency
• Or use shared network storage for progress_state/ directory
• The /hops endpoint should be called on all instances to keep target pools synchronized