MrKalzu/ping_service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Claude Code session 1.

Browse Source
This commit is contained in:
Kalzu Rekku
2026-01-08 12:11:26 +02:00
parent c59523060d
commit 6db2e58dcd
20 changed files with 5497 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

146
input_service/README.md
View File

@@ -1,44 +1,112 @@
# HTTP Input Service
A lightweight HTTP server that serves individual IPv4 addresses from cloud provider CIDR ranges.
A lightweight HTTP server that serves individual IPv4 addresses from cloud provider CIDR ranges and accepts discovered hop IPs from traceroute results to organically grow the target pool.
## Purpose
Provides a continuous stream of IPv4 addresses to network scanning tools. Each consumer (identified by IP) receives addresses in randomized order from cloud provider IP ranges.
Provides a continuous stream of IPv4 addresses to network scanning tools. Each consumer (identified by IP) receives addresses in highly interleaved order from cloud provider IP ranges, avoiding consecutive IPs from the same subnet. Accepts discovered hop IPs from output_service to expand the target pool.
## Requirements
- Go 1.16+
- Go 1.25+
- Cloud provider IP repository cloned at `./cloud-provider-ip-addresses/`
## Usage
```bash
# Build
go build -ldflags="-s -w" -o ip-feeder main.go
## Building
# Run
./ip-feeder
```bash
go build -ldflags="-s -w" -o http_input_service http_input_service.go
```
## Usage
```bash
./http_input_service
```
Server starts on `http://localhost:8080`
## API
## API Endpoints
**GET /**
### `GET /`
Returns a single IPv4 address per request.
```bash
curl http://localhost:8080
# Output: 13.248.118.1
```
Each consumer (identified by source IP) gets their own independent sequence with interleaved IPs from different subnets.
### `POST /hops`
Accept discovered hop IPs from traceroute results.
**Request Body:**
```json
{
"hops": ["10.0.0.1", "172.16.5.3", "8.8.8.8"]
}
```
**Response:**
```json
{
"status": "ok",
"received": 3,
"added": 2,
"duplicates": 1
}
```
- Validates and filters out private, multicast, loopback IPs
- Global deduplication prevents re-adding seen IPs
- Automatically adds new hops to all consumer pools
### `GET /status`
View current service status and consumer information.
**Response:**
```json
{
"total_consumers": 2,
"consumers": [
{
"consumer": "192.168.1.100",
"remaining_cidrs": 1234,
"has_active_gen": true,
"total_cidrs": 5000
}
],
"state_directory": "progress_state",
"save_interval": "30s"
}
```
### `GET /export`
Export all consumer states for backup/migration.
Downloads a JSON file with all consumer progress states.
### `POST /import`
Import previously exported consumer states.
**Request:** Upload JSON from `/export` endpoint
## Features
- **Subnet Interleaving** - Maintains 10 active CIDR generators, rotating between them to avoid serving consecutive IPs from the same subnet
- **Per-consumer state** - Each client gets independent, deterministic sequence
- **Deduplication** - Both per-consumer and global deduplication to prevent serving duplicate IPs
- **Hop Discovery** - Accepts discovered traceroute hops via `/hops` endpoint to grow target pool organically
- **Memory efficient** - Loads CIDR files lazily (~5-15MB RAM usage)
- **Lazy expansion** - IPs generated on-demand from CIDR notation
- **Randomized order** - Interleaves IPs from multiple ranges randomly
- **IPv4 only** - Filters IPv6, multicast, network/broadcast addresses
- **Persistent state** - Progress saved every 30s, survives restarts
- **State export/import** - Backup and migrate consumer states between instances
- **IPv4 only** - Filters IPv6, multicast, network/broadcast, private addresses
- **Graceful shutdown** - Ctrl+C drains connections cleanly
## Expected Input Format
@@ -51,6 +119,54 @@ Scans `./cloud-provider-ip-addresses/` for `.txt` files containing IP ranges:
3.5.140.0/22
```
## Shutdown
## How Interleaving Works
Press `Ctrl+C` for graceful shutdown with 10s timeout.
To avoid consecutive IPs from the same subnet (e.g., `8.8.8.1`, `8.8.8.2`, `8.8.8.3`), the service:
1. Maintains **10 active CIDR generators** concurrently
2. **Rotates** between them in round-robin fashion
3. Each request pulls from the next generator in sequence
**Example output:**
```
9.9.9.1 # From CIDR 9.9.9.0/29
208.67.222.1 # From CIDR 208.67.222.0/29
1.1.1.1 # From CIDR 1.1.1.0/29
8.8.8.1 # From CIDR 8.8.8.0/29
8.8.4.1 # From CIDR 8.8.4.0/29
9.9.9.2 # Back to first CIDR
208.67.222.2 # Second CIDR
...
```
This ensures diverse network targeting and better coverage.
## Integration with Output Service
The `/hops` endpoint is designed to receive discovered hop IPs from `output_service`:
```bash
# Example from output_service
curl -X POST http://localhost:8080/hops \
-H "Content-Type: application/json" \
-d '{"hops": ["10.0.0.1", "172.16.5.3", "8.8.8.8"]}'
```
- Output service extracts intermediate hops from traceroute results
- POSTs them to input service `/hops` endpoint
- Input service validates, deduplicates, and adds to target pool
- Future consumers will receive these discovered IPs
This creates a feedback loop where the system organically discovers new targets through network exploration.
## Graceful Shutdown
Press `Ctrl+C` for graceful shutdown with 10s timeout. All consumer states are saved before exit.
## Multi-Instance Deployment
Each instance maintains its own consumer state files in `progress_state/` directory. For load-balanced deployments:
- Use **session affinity** (stick consumers to same instance) for optimal state consistency
- Or use **shared network storage** for `progress_state/` directory
- The `/hops` endpoint should be called on **all instances** to keep target pools synchronized