# BTCUSDT Signal Generator

Production-ready signal generation service for the Bybit BTCUSDT paper trading pipeline.

## Features

- **Dual Personality Support**: Scalping (1m/5m) and Swing (15m/1h) strategies
- **Real-time Processing**: Sub-second polling of candles.db and analysis.db
- **Unix Socket Streaming**: Low-latency signal delivery (~130μs RTT)
- **Health Check Interface**: Separate socket for status monitoring
- **Comprehensive Logging**: File and stdout logging with detailed diagnostics
- **Signal Cooldown**: Prevents signal spam with configurable cooldown periods
- **Graceful Shutdown**: SIGINT/SIGTERM handling with proper cleanup
- **Error Recovery**: Robust error handling with automatic recovery

## Architecture

```
┌──────────────┐     ┌──────────────┐
│ candles.db   │────▶│              │
└──────────────┘     │   Signal     │     ┌─────────────────┐
                     │  Generator   │────▶│ Unix Socket     │
┌──────────────┐     │              │     │ /tmp/signals.sock│
│ analysis.db  │────▶│              │     └─────────────────┘
└──────────────┘     └──────────────┘              │
                            │                      ▼
                            │              ┌─────────────────┐
                            │              │ Trading Bot     │
                            ▼              └─────────────────┘
                     ┌─────────────────┐
                     │ Health Socket   │
                     │ /tmp/signals_   │
                     │ health.sock     │
                     └─────────────────┘
```

## Installation

### 1. Install TA-Lib System Library

**Ubuntu/Debian:**
```bash
wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
tar -xzf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
./configure --prefix=/usr
make
sudo make install
```

**macOS:**
```bash
brew install ta-lib
```

### 2. Install Python Dependencies

```bash
pip install -r requirements.txt
```

## Configuration

Edit `config.json` to customize behavior:

### Key Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `personality` | Trading style: "scalping" or "swing" | "scalping" |
| `timeframes` | List of timeframes to monitor | ["1m", "5m"] |
| `poll_interval` | Seconds between checks | 0.5 |
| `min_confidence` | Minimum confidence threshold (0-1) | 0.6 |
| `cooldown_seconds` | Cooldown between signals | 60 |
| `lookback` | Candles to fetch for analysis | 200 |

### Personality Configurations

**Scalping** (Short-term, high-frequency):
- Timeframes: 1m, 5m
- Focus: EMA crossovers, Stochastic, volume surges
- Best for: Quick moves, scalp trades
- Signal frequency: Higher (10-50/day)

**Swing** (Medium-term, trend-following):
- Timeframes: 15m, 1h
- Focus: Regime filters, Bollinger squeezes, MACD
- Best for: Trend captures, larger moves
- Signal frequency: Lower (2-10/week)

### Signal Weights

Adjust weights in `config.json` to tune signal generation:

```json
"weights": {
  "scalping": {
    "ema_cross": 0.3,    // EMA 9/21 crossover
    "stoch": 0.25,       // Stochastic oscillator
    "rsi": 0.2,          // RSI oversold/overbought
    "volume": 0.15,      // Volume surge
    "macd": 0.1          // MACD confirmation
  },
  "swing": {
    "regime": 0.35,      // SMA 50/200 filter
    "bb_squeeze": 0.25,  // Bollinger squeeze
    "macd": 0.2,         // MACD crossover
    "flow": 0.15,        // Buy/sell pressure
    "rsi": 0.05          // RSI filter
  }
}
```

## Running

### Start the Generator

```bash
python signal_generator.py
```

Or with custom config:
```bash
python signal_generator.py -c custom_config.json
```

### Run as Background Service

```bash
nohup python signal_generator.py > /dev/null 2>&1 &
```

### Systemd Service (Recommended)

Create `/etc/systemd/system/signal-generator.service`:

```ini
[Unit]
Description=BTCUSDT Signal Generator
After=network.target

[Service]
Type=simple
User=youruser
WorkingDirectory=/path/to/bybitbtc/signals
ExecStart=/usr/bin/python3 signal_generator.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

Then:
```bash
sudo systemctl daemon-reload
sudo systemctl enable signal-generator
sudo systemctl start signal-generator
sudo systemctl status signal-generator
```

## Health Checks

### Query Current Status

```bash
python health_check.py
```

Output example:
```
============================================================
SIGNAL GENERATOR HEALTH STATUS
============================================================
Status:           running
Personality:      scalping
Timeframes:       1m, 5m
Uptime:           3456s (57m)
Total Signals:    23
  - Buy:          14
  - Sell:         9
Last Signal:      2026-01-15T14:32:11.234Z
Errors:           0
Connected Clients: 1

Recent Signals:
  [1m] BUY @ $43250.50 (conf: 0.75)
      Reasons: EMA9 crossed above EMA21, Stochastic oversold crossover
  [5m] SELL @ $43180.25 (conf: 0.68)
      Reasons: EMA9 crossed below EMA21, RSI oversold (62.3)
============================================================
```

### Monitor Logs

```bash
tail -f logs/signal_generator.log
```

## Testing the Signal Stream

### Connect Test Client

```bash
python test_client.py
```

This will connect and display all incoming signals in real-time:

```
======================================================================
[2026-01-15 14:32:11] BUY SIGNAL
======================================================================
Timeframe:   1m
Price:       $43250.50
Confidence:  75.0%
Personality: scalping
Reasons:
  • EMA9 crossed above EMA21
  • Stochastic oversold crossover
  • Volume surge detected
======================================================================
```

### Integration with Trading Bot

Your trading bot should connect to `/tmp/signals.sock`:

```python
import socket
import json

sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect("/tmp/signals.sock")

buffer = ""
while True:
    chunk = sock.recv(4096).decode('utf-8')
    buffer += chunk
    
    while '\n' in buffer:
        line, buffer = buffer.split('\n', 1)
        if line.strip():
            signal = json.loads(line)
            # Process signal
            print(f"Received: {signal['signal']} @ ${signal['price']}")
```

## Signal Format

Each signal is a JSON object with the following structure:

```json
{
  "signal": "BUY",
  "timeframe": "1m",
  "confidence": 0.75,
  "price": 43250.50,
  "timestamp": 1705329131,
  "reasons": [
    "EMA9 crossed above EMA21",
    "Stochastic oversold crossover",
    "Volume surge detected"
  ],
  "personality": "scalping",
  "generated_at": "2026-01-15T14:32:11.234567Z"
}
```

## Signal Logic

### Scalping Strategy

Generates signals on:
- **EMA Crossover**: 9/21 cross with direction confirmation
- **Stochastic**: Oversold (<30) or overbought (>70) with crossover
- **RSI**: Confirmation filter (<40 buy, >60 sell)
- **Volume**: 1.5x average indicates momentum
- **MACD**: Directional confirmation

### Swing Strategy

Generates signals on:
- **Regime Filter**: Price vs SMA 50/200 trend structure
- **BB Squeeze**: Volatility compression + breakout
- **MACD**: Crossover for momentum shift
- **Flow**: Buy/sell pressure ratio (55% threshold)
- **RSI**: Light filter to avoid extremes

## Performance Tuning

### Reduce Signal Frequency

Increase `min_confidence`:
```json
"min_confidence": 0.75
```

Increase `cooldown_seconds`:
```json
"cooldown_seconds": 300
```

### Increase Sensitivity

Lower confidence threshold:
```json
"min_confidence": 0.5
```

Reduce cooldown:
```json
"cooldown_seconds": 30
```

### Change Polling Rate

Faster updates (higher CPU):
```json
"poll_interval": 0.2
```

Slower updates (lower CPU):
```json
"poll_interval": 1.0
```

## Troubleshooting

### "Socket not found" Error

Check if generator is running:
```bash
python health_check.py
```

Check for socket file:
```bash
ls -la /tmp/signals*.sock
```

### No Signals Generated

1. Check database connections:
   ```bash
   sqlite3 ../onramp/candles.db "SELECT COUNT(*) FROM candles WHERE timeframe='1m';"
   ```

2. Lower confidence threshold temporarily:
   ```json
   "min_confidence": 0.4
   ```

3. Check logs for errors:
   ```bash
   grep ERROR logs/signal_generator.log
   ```

### Database Locked Errors

Ensure WAL mode is enabled (automatic in code):
```bash
sqlite3 ../onramp/candles.db "PRAGMA journal_mode=WAL;"
```

### High CPU Usage

Increase poll interval:
```json
"poll_interval": 1.0
```

Reduce lookback:
```json
"lookback": 100
```

## Maintenance

### Log Rotation

Use logrotate or manually clear:
```bash
echo "" > logs/signal_generator.log
```

### Database Optimization

The generator reads in WAL mode and doesn't modify data, so no maintenance needed on its side.

## Next Steps

1. **Backtest**: Use historical data to validate signal quality
2. **Tune Weights**: Adjust based on paper trading results
3. **Add Personalities**: Create custom strategies (e.g., "momentum", "mean-reversion")
4. **ML Integration**: Replace rule-based logic with trained models
5. **Multi-Asset**: Extend to other trading pairs

## Notes

- Signals are based on **closed candles only** - no repainting risk
- The latest forming candle is automatically excluded
- All timestamps are UTC
- Confidence scores are additive from multiple indicators
- Cooldown prevents signal spam per timeframe/personality combination