406 lines
9.7 KiB
Markdown
406 lines
9.7 KiB
Markdown
# BTCUSDT Signal Generator
|
|
|
|
Production-ready signal generation service for the Bybit BTCUSDT paper trading pipeline.
|
|
|
|
## Features
|
|
|
|
- **Dual Personality Support**: Scalping (1m/5m) and Swing (15m/1h) strategies
|
|
- **Real-time Processing**: Sub-second polling of candles.db and analysis.db
|
|
- **Unix Socket Streaming**: Low-latency signal delivery (~130μs RTT)
|
|
- **Health Check Interface**: Separate socket for status monitoring
|
|
- **Comprehensive Logging**: File and stdout logging with detailed diagnostics
|
|
- **Signal Cooldown**: Prevents signal spam with configurable cooldown periods
|
|
- **Graceful Shutdown**: SIGINT/SIGTERM handling with proper cleanup
|
|
- **Error Recovery**: Robust error handling with automatic recovery
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌──────────────┐ ┌──────────────┐
|
|
│ candles.db │────▶│ │
|
|
└──────────────┘ │ Signal │ ┌─────────────────┐
|
|
│ Generator │────▶│ Unix Socket │
|
|
┌──────────────┐ │ │ │ /tmp/signals.sock│
|
|
│ analysis.db │────▶│ │ └─────────────────┘
|
|
└──────────────┘ └──────────────┘ │
|
|
│ ▼
|
|
│ ┌─────────────────┐
|
|
│ │ Trading Bot │
|
|
▼ └─────────────────┘
|
|
┌─────────────────┐
|
|
│ Health Socket │
|
|
│ /tmp/signals_ │
|
|
│ health.sock │
|
|
└─────────────────┘
|
|
```
|
|
|
|
## Installation
|
|
|
|
### 1. Install TA-Lib System Library
|
|
|
|
**Ubuntu/Debian:**
|
|
```bash
|
|
wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
|
|
tar -xzf ta-lib-0.4.0-src.tar.gz
|
|
cd ta-lib/
|
|
./configure --prefix=/usr
|
|
make
|
|
sudo make install
|
|
```
|
|
|
|
**macOS:**
|
|
```bash
|
|
brew install ta-lib
|
|
```
|
|
|
|
### 2. Install Python Dependencies
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Edit `config.json` to customize behavior:
|
|
|
|
### Key Parameters
|
|
|
|
| Parameter | Description | Default |
|
|
|-----------|-------------|---------|
|
|
| `personality` | Trading style: "scalping" or "swing" | "scalping" |
|
|
| `timeframes` | List of timeframes to monitor | ["1m", "5m"] |
|
|
| `poll_interval` | Seconds between checks | 0.5 |
|
|
| `min_confidence` | Minimum confidence threshold (0-1) | 0.6 |
|
|
| `cooldown_seconds` | Cooldown between signals | 60 |
|
|
| `lookback` | Candles to fetch for analysis | 200 |
|
|
|
|
### Personality Configurations
|
|
|
|
**Scalping** (Short-term, high-frequency):
|
|
- Timeframes: 1m, 5m
|
|
- Focus: EMA crossovers, Stochastic, volume surges
|
|
- Best for: Quick moves, scalp trades
|
|
- Signal frequency: Higher (10-50/day)
|
|
|
|
**Swing** (Medium-term, trend-following):
|
|
- Timeframes: 15m, 1h
|
|
- Focus: Regime filters, Bollinger squeezes, MACD
|
|
- Best for: Trend captures, larger moves
|
|
- Signal frequency: Lower (2-10/week)
|
|
|
|
### Signal Weights
|
|
|
|
Adjust weights in `config.json` to tune signal generation:
|
|
|
|
```json
|
|
"weights": {
|
|
"scalping": {
|
|
"ema_cross": 0.3, // EMA 9/21 crossover
|
|
"stoch": 0.25, // Stochastic oscillator
|
|
"rsi": 0.2, // RSI oversold/overbought
|
|
"volume": 0.15, // Volume surge
|
|
"macd": 0.1 // MACD confirmation
|
|
},
|
|
"swing": {
|
|
"regime": 0.35, // SMA 50/200 filter
|
|
"bb_squeeze": 0.25, // Bollinger squeeze
|
|
"macd": 0.2, // MACD crossover
|
|
"flow": 0.15, // Buy/sell pressure
|
|
"rsi": 0.05 // RSI filter
|
|
}
|
|
}
|
|
```
|
|
|
|
## Running
|
|
|
|
### Start the Generator
|
|
|
|
```bash
|
|
python signal_generator.py
|
|
```
|
|
|
|
Or with custom config:
|
|
```bash
|
|
python signal_generator.py -c custom_config.json
|
|
```
|
|
|
|
### Run as Background Service
|
|
|
|
```bash
|
|
nohup python signal_generator.py > /dev/null 2>&1 &
|
|
```
|
|
|
|
### Systemd Service (Recommended)
|
|
|
|
Create `/etc/systemd/system/signal-generator.service`:
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=BTCUSDT Signal Generator
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=youruser
|
|
WorkingDirectory=/path/to/bybitbtc/signals
|
|
ExecStart=/usr/bin/python3 signal_generator.py
|
|
Restart=always
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
Then:
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable signal-generator
|
|
sudo systemctl start signal-generator
|
|
sudo systemctl status signal-generator
|
|
```
|
|
|
|
## Health Checks
|
|
|
|
### Query Current Status
|
|
|
|
```bash
|
|
python health_check.py
|
|
```
|
|
|
|
Output example:
|
|
```
|
|
============================================================
|
|
SIGNAL GENERATOR HEALTH STATUS
|
|
============================================================
|
|
Status: running
|
|
Personality: scalping
|
|
Timeframes: 1m, 5m
|
|
Uptime: 3456s (57m)
|
|
Total Signals: 23
|
|
- Buy: 14
|
|
- Sell: 9
|
|
Last Signal: 2026-01-15T14:32:11.234Z
|
|
Errors: 0
|
|
Connected Clients: 1
|
|
|
|
Recent Signals:
|
|
[1m] BUY @ $43250.50 (conf: 0.75)
|
|
Reasons: EMA9 crossed above EMA21, Stochastic oversold crossover
|
|
[5m] SELL @ $43180.25 (conf: 0.68)
|
|
Reasons: EMA9 crossed below EMA21, RSI oversold (62.3)
|
|
============================================================
|
|
```
|
|
|
|
### Monitor Logs
|
|
|
|
```bash
|
|
tail -f logs/signal_generator.log
|
|
```
|
|
|
|
## Testing the Signal Stream
|
|
|
|
### Connect Test Client
|
|
|
|
```bash
|
|
python test_client.py
|
|
```
|
|
|
|
This will connect and display all incoming signals in real-time:
|
|
|
|
```
|
|
======================================================================
|
|
[2026-01-15 14:32:11] BUY SIGNAL
|
|
======================================================================
|
|
Timeframe: 1m
|
|
Price: $43250.50
|
|
Confidence: 75.0%
|
|
Personality: scalping
|
|
Reasons:
|
|
• EMA9 crossed above EMA21
|
|
• Stochastic oversold crossover
|
|
• Volume surge detected
|
|
======================================================================
|
|
```
|
|
|
|
### Integration with Trading Bot
|
|
|
|
Your trading bot should connect to `/tmp/signals.sock`:
|
|
|
|
```python
|
|
import socket
|
|
import json
|
|
|
|
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
sock.connect("/tmp/signals.sock")
|
|
|
|
buffer = ""
|
|
while True:
|
|
chunk = sock.recv(4096).decode('utf-8')
|
|
buffer += chunk
|
|
|
|
while '\n' in buffer:
|
|
line, buffer = buffer.split('\n', 1)
|
|
if line.strip():
|
|
signal = json.loads(line)
|
|
# Process signal
|
|
print(f"Received: {signal['signal']} @ ${signal['price']}")
|
|
```
|
|
|
|
## Signal Format
|
|
|
|
Each signal is a JSON object with the following structure:
|
|
|
|
```json
|
|
{
|
|
"signal": "BUY",
|
|
"timeframe": "1m",
|
|
"confidence": 0.75,
|
|
"price": 43250.50,
|
|
"timestamp": 1705329131,
|
|
"reasons": [
|
|
"EMA9 crossed above EMA21",
|
|
"Stochastic oversold crossover",
|
|
"Volume surge detected"
|
|
],
|
|
"personality": "scalping",
|
|
"generated_at": "2026-01-15T14:32:11.234567Z"
|
|
}
|
|
```
|
|
|
|
## Signal Logic
|
|
|
|
### Scalping Strategy
|
|
|
|
Generates signals on:
|
|
- **EMA Crossover**: 9/21 cross with direction confirmation
|
|
- **Stochastic**: Oversold (<30) or overbought (>70) with crossover
|
|
- **RSI**: Confirmation filter (<40 buy, >60 sell)
|
|
- **Volume**: 1.5x average indicates momentum
|
|
- **MACD**: Directional confirmation
|
|
|
|
### Swing Strategy
|
|
|
|
Generates signals on:
|
|
- **Regime Filter**: Price vs SMA 50/200 trend structure
|
|
- **BB Squeeze**: Volatility compression + breakout
|
|
- **MACD**: Crossover for momentum shift
|
|
- **Flow**: Buy/sell pressure ratio (55% threshold)
|
|
- **RSI**: Light filter to avoid extremes
|
|
|
|
## Performance Tuning
|
|
|
|
### Reduce Signal Frequency
|
|
|
|
Increase `min_confidence`:
|
|
```json
|
|
"min_confidence": 0.75
|
|
```
|
|
|
|
Increase `cooldown_seconds`:
|
|
```json
|
|
"cooldown_seconds": 300
|
|
```
|
|
|
|
### Increase Sensitivity
|
|
|
|
Lower confidence threshold:
|
|
```json
|
|
"min_confidence": 0.5
|
|
```
|
|
|
|
Reduce cooldown:
|
|
```json
|
|
"cooldown_seconds": 30
|
|
```
|
|
|
|
### Change Polling Rate
|
|
|
|
Faster updates (higher CPU):
|
|
```json
|
|
"poll_interval": 0.2
|
|
```
|
|
|
|
Slower updates (lower CPU):
|
|
```json
|
|
"poll_interval": 1.0
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "Socket not found" Error
|
|
|
|
Check if generator is running:
|
|
```bash
|
|
python health_check.py
|
|
```
|
|
|
|
Check for socket file:
|
|
```bash
|
|
ls -la /tmp/signals*.sock
|
|
```
|
|
|
|
### No Signals Generated
|
|
|
|
1. Check database connections:
|
|
```bash
|
|
sqlite3 ../onramp/candles.db "SELECT COUNT(*) FROM candles WHERE timeframe='1m';"
|
|
```
|
|
|
|
2. Lower confidence threshold temporarily:
|
|
```json
|
|
"min_confidence": 0.4
|
|
```
|
|
|
|
3. Check logs for errors:
|
|
```bash
|
|
grep ERROR logs/signal_generator.log
|
|
```
|
|
|
|
### Database Locked Errors
|
|
|
|
Ensure WAL mode is enabled (automatic in code):
|
|
```bash
|
|
sqlite3 ../onramp/candles.db "PRAGMA journal_mode=WAL;"
|
|
```
|
|
|
|
### High CPU Usage
|
|
|
|
Increase poll interval:
|
|
```json
|
|
"poll_interval": 1.0
|
|
```
|
|
|
|
Reduce lookback:
|
|
```json
|
|
"lookback": 100
|
|
```
|
|
|
|
## Maintenance
|
|
|
|
### Log Rotation
|
|
|
|
Use logrotate or manually clear:
|
|
```bash
|
|
echo "" > logs/signal_generator.log
|
|
```
|
|
|
|
### Database Optimization
|
|
|
|
The generator reads in WAL mode and doesn't modify data, so no maintenance needed on its side.
|
|
|
|
## Next Steps
|
|
|
|
1. **Backtest**: Use historical data to validate signal quality
|
|
2. **Tune Weights**: Adjust based on paper trading results
|
|
3. **Add Personalities**: Create custom strategies (e.g., "momentum", "mean-reversion")
|
|
4. **ML Integration**: Replace rule-based logic with trained models
|
|
5. **Multi-Asset**: Extend to other trading pairs
|
|
|
|
## Notes
|
|
|
|
- Signals are based on **closed candles only** - no repainting risk
|
|
- The latest forming candle is automatically excluded
|
|
- All timestamps are UTC
|
|
- Confidence scores are additive from multiple indicators
|
|
- Cooldown prevents signal spam per timeframe/personality combination
|