### Key Points on Signal Generator Design
- **Core Functionality**: The system processes real-time BTCUSDT data from SQLite databases to generate buy/sell signals based on technical indicators like EMAs, RSI, MACD, Bollinger Bands, and Stochastic Oscillator, supporting scalping (short-term, high-frequency) and swing (medium-term, trend-focused) personalities.
- **Low-Latency Focus**: Uses Unix Domain Sockets (UDS) for local streaming to minimize delays (typically 130-200μs RTT), ideal for your 0.5-1.7s pipeline lag and single-trade limit.
- **Implementation Choice**: Python 3 with pandas, TA-Lib, and sqlite3 for rapid development; incorporate backtesting via pandas or Backtesting.py to validate strategies before live use.
- **Risk Considerations**: Strategies like scalping may yield quick wins but face higher fees and noise; swing trading offers better risk-reward in trends but requires patience—backtest extensively to assess win rates (often 50-60%).

### System Overview
This design outlines a modular signal generator that ingests data from `candles.db` and `analysis.db`, computes signals using rule-based logic, and streams them to a paper trading bot. It emphasizes determinism, low overhead, and extensibility for your local setup.

### Architecture Components
- **Input**: Polls databases every 0.5-1s for closed candles.
- **Processing**: Applies personality-specific rules with confluence scoring.
- **Output**: JSON signals via UDS (fallback TCP for dual machines).

### Recommended Strategies
For scalping: Focus on 1m/5m timeframes with EMA crossovers and Stochastic. For swing: Use 15m/1h with regime filters and squeezes. Validate via backtesting to ensure profitability.

---
### Comprehensive Design and Plan for BTCUSDT Signal Generator

#### 1. Introduction and Objectives
The signal generator is a critical component in your Bybit BTCUSDT websocket data pipeline, transforming aggregated OHLCV and precomputed indicators into actionable buy/sell signals for a paper trading bot. This document provides a ready-to-use blueprint, incorporating best practices from Python-based trading systems, such as modular code for indicators, efficient data handling with pandas, and low-latency communication. Objectives include:
- Supporting "scalping" (high-frequency, short-term trades) and "swing" (medium-term, trend-capturing) personalities.
- Ensuring sub-second latency for signal delivery, aligning with your 0.5-1.7s pipeline.
- Enabling backtesting to evaluate strategy performance, targeting win rates of 50-60% in crypto markets.
- Facilitating easy extension for new indicators or personalities.

The system runs as a single-threaded daemon on one or two local machines, handling ~600 messages/minute without high concurrency needs.

#### 2. System Architecture
The architecture follows a layered, event-driven design inspired by high-performance trading systems: input, processing, and output. It uses Python 3 for prototyping, with pandas for data manipulation and TA-Lib for indicators.

- **Input Layer**: Connects to `candles.db` and `analysis.db` using WAL mode for concurrent reads. Polls every 0.5-1s, fetching the last 200-500 rows per timeframe to compute trends.
- **Processing Layer**: Loads data into pandas DataFrames, computes additional indicators (e.g., Stochastic), applies rules, and scores signals (0-1 confidence based on confluence).
- **Output Layer**: Streams JSON signals (e.g., {"signal": "BUY", "timeframe": "1m", "confidence": 0.75}) via UDS for lowest latency (~130μs RTT); fallback to TCP with NODELAY for dual machines.
- **Configuration**: YAML file for personalities, thresholds (e.g., RSI<30), and polling intervals.
- **Monitoring**: Logs to file; optional metrics for signal frequency.

This setup ensures determinism—no repainting—and handles your throughput reduction goal by filtering to 1-10 signals/hour.

#### 3. Data Integration and Handling
Leverage your existing schemas: Join `candles` and `analysis` tables on timeframe/timestamp for unified DataFrames. Use TA-Lib for Stochastic (%K=14, %D=3) and extend precomputed indicators (EMA9/21, RSI14, MACD, Bollinger, volume MA20).

Example Fetch Function:
```python
import sqlite3
import pandas as pd
import talib

def fetch_and_enrich(timeframe, limit=200, candles_db='candles.db', analysis_db='analysis.db'):
    conn_c = sqlite3.connect(candles_db, timeout=10)
    conn_a = sqlite3.connect(analysis_db, timeout=10)
    query = """
        SELECT c.*, a.*
        FROM candles c
        JOIN analysis a ON c.timeframe = a.timeframe AND c.timestamp = a.timestamp
        WHERE c.timeframe = ?
        ORDER BY c.timestamp DESC
        LIMIT ?
    """
    df = pd.read_sql_query(query, conn_c, params=(timeframe, limit))
    conn_c.close(); conn_a.close()
    df['datetime'] = pd.to_datetime(df['timestamp'], unit='s')
    df = df.sort_values('timestamp').reset_index(drop=True)
    # Add Stochastic
    df['stoch_k'], df['stoch_d'] = talib.STOCH(df['high'], df['low'], df['close'], fastk_period=14, slowk_period=3, slowd_period=3)
    return df
```

Filter for closed candles (timestamp < current time) to avoid instability.

#### 4. Indicators and Signal Logic
Build on your `analysis.db` with rule-based logic for confluence. Key indicators:
- MA Crossings: EMA9 > EMA21 (buy); reverse for sell.
- RSI: <30 (oversold, buy); >70 (overbought, sell).
- MACD: Line > signal (buy).
- Bollinger Squeeze: bb_squeeze=1 + breakout.
- Stochastic: %K > %D and <20 (buy).
- Volume: >1.5 * MA20; buy_ratio >0.6.

Confidence: Weighted sum (e.g., 0.3 for MA, 0.2 for volume) >0.6 triggers signal.

Example Logic:
```python
def generate_signal(df, personality='scalping'):
    latest = df.iloc[-1]
    score = 0
    reasons = []
    if personality == 'scalping':
        if latest['ema_9'] > latest['ema_21'] and latest['stoch_k'] > latest['stoch_d'] and latest['stoch_k'] < 30:
            score += 0.4; reasons.append('EMA crossover + Stochastic buy')
        if latest['rsi_14'] < 50 and latest['volume'] > 1.5 * latest['volume_ma_20']:
            score += 0.3; reasons.append('RSI undersold + volume surge')
    elif personality == 'swing':
        if latest['close'] > latest['sma_50'] and latest['bb_squeeze'] == 1 and latest['close'] > latest['bb_upper']:
            score += 0.4; reasons.append('Regime bull + BB breakout')
        if latest['macd'] > latest['macd_signal'] and latest['buy_volume'] / latest['volume'] > 0.55:
            score += 0.3; reasons.append('MACD crossover + buy bias')
    if score > 0.6:
        return {'signal': 'BUY' if score > 0 else 'SELL', 'confidence': score, 'reasons': reasons}
    return None
```

#### 5. Trading Personalities
Tailor strategies to crypto volatility.

| Personality | Timeframes | Focus Indicators | Trade Frequency | Risk Notes |
|-------------|------------|------------------|-----------------|------------|
| Scalping   | 1m, 5m    | EMA9/21, Stochastic, MACD hist, volume | 10-50/day | High fees/slippage; aim 0.25-1% per trade, 65% win rate possible with BB/RSI. |
| Swing      | 15m, 1h   | SMA50/200, BB squeeze, RSI, net flow | 2-10/week | Lower stress; target 5-10% moves, suits trends but lags in ranges. |

Switch via config; add filters like regime bias (close > SMA200 for longs).

#### 6. Streaming and Bot Integration
Use UDS for polling-based feed: Generator writes signals; bot polls every 50-100ms. Defer stops/positioning to bot.

Server Code:
```python
import socket
import json
import time

SOCKET_PATH = '/tmp/signal.sock'
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try: os.remove(SOCKET_PATH)
except: pass
server.bind(SOCKET_PATH)
server.listen(1)
conn, _ = server.accept()
while True:
    signal = generate_signal(fetch_and_enrich('1m'))  # Example
    if signal:
        conn.sendall(json.dumps(signal).encode() + b'\n')
    time.sleep(0.5)  # Poll interval
```

Bot polls non-blockingly.

#### 7. Implementation Plan
- **Dependencies**: pandas, sqlite3, talib, backtesting.py.
- **Phases**: 1) Setup DB connections (1 day); 2) Indicator/signal logic (2-3 days); 3) Streaming (1 day); 4) Backtesting (2 days); 5) Testing/deployment (1-2 days).
- **Code Structure**: Main loop in `signal_generator.py`; separate modules for data, logic, config.

#### 8. Backtesting and Validation
Use pandas for simple backtests or Backtesting.py for advanced. Simulate slippage (0.1%) and fees; track Sharpe ratio, drawdown.

Example with Backtesting.py:
```python
from backtesting import Backtest, Strategy

class ScalpStrategy(Strategy):
    def init(self):
        self.ema9 = self.I(talib.EMA, self.data.Close, 9)
        self.stoch_k, self.stoch_d = self.I(talib.STOCH, self.data.High, self.data.Low, self.data.Close)
    def next(self):
        if crossover(self.ema9, self.ema21) and self.stoch_k > self.stoch_d and self.stoch_k < 30:
            self.buy()

bt = Backtest(df, ScalpStrategy, cash=10000, commission=.001)
stats = bt.run()
print(stats)
```

Aim for positive expectancy; re-optimize thresholds quarterly.

#### 9. Deployment and Maintenance
Run as systemd service; monitor with logs. Vacuum DBs monthly. Scale by adding personalities or ML later.

#### 10. Risks and Mitigations
- **Market Risks**: Crypto volatility; mitigate with volume filters and backtesting.
- **Technical Risks**: DB locks; use timeouts. Overfitting; use out-of-sample data.
- **Performance**: CPU spikes from polling; adaptive intervals.
- **Disclaimer**: Signals for paper trading only; not financial advice.

This plan equips you to build a robust system, validated by industry practices.

### Key Citations
- [Creating a Custom Python Library for Trading Indicators and Signal Generation - Medium](https://medium.com/@deepml1818/creating-a-custom-python-library-for-trading-indicators-and-signal-generation-2097d0d526f7)
- [How to build a macro trading strategy (with open-source Python) - Macrosynergy](https://macrosynergy.com/research/how-to-build-a-macro-trading-strategy-with-open-source-python)
- [Python in FinTech: Building High-Performance Trading Systems - CMARIX](https://www.cmarix.com/blog/build-trading-systems-with-python-in-fintech)
- [Python in High-Frequency Trading: Low-Latency Techniques - PyQuant News](https://www.pyquantnews.com/free-python-resources/python-in-high-frequency-trading-low-latency-techniques)
- [A Look at the Differences Between Scalping and Swing Trading in Crypto | by JeNovation](https://medium.com/@ByTrade.io/a-look-at-the-differences-between-scalping-and-swing-trading-in-crypto-d7a83623c277)
- [Day trade - swing or scalp? : r/BitcoinMarkets - Reddit](https://www.reddit.com/r/BitcoinMarkets/comments/1jklil5/day_trade_swing_or_scalp)
- [Best Crypto Trading Strategies (Beginner to Intermediate Guide) - Changelly](https://changelly.com/blog/cryptocurrency-trading-strategies)
- [Crypto Scalping vs Swing Trading: Which is More Profitable for Retail Investors?](https://www.blockchain-council.org/cryptocurrency/crypto-scalping-vs-swing-trading)
- [Crypto Scalping: What It Is, How It Works, Strategies & Tools (2025) - CryptoNinjas](https://www.cryptoninjas.net/crypto/cryptocurrency-scalping)
- [7 Best Crypto Trading Strategies for 2025 - CMC Markets](https://www.cmcmarkets.com/en/cryptocurrencies/7-crypto-trading-strategies)
- [I Tried 1-Minute Scalping on BTC — My Honest Results - InsiderFinance Wire](https://wire.insiderfinance.io/i-tried-1-minute-scalping-on-btc-my-honest-results-c80989f7335b)
- [Algorithmic Trading with Stochastic Oscillator in Python | by Nikhil Adithyan - Medium](https://medium.com/codex/algorithmic-trading-with-stochastic-oscillator-in-python-7e2bec49b60d)
- [Python Trading Strategy: Synergizing Stochastic Oscillator and MACD Indicator - EODHD](https://eodhd.com/financial-academy/backtesting-strategies-examples/using-python-to-create-an-innovative-trading-strategy-and-achieve-better-results)
- [Using the Stochastic Oscillator in Python for Algorithmic Trading - αlphαrithms](https://www.alpharithms.com/stochastic-oscillator-in-python-483214)
- [Momentum Indicators - Technical Analysis Library in Python's documentation!](https://technical-analysis-library-in-python.readthedocs.io/en/latest/ta.html)
- [Backtesting with Pandas and TA-lib - YouTube](https://www.youtube.com/watch?v=WBPnN8DIMYI)
- [Backtesting.py - Backtest trading strategies in Python](https://kernc.github.io/backtesting.py)
- [Mastering Python Backtesting for Trading Strategies | by Time Money Code | Medium](https://medium.com/@timemoneycode/mastering-python-backtesting-for-trading-strategies-1f7df773fdf5)
- [Python Libraries for Quantitative Trading | QuantStart](https://www.quantstart.com/articles/python-libraries-for-quantitative-trading)