# Bybit BTCUSDT Signal Engine

A high-performance, multi-stage pipeline for streaming, processing, and generating trading signals from Bybit's BTCUSDT public trade WebSocket.

This project is architected as a series of decoupled microservices that move data from raw WebSocket packets to actionable trading signals via SQLite persistence and Unix Domain Sockets.

## 🏗 Architecture & Data Flow

The engine operates as a linear pipeline, ensuring data integrity and separation of concerns:

1.  **`input` (Go)**: Connects to Bybit V5 WS. Streams raw JSON payloads into hourly-rotated `.jsonl` files.
2.  **`onramp` (Go)**: Tails the JSONL files. Aggregates individual trades into OHLCV candles across multiple timeframes (1m, 5m, 15m, 1h) and saves them to `candles.db`.
3.  **`analysis` (Python)**: Reads `candles.db`. Computes technical indicators (EMA, RSI, MACD, Bollinger Bands) and saves them to `analysis.db`.
4.  **`signals` (Python)**: Monitors both databases. Applies "Personalities" (Scalping or Swing logic) to generate signals, broadcasted via Unix Domain Sockets.
5.  **`monitor` (Python)**: Provides a real-time dashboard of the system health and signal flow.

```mermaid
graph LR
    Bybit((Bybit WS)) --> input[input service]
    input --> JSONL[(JSONL Files)]
    JSONL --> onramp[onramp service]
    onramp --> CDB[(candles.db)]
    CDB --> analysis[analysis service]
    analysis --> ADB[(analysis.db)]
    ADB --> signals[signals service]
    CDB --> signals
    signals --> Socket((/tmp/signals.sock))
    Socket --> Bot[Trading Bot / Monitor]
```

---

## 🛠 The "Mother" Script (`mother.sh`)

The entire stack is managed by `mother.sh`, a centralized orchestration script located in the root. It handles dependency management, system health checks, and process multiplexing using **tmux**.

### Key Features
*   **Dependency Management**: One-command setup for both Go modules and Python `pipenv` environments.
*   **Tick Verification**: Before starting, the script polls `pool.ntp.org` to ensure system clock drift is `< 0.5s`, preventing timestamp rejection from Bybit.
*   **Sequential Bootstrapping**: Starts services in the correct order (`input` -> `onramp` -> `analysis` -> `signals`) with specific delays to allow database initialization.
*   **Process Isolation**: Each service runs in its own named window within a `tmux` session.

### Usage
| Command | Description |
| :--- | :--- |
| `./mother.sh setup` | Installs all Go and Python dependencies in their respective subdirectories. |
| `./mother.sh start` | Verifies NTP time and launches the full pipeline in a tmux session. |
| `./mother.sh status` | Checks if the services are currently running. |
| `./mother.sh stop` | Gracefully kills the tmux session and all underlying processes. |
| `./mother.sh restart` | Full stop and sequential restart. |

---

## 🚀 Getting Started

### Prerequisites
*   **Go** 1.20+
*   **Python** 3.10+ & **pipenv**
*   **tmux** (for process management)
*   **TA-Lib** (System library required for Python signal analysis)
    ```bash
    # Ubuntu/Debian
    sudo apt-get install ntpdate bc
    # See signals/README.md for TA-Lib compilation instructions
    ```

### Installation & Launch
1.  **Clone the repo**:
    ```bash
    git clone <repo-url>
    cd bybitbtc
    ```
2.  **Run Setup**:
    ```bash
    ./mother.sh setup
    ```
3.  **Start the Engine**:
    ```bash
    ./mother.sh start
    ```
4.  **Attach to Logs**:
    ```bash
    tmux attach -t bybitbtc
    ```

---

## 📊 Service Details

### 1. Ingestion (`input/`)
*   **Language**: Go
*   **Output**: `./input/output/publicTrade.BTCUSDT_TIMESTAMP.jsonl`
*   **Rotation**: Hourly.
*   **Compression**: Automatically gzips files older than 2 hours to save disk space.

### 2. Aggregation (`onramp/`)
*   **Language**: Go
*   **Database**: `candles.db` (SQLite with WAL mode).
*   **Features**: Real-time OHLCV calculation, Buy/Sell volume tracking, and a Janitor loop that prunes data older than 30 days.

### 3. Analysis (`analysis/`)
*   **Language**: Python (pandas, talib)
*   **Database**: `analysis.db`.
*   **Indicators**: EMA (9, 21), SMA (50, 200), RSI (14), MACD (12, 26, 9), Bollinger Bands with Squeeze detection.

### 4. Signals (`signals/`)
*   **Language**: Python
*   **Personalities**:
    *   **Scalping**: High-frequency (1m/5m), EMA-cross and Stochastic focused.
    *   **Swing**: Medium-frequency (15m/1h), Trend regime and BB-Squeeze focused.
*   **Delivery**: Streams JSON signals over `/tmp/signals.sock`.

---

## 📡 Monitoring & Health

Each service exposes a status interface:
*   **Input**: `go run input.go -status`
*   **Onramp**: `echo "status" | nc 127.0.0.1 9999`
*   **Signals**: `python signals/signal_health_client.py` (via `/tmp/signals_health.sock`)

---

## ⚖️ License
 MIT