This project is architected as a series of decoupled microservices that move data from raw WebSocket packets to actionable trading signals via SQLite persistence and Unix Domain Sockets.

🏗 Architecture & Data Flow

The engine operates as a linear pipeline, ensuring data integrity and separation of concerns:

input (Go): Connects to Bybit V5 WS. Streams raw JSON payloads into hourly-rotated .jsonl files.
onramp (Go): Tails the JSONL files. Aggregates individual trades into OHLCV candles across multiple timeframes (1m, 5m, 15m, 1h) and saves them to candles.db.
analysis (Python): Reads candles.db. Computes technical indicators (EMA, RSI, MACD, Bollinger Bands) and saves them to analysis.db.
signals (Python): Monitors both databases. Applies "Personalities" (Scalping or Swing logic) to generate signals, broadcasted via Unix Domain Sockets.
monitor (Python): Provides a real-time dashboard of the system health and signal flow.

graph LR
    Bybit((Bybit WS)) --> input[input service]
    input --> JSONL[(JSONL Files)]
    JSONL --> onramp[onramp service]
    onramp --> CDB[(candles.db)]
    CDB --> analysis[analysis service]
    analysis --> ADB[(analysis.db)]
    ADB --> signals[signals service]
    CDB --> signals
    signals --> Socket((/tmp/signals.sock))
    Socket --> Bot[Trading Bot / Monitor]

🛠 The "Mother" Script (`mother.sh`)

The entire stack is managed by mother.sh, a centralized orchestration script located in the root. It handles dependency management, system health checks, and process multiplexing using tmux.

Key Features

Dependency Management: One-command setup for both Go modules and Python pipenv environments.
Tick Verification: Before starting, the script polls pool.ntp.org to ensure system clock drift is < 0.5s, preventing timestamp rejection from Bybit.
Sequential Bootstrapping: Starts services in the correct order (input -> onramp -> analysis -> signals) with specific delays to allow database initialization.
Process Isolation: Each service runs in its own named window within a tmux session.

Usage

Command	Description
`./mother.sh setup`	Installs all Go and Python dependencies in their respective subdirectories.
`./mother.sh start`	Verifies NTP time and launches the full pipeline in a tmux session.
`./mother.sh status`	Checks if the services are currently running.
`./mother.sh stop`	Gracefully kills the tmux session and all underlying processes.
`./mother.sh restart`	Full stop and sequential restart.

🚀 Getting Started

Prerequisites

Go 1.20+
Python 3.10+ & pipenv
tmux (for process management)

TA-Lib (System library required for Python signal analysis)

# Ubuntu/Debian
sudo apt-get install ntpdate bc
# See signals/README.md for TA-Lib compilation instructions

Installation & Launch

Clone the repo:
```
git clone <repo-url>
cd bybitbtc
```
Run Setup:
```
./mother.sh setup
```
Start the Engine:
```
./mother.sh start
```
Attach to Logs:
```
tmux attach -t bybitbtc
```

📊 Service Details

1. Ingestion (`input/`)

Language: Go
Output: ./input/output/publicTrade.BTCUSDT_TIMESTAMP.jsonl
Rotation: Hourly.
Compression: Automatically gzips files older than 2 hours to save disk space.

2. Aggregation (`onramp/`)

Language: Go
Database: candles.db (SQLite with WAL mode).
Features: Real-time OHLCV calculation, Buy/Sell volume tracking, and a Janitor loop that prunes data older than 30 days.

3. Analysis (`analysis/`)

Language: Python (pandas, talib)
Database: analysis.db.
Indicators: EMA (9, 21), SMA (50, 200), RSI (14), MACD (12, 26, 9), Bollinger Bands with Squeeze detection.

4. Signals (`signals/`)

Language: Python
Personalities:
- Scalping: High-frequency (1m/5m), EMA-cross and Stochastic focused.
- Swing: Medium-frequency (15m/1h), Trend regime and BB-Squeeze focused.
Delivery: Streams JSON signals over /tmp/signals.sock.

📡 Monitoring & Health

Each service exposes a status interface:

Input: go run input.go -status
Onramp: echo "status" | nc 127.0.0.1 9999
Signals: python signals/signal_health_client.py (via /tmp/signals_health.sock)

⚖️ License

MIT

README.md

Bybit BTCUSDT Signal Engine