SlidingSQLite Library Documentation

Overview

SlidingSQLite is a thread-safe SQLite implementation with automatic time-based database rotation. It allows concurrent read and write operations, supports database rotation based on time intervals, and ensures old databases are automatically cleaned up.

Features

Automatic database file rotation at configurable intervals.
Retention-based cleanup of old database files.
Thread-safe, queue-based execution for write operations.
Transparent access to all historical databases for read queries.
Synchronous and asynchronous query execution.

Installation

Ensure you have Python 3.10+ installed, as well as the necessary standard library modules.

pip install sqlite3

Initialization

Create a SlidingSQLite instance by specifying a directory to store databases and providing a schema for table initialization.

from SlidingSqlite import SlidingSQLite

db = SlidingSQLite(
    db_dir="./databases",
    schema="""
        CREATE TABLE IF NOT EXISTS data (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            timestamp REAL NOT NULL,
            value TEXT NOT NULL
        );
    """,
    retention_period=604800,  # 7 days
    rotation_interval=3600    # 1 hour
)

Writing Data

Use execute_write_sync for synchronous writes:

result = db.execute_write_sync("INSERT INTO data (timestamp, value) VALUES (?, ?)", (time.time(), "Hello"))
if result.success:
    print("Write successful")
else:
    print("Write failed:", result.error)

For asynchronous writes, use execute_write:

query_id = db.execute_write("INSERT INTO data (timestamp, value) VALUES (?, ?)", (time.time(), "Async Entry"))

Reading Data

Perform synchronous reads:

result = db.execute_read_sync("SELECT * FROM data")
if result.success:
    print("Data:", result.data)
else:
    print("Read failed:", result.error)

For asynchronous reads, use execute_read:

query_id = db.execute_read("SELECT * FROM data")
response = db.get_read_result(query_id)
if response.success:
    print("Results:", response.data)

Managing Databases

List all databases:

print(db.get_databases_info())

Delete old databases:

deleted_count = db.delete_databases_before(time.time() - 7 * 86400)
print(f"Deleted {deleted_count} old databases")

Shutdown

To gracefully close connections:

db.shutdown()

Notes

Ensure the schema is consistent across all rotated databases.
All queries execute in separate threads, making it suitable for high-concurrency environments.
The metadata database (metadata.db) tracks all database files and their time ranges.

2.6 KiB Raw Blame History