Pando, Pando the Aspen colony.
This commit is contained in:
1 parent
a6d014601d
commit
ac73968de7
11 files changed
+152
-89
No files matched your search
@@ -1,4 +1,5 @@
|
||||
# Dynamic Schema Records API
|
||||
# Pando! 🌳
|
||||
* Dynamic Schema Records API *
|
||||
|
||||
A lightweight, high-performance JSON records service built with **Rust**, **Axum**, and **SQLite**.
|
||||
|
||||
@@ -17,7 +18,7 @@ Instances can run as a **primary** (accepts writes) or as a **read-only replica*
|
||||
- **`strict`**: All defined fields are required, and unknown fields are rejected.
|
||||
- **`loose`**: All defined fields are required, but additional unknown fields are accepted.
|
||||
- **Hot-Reloadable Configuration**: Reload `config.json` via an HTTP admin endpoint with atomic swap and rollback on error.
|
||||
- **Persistent Storage**: Validated records are stored with a generated UUID, an ISO-8601 UTC timestamp and a global sequence number (`seq`) in an SQLite database (`messages.db`).
|
||||
- **Persistent Storage**: Validated records are stored with a generated UUID, an ISO-8601 UTC timestamp and a global sequence number (`seq`) in an SQLite database (`pando.db`).
|
||||
- **Replication & Dynamic Role Switching**:
|
||||
- Pull-based and cursor-driven: the replica asks for everything after its last `seq`.
|
||||
- Append-only journal with a global tick (`seq`) that is preserved on replicas.
|
||||
@@ -86,7 +87,7 @@ The schema is only enforced on writes to a primary. A replica still loads the fi
|
||||
|
||||
| Variable | Default | Description |
|
||||
| :--- | :--- | :--- |
|
||||
| `APP_CONFIG` | `config.json` | Path to the schema file. |
|
||||
| `APP_CONFIG` | `pando.json` | Path to the schema file. |
|
||||
| `APP_BIND` | `127.0.0.1:3000` | Listen address. Change this for a primary that replicas must reach over a network. |
|
||||
| `APP_ROLE` | `primary` | `primary` or `replica`. |
|
||||
| `APP_PRIMARY_URL` | *none* | Base URL of the replication source, e.g. `https://primary.example:3000`. **Required for a replica.** |
|
||||
@@ -104,10 +105,10 @@ The schema is only enforced on writes to a primary. A replica still loads the fi
|
||||
|
||||
```bash
|
||||
# Clone the repository and enter the directory
|
||||
cd your-project-name
|
||||
cd pando
|
||||
|
||||
# Create your config file
|
||||
cat <<EOF > config.json
|
||||
cat <<EOF > pando.json
|
||||
{
|
||||
"mode": "strict",
|
||||
"fields": {
|
||||
@@ -134,23 +135,23 @@ APP_CONFIG=custom_config.json cargo run
|
||||
|
||||
### 2. Run a primary and a replica
|
||||
|
||||
`messages.db` is opened relative to the working directory, so run each instance from its own directory.
|
||||
`pando.db` is opened relative to the working directory, so run each instance from its own directory.
|
||||
|
||||
```bash
|
||||
# Primary
|
||||
mkdir -p primary && cp config.json primary/ && cd primary
|
||||
mkdir -p primary && cp pando.json primary/ && cd primary
|
||||
APP_ROLE=primary \
|
||||
APP_BIND=0.0.0.0:3000 \
|
||||
APP_REPL_TOKEN=change-me \
|
||||
../target/debug/rust_test
|
||||
../target/debug/pando
|
||||
|
||||
# Replica (in another terminal; start with an EMPTY database)
|
||||
mkdir -p replica && cp config.json replica/ && cd replica
|
||||
mkdir -p replica && cp pando.json replica/ && cd replica
|
||||
APP_ROLE=replica \
|
||||
APP_BIND=127.0.0.1:3001 \
|
||||
APP_PRIMARY_URL=http://127.0.0.1:3000 \
|
||||
APP_REPL_TOKEN=change-me \
|
||||
../target/debug/rust_test
|
||||
../target/debug/pando
|
||||
```
|
||||
|
||||
A ready-made script, `repl-test.sh`, starts both on localhost, writes a few records and checks that they arrive:
|
||||
@@ -407,11 +408,11 @@ Step 4 is atomic, so a crash can never leave the cursor out of step with the dat
|
||||
| Source `instance_id` equals the replica's own | **Fatal.** Typically a cloned database file. |
|
||||
| Source `head_seq` < replica cursor | **Fatal.** Source was probably restored from an older backup. |
|
||||
|
||||
On a fatal mismatch the replica halts, because retrying cannot fix it, and the reason is in `sync.last_error` and the log. To repoint a replica at a different source, stop it, delete its `messages.db` and start it again so it resyncs from `seq` 0.
|
||||
On a fatal mismatch the replica halts, because retrying cannot fix it, and the reason is in `sync.last_error` and the log. To repoint a replica at a different source, stop it, delete its `pando.db` and start it again so it resyncs from `seq` 0.
|
||||
|
||||
### Operational notes
|
||||
|
||||
- **Bootstrapping:** start a replica with an empty database. Don't seed it by copying the primary's `messages.db`, because the copy shares the primary's instance ID and the replica will refuse to sync.
|
||||
- **Bootstrapping:** start a replica with an empty database. Don't seed it by copying the primary's `pando.db`, because the copy shares the primary's instance ID and the replica will refuse to sync.
|
||||
- **Transport security:** replication uses a bearer token over whatever transport you give it. Put TLS in front of the primary (reverse proxy) for anything beyond a trusted network.
|
||||
- **Chaining:** a replica serves `/api/replication/changes` too (if it has a token), so replicas can follow other replicas. Downstream replicas see the upstream replica's instance ID as their source.
|
||||
- **Promotion & Role Switching:** Replicas can be dynamically promoted at runtime without restart via `POST /api/admin/promote` or gracefully swapped via `POST /api/admin/switchover`. Nodes share Cluster CA credentials during initial join, allowing promoted primaries to issue join tokens and admit new nodes. Existing replicas can follow the new primary without resetting their databases via `POST /api/admin/repoint`.
|
||||
@@ -456,7 +457,7 @@ curl -s http://127.0.0.1:3001/api/replication/status
|
||||
|
||||
## Database Architecture
|
||||
|
||||
On startup, the application creates (or opens) a local SQLite database named `messages.db`:
|
||||
On startup, the application creates (or opens) a local SQLite database named `pando.db`:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS meta (
|
||||
|
||||
Reference in new issue
Block a user