ryyst/kalzu-value-store
1
0
Fork 1
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

Fragile hare

Browse Source
This commit is contained in:
ryyst
2025-09-09 22:23:20 +03:00
commit f86455dedb
5 changed files with 1186 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
.claude/

323
design_v2.md Normal file
View File

@ -0,0 +1,323 @@
# Gossip in GO, lazy syncing K/J database
## Software Design Document: Clustered Key-Value Store
### 1. Introduction
#### 1.1 Goals
This document outlines the design for a minimalistic, clustered key-value database system written in Go. The primary goals are:
* **Eventual Consistency:** Prioritize availability and partition tolerance over strong consistency.
* **Local-First Truth:** Local operations should be fast, with replication happening in the background.
* **Gossip-Style Membership:** Decentralized mechanism for nodes to discover and track each other.
* **Hierarchical Keys:** Support for structured keys (e.g., `/home/room/closet/socks`).
* **Minimalistic Footprint:** Efficient resource usage on servers.
* **Simple Configuration & Operation:** Easy to deploy and manage.
* **Read-Only Mode:** Ability for nodes to restrict external writes.
* **Gradual Bootstrapping:** New nodes integrate smoothly without overwhelming the cluster.
* **Sophisticated Conflict Resolution:** Handle timestamp collisions using majority vote, with oldest node as tie-breaker.
#### 1.2 Non-Goals
* Strong (linearizable/serializable) consistency.
* Complex querying or indexing beyond key-based lookups and timestamp-filtered UUID lists.
* Transaction support across multiple keys.
### 2. Architecture Overview
The system will consist of independent Go services (nodes) that communicate via HTTP/REST. Each node will embed a BadgerDB instance for local data storage and manage its own membership list through a gossip protocol. External clients interact with any available node, which then participates in the cluster's eventual consistency model.
**Key Architectural Principles:**
* **Decentralized:** No central coordinator or leader.
* **Peer-to-Peer:** Nodes communicate directly with each other for replication and membership.
* **API-Driven:** All interactions, both external (clients) and internal (replication), occur over a RESTful HTTP API.
```
+----------------+ +----------------+ +----------------+
| Node A | | Node B | | Node C |
| (Go Service) | | (Go Service) | | (Go Service) |
| | | | | |
| +------------+ | | +------------+ | | +------------+ |
| | HTTP Server| | <---- | | HTTP Server| | <---- | | HTTP Server| |
| | (API) | | ---> | | (API) | | ---> | | (API) | |
| +------------+ | | +------------+ | | +------------+ |
| | | | | | | | |
| +------------+ | | +------------+ | | +------------+ |
| | Gossip | | <---> | | Gossip | | <---> | | Gossip | |
| | Manager | | | | Manager | | | | Manager | |
| +------------+ | | +------------+ | | +------------+ |
| | | | | | | | |
| +------------+ | | +------------+ | | +------------+ |
| | Replication| | <---> | | Replication| | <---> | | Replication| |
| | Logic | | | | Logic | | | | Logic | |
| +------------+ | | +------------+ | | +------------+ |
| | | | | | | | |
| +------------+ | | +------------+ | | +------------+ |
| | BadgerDB | | | | BadgerDB | | | | BadgerDB | |
| | (Local KV) | | | | (Local KV) | | | | (Local KV) | |
| +------------+ | | +------------+ | | +------------+ |
+----------------+ +----------------+ +----------------+
^
|
+----- External Clients (Interact with any Node's API)
```
### 3. Data Model
#### 3.1 Logical Data Structure
Data is logically stored as a key-value pair, where the key is a hierarchical path and the value is a JSON object. Each pair also carries metadata for consistency and conflict resolution.
* **Logical Key:** `string` (e.g., `/home/room/closet/socks`)
* **Logical Value:** `JSON object` (e.g., `{"count":7,"colors":["blue","red","black"]}`)
#### 3.2 Internal Storage Structure (BadgerDB)
BadgerDB is a flat key-value store. To accommodate hierarchical keys and metadata, the following mapping will be used:
* **BadgerDB Key:** The full logical key path, with the leading `/kv/` prefix removed. Path segments will be separated by `/`. **No leading `/` will be stored in the BadgerDB key.**
* Example: For logical key `/kv/home/room/closet/socks`, the BadgerDB key will be `home/room/closet/socks`.
* **BadgerDB Value:** A marshaled JSON object containing the `uuid`, `timestamp`, and the actual `data` JSON object. This allows for consistent versioning and conflict resolution.
```json
// Example BadgerDB Value (marshaled JSON string)
{
"uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
"timestamp": 1672531200000, // Unix timestamp in milliseconds
"data": {
"count": 7,
"colors": ["blue", "red", "black"]
}
}
``` * **`uuid` (string):** A UUIDv4, unique identifier for this specific version of the data.
* **`timestamp` (int64):** Unix timestamp representing the time of the last modification. **This will be in milliseconds since epoch**, providing higher precision and reducing collision risk. This is the primary mechanism for conflict resolution ("newest data wins").
* **`data` (JSON object):** The actual user-provided JSON payload.
### 4. API Endpoints
All endpoints will communicate over HTTP/1.1 and utilize JSON for request/response bodies.
#### 4.1 `/kv/` Endpoints (Data Operations - External/Internal)
These endpoints are for direct key-value manipulation by external clients and are also used internally by nodes when fetching full data during replication.
* **`GET /kv/{path}`**
* **Description:** Retrieves the JSON object associated with the given hierarchical key path.
* **Request:** No body.
* **Responses:**
* `200 OK`: `Content-Type: application/json` with the stored JSON object.
* `404 Not Found`: If the key does not exist.
* `500 Internal Server Error`: For server-side issues (e.g., BadgerDB error).
* **Example:** `GET /kv/home/room/closet/socks` -> `{"count":7,"colors":["blue","red","black"]}`
* **`PUT /kv/{path}`**
* **Description:** Creates or updates a JSON object at the given path. This operation will internally generate a new UUIDv4 and assign the current Unix timestamp (milliseconds) to the stored value.
* **Request:**
* `Content-Type: application/json`
* Body: The JSON object to store.
* **Responses:**
* `200 OK` (Update) or `201 Created` (New): On success, returns `{"uuid": "new-uuid", "timestamp": new-timestamp_ms}`.
* `400 Bad Request`: If the request body is not valid JSON.
* `403 Forbidden`: If the node is in "read-only" mode and the request's origin is not a recognized cluster member (checked via IP/hostname).
* `500 Internal Server Error`: For server-side issues.
* **Example:** `PUT /kv/settings/theme` with body `{"color":"dark","font_size":14}` -> `{"uuid": "...", "timestamp": ...}`
* **`DELETE /kv/{path}`**
* **Description:** Deletes the key-value pair at the given path.
* **Request:** No body.
* **Responses:**
* `204 No Content`: On successful deletion.
* `404 Not Found`: If the key does not exist.
* `403 Forbidden`: If the node is in "read-only" mode and the request is not from a recognized cluster member.
* `500 Internal Server Error`: For server-side issues.
#### 4.2 `/members/` Endpoints (Membership & Internal Replication)
These endpoints are primarily for internal communication between cluster nodes, managing membership and facilitating data synchronization.
* **`GET /members/`**
* **Description:** Returns a list of known active members in the cluster. This list is maintained locally by each node based on the gossip protocol.
* **Request:** No body.
* **Responses:**
* `200 OK`: `Content-Type: application/json` with a JSON array of member details.
```json
[
{"id": "node-alpha", "address": "192.168.1.10:8080", "last_seen": 1672531200000, "joined_timestamp": 1672530000000},
{"id": "node-beta", "address": "192.168.1.11:8080", "last_seen": 1672531205000, "joined_timestamp": 1672530100000}
]
```
* `id` (string): Unique identifier for the node.
* `address` (string): `host:port` of the node's API endpoint.
* `last_seen` (int64): Unix timestamp (milliseconds) of when this node was last successfully contacted or heard from.
* `joined_timestamp` (int64): Unix timestamp (milliseconds) of when this node first joined the cluster. This is crucial for tie-breaking conflicts.
* `500 Internal Server Error`: For server-side issues.
* **`POST /members/join`**
* **Description:** Used by a new node to announce its presence and attempt to join the cluster. Existing nodes use this to update their member list and respond with their current view of the cluster.
* **Request:**
* `Content-Type: application/json`
* Body:
```json
{"id": "node-gamma", "address": "192.168.1.12:8080", "joined_timestamp": 1672532000000}
```
* `joined_timestamp` will be set by the joining node (its startup time).
* **Responses:**
* `200 OK`: Acknowledgment, returning the current list of known members to the joining node (same format as `GET /members/`).
* `400 Bad Request`: If the request body is malformed.
* `500 Internal Server Error`: For server-side issues.
* **`DELETE /members/leave` (Optional, for graceful shutdown)**
* **Description:** A member can proactively announce its departure from the cluster. This allows other nodes to quickly mark it as inactive.
* **Request:**
* `Content-Type: application/json`
* Body: `{"id": "node-gamma"}`
* **Responses:**
* `204 No Content`: On successful processing.
* `400 Bad Request`: If the request body is malformed.
* `500 Internal Server Error`: For server-side issues.
* **`POST /members/pairs_by_time` (Internal/Replication Endpoint)**
* **Description:** Used by other cluster members to request a list of key paths, their UUIDs, and their timestamps within a specified time range, optionally filtered by a key prefix. This is critical for both gradual bootstrapping and the regular 5-minute synchronization.
* **Request:**
* `Content-Type: application/json`
* Body:
```json
{
"start_timestamp": 1672531200000, // Unix milliseconds (inclusive)
"end_timestamp": 1672617600000, // Unix milliseconds (exclusive), or 0 for "up to now"
"limit": 15, // Max number of pairs to return
"prefix": "home/room/" // Optional: filter by BadgerDB key prefix
}
```
* `start_timestamp`: Earliest timestamp for data to be included.
* `end_timestamp`: Latest timestamp (exclusive). If `0` or omitted, it implies "up to the current time".
* `limit`: **Fixed at 15** for this design, to control batch size during sync.
* `prefix`: Optional, to filter keys by a common BadgerDB key prefix.
* **Responses:**
* `200 OK`: `Content-Type: application/json` with a JSON array of objects:
```json
[
{"path": "home/room/closet/socks", "uuid": "...", "timestamp": 1672531200000},
{"path": "users/john/profile", "uuid": "...", "timestamp": 1672531205000}
]
```
* `204 No Content`: If no data matches the criteria.
* `400 Bad Request`: If request body is malformed or timestamps are invalid.
* `500 Internal Server Error`: For server-side issues.
### 5. BadgerDB Integration
BadgerDB will be used as the embedded, local, single-node key-value store.
* **Key Storage:** As described in section 3.2, the HTTP path (without `/kv/` prefix and no leading `/`) will directly map to the BadgerDB key.
* **Value Storage:** Values will be marshaled JSON objects (`uuid`, `timestamp`, `data`).
* **Timestamp Indexing (for `pairs_by_time`):** To efficiently query by timestamp, a manual secondary index will be maintained. Each `PUT` operation will write two BadgerDB entries:
1. The primary data entry: `{badger_key}` -> `{uuid, timestamp, data}`.
2. A secondary timestamp index entry: `_ts:{timestamp_ms}:{badger_key}` -> `{uuid}`.
* The `_ts` prefix ensures these index keys are grouped and don't conflict with data keys.
* The timestamp (milliseconds) ensures lexicographical sorting by time.
* The `badger_key` in the index key allows for uniqueness and points back to the main data.
* The value can simply be the `uuid` or even an empty string if only the key is needed. Storing the `uuid` here is useful for direct lookups.
* **`DELETE` Operations:** A `DELETE /kv/{path}` will remove both the primary data entry and its corresponding secondary index entry from BadgerDB.
### 6. Clustering and Consistency
#### 6.1 Membership Management (Gossip Protocol)
* Each node maintains a local list of known cluster members (Node ID, Address, Last Seen Timestamp, Joined Timestamp).
* Every node will randomly pick a time **between 1-2 minutes** after its last check-up to initiate a gossip round.
* In a gossip round, the node randomly selects a subset of its healthy known members (e.g., 1-3 nodes) and performs a "gossip exchange":
1. It sends its current local member list to the selected peers.
2. Peers merge the received list with their own, updating `last_seen` timestamps for existing members and adding new ones.
3. If a node fails to respond to multiple gossip attempts, it is eventually marked as "suspected down" and then "dead" after a configurable timeout.
#### 6.2 Data Replication (Periodic Syncs)
* The system uses two types of data synchronization:
1. **Regular 5-Minute Sync:** Catching up on recent changes.
2. **Catch-Up Sync (2-Minute Cycles):** For nodes that detect they are significantly behind.
* **Regular 5-Minute Sync:**
* Every **5 minutes**, each node initiates a data synchronization cycle.
* It selects a random healthy peer.
* It sends `POST /members/pairs_by_time` to the peer, requesting **the 15 latest UUIDs** (by setting `limit: 15` and `end_timestamp: current_time_ms`, with `start_timestamp: 0` or a very old value to ensure enough items are considered).
* The remote node responds with its 15 latest (path, uuid, timestamp) pairs.
* The local node compares these with its own latest 15. If it finds any data it doesn't have, or an older version of data it does have, it will fetch the full data via `GET /kv/{path}` and update its local store.
* If the local node detects it's significantly behind (e.g., many of the remote node's latest 15 UUIDs are missing or much newer locally, indicating a large gap), it triggers the **Catch-Up Sync**.
* **Catch-Up Sync (2-Minute Cycles):**
* This mode is activated when a node determines it's behind its peers (e.g., during the 5-minute sync or bootstrapping).
* It runs every **2 minutes** (ensuring it doesn't align with the 5-minute sync).
* The node identifies the `oldest_known_timestamp_among_peers_latest_15` from its last regular sync.
* It then sends `POST /members/pairs_by_time` to a random healthy peer, requesting **15 UUIDs older than that timestamp** (e.g., `end_timestamp: oldest_known_timestamp_ms`, `limit: 15`, `start_timestamp: 0` or further back).
* It continuously iterates backwards in time in 2-minute cycles, progressively asking for older sets of 15 UUIDs until it has caught up to a reasonable historical depth (e.g., configured `BOOTSTRAP_MAX_AGE_HOURS`).
* **History Depth:** The system aims to keep track of **at least 3 revisions per path** for conflict resolution and eventually for versioning. The `BOOTSTRAP_MAX_AGE_MILLIS` (defaulting to 30 days) governs how far back in time a node will actively fetch during a full sync.
#### 6.3 Conflict Resolution
When two nodes have different versions of the same key (same BadgerDB key), the conflict resolution logic is applied:
1. **Timestamp Wins:** The data with the **most recent `timestamp` (Unix milliseconds)** is considered the correct version.
2. **Timestamp Collision (Tie-Breaker):** If two conflicting versions have the **exact same `timestamp`**:
* **Majority Vote:** The system will query a quorum of healthy peers (`GET /kv/{path}` or an internal check for UUID/timestamp) to see which UUID/timestamp pair the majority holds. The version held by the majority wins.
* **Oldest Node Priority (Tie-Breaker for Majority):** If there's an even number of nodes, and thus a tie in the majority vote (e.g., 2 nodes say version A, 2 nodes say version B), the version held by the node with the **oldest `joined_timestamp`** (i.e., the oldest active member in the cluster) takes precedence. This provides a deterministic tie-breaker.
* *Implementation Note:* For majority vote, a node might need to request the `{"uuid", "timestamp"}` pairs for a specific `path` from multiple peers. This implies an internal query mechanism or aggregating responses from `pairs_by_time` for the specific key.
### 7. Bootstrapping New Nodes (Gradual Full Sync)
This process is initiated when a new node starts up and has no existing data or member list.
1. **Seed Node Configuration:** The new node must be configured with a list of initial `seed_nodes` (e.g., `["host1:port", "host2:port"]`).
2. **Join Request:** The new node attempts to `POST /members/join` to one of its configured seed nodes, providing its own `id`, `address`, and its `joined_timestamp` (its startup time).
3. **Member List Discovery:** Upon a successful join, the seed node responds with its current list of known cluster members. The new node populates its local member list.
4. **Gradual Data Synchronization Loop (Catch-Up Mode):**
* The new node sets its `current_end_timestamp = current_time_ms`.
* It defines a `sync_batch_size` (e.g., 15 UUIDs per request, as per `pairs_by_time` `limit`).
* It also defines a `throttle_delay` (e.g., 100ms between `pairs_by_time` requests to different peers) and a `fetch_delay` (e.g., 50ms between individual `GET /kv/{path}` requests for full data).
* **Loop backwards in time:**
* The node determines the `oldest_timestamp_fetched` from its *last* batch of `sync_batch_size` items. Initially, this would be `current_time_ms`.
* Randomly pick a healthy peer from its member list.
* Send `POST /members/pairs_by_time` to the peer with `end_timestamp: oldest_timestamp_fetched`, `limit: sync_batch_size`, and `start_timestamp: 0`. This asks for 15 items *older than* the oldest one just processed.
* Process the received `{"path", "uuid", "timestamp"}` pairs:
* For each remote pair, it fetches its local version from BadgerDB.
* **Conflict Resolution:** Apply the logic from section 6.3. If local data is missing or older, initiate a `GET /kv/{path}` to fetch the full data and store it.
* **Throttling:**
* Wait for `throttle_delay` after each `pairs_by_time` request.
* Wait for `fetch_delay` after each individual `GET /kv/{path}` request for full data.
* **Termination:** The loop continues until the `oldest_timestamp_fetched` goes below the configured `BOOTSTRAP_MAX_AGE_MILLIS` (defaulting to 30 days ago, configurable value). The node may also terminate if multiple consecutive `pairs_by_time` queries return no new (older) data.
5. **Full Participation:** Once the gradual sync is complete, the node fully participates in the regular 5-minute replication cycles and accepts external client writes (if not in read-only mode). During the sync, the node will operate in a `syncing` mode, rejecting external client writes with `503 Service Unavailable`.
### 8. Operational Modes
* **Normal Mode:** Full read/write capabilities, participates in all replication and gossip activities.
* **Read-Only Mode:**
* Node will reject `PUT` and `DELETE` requests from **external clients** with a `403 Forbidden` status.
* It will **still accept** `PUT` and `DELETE` operations that originate from **recognized cluster members** during replication, allowing it to remain eventually consistent.
* `GET` requests are always allowed.
* This mode is primarily for reducing write load or protecting data on specific nodes.
* **Syncing Mode (Internal during Bootstrap):**
* While a new node is undergoing its initial gradual sync, it operates in this internal mode.
* External `PUT`/`DELETE` requests will be **rejected with `503 Service Unavailable`**.
* Internal replication from other members is fully active.
### 9. Logging
A structured logging library (e.g., `zap` or `logrus`) will be used.
* **Log Levels:** Support for `DEBUG`, `INFO`, `WARN`, `ERROR`, `FATAL`. Configurable.
* **Log Format:** JSON for easy parsing by log aggregators.
* **Key Events to Log:**
* **Startup/Shutdown:** Server start/stop, configuration loaded.
* **API Requests:** Incoming HTTP request details (method, path, client IP, status code, duration).
* **BadgerDB Operations:** Errors during put/get/delete, database open/close, secondary index operations.
* **Membership:** Node joined/left, gossip rounds initiated/received, member status changes (up, suspected, down), tie-breaker decisions.
* **Replication:** Sync cycle start/end, type of sync (regular/catch-up), number of keys compared, number of keys fetched, conflict resolutions (including details of timestamp collision resolution).
* **Errors:** Data serialization/deserialization, network errors, unhandled exceptions.
* **Operational Mode Changes:** Entering/exiting read-only mode, syncing mode.
### 10. Future Work (Rough Order of Priority)
These items are considered out of scope for the initial design but are planned for future versions.
* **Authentication/Authorization (Before First Release):** Implement robust authentication for API endpoints (e.g., API keys, mTLS) and potentially basic authorization for access to `kv` paths.
* **Client Libraries/Functions (Bash, Python, Go):** Develop official client libraries or helper functions to simplify interaction with the API for common programming environments.
* **Data Compression (gzip):** Implement Gzip compression for data values stored in BadgerDB to reduce storage footprint and potentially improve I/O performance.
* **Data Revisions & Simple Backups:**
* Hold **at least 3 revisions per path**. This would involve a mechanism to store previous versions of data when a `PUT` occurs, potentially using a separate BadgerDB key namespace (e.g., `_rev:{badger_key}:{timestamp_of_revision}`).
* The current `GET /kv/{path}` would continue to return only the latest. A new API might be introduced to fetch specific historical revisions.
* Simple backup strategies could leverage these revisions or BadgerDB's native snapshot capabilities.
* **Monitoring & Metrics (Grafana Support in v3):** Integrate with a metrics system like Prometheus, exposing key performance indicators (e.g., request rates, error rates, replication lag, BadgerDB stats) for visualization in dashboards like Grafana.

30
go.mod Normal file
View File

@ -0,0 +1,30 @@
module kvs
go 1.21
require (
github.com/dgraph-io/badger/v4 v4.2.0
github.com/google/uuid v1.4.0
github.com/gorilla/mux v1.8.1
github.com/sirupsen/logrus v1.9.3
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dustin/go-humanize v1.0.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.3 // indirect
github.com/google/flatbuffers v1.12.1 // indirect
github.com/klauspost/compress v1.12.3 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
go.opencensus.io v0.22.5 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/sys v0.5.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
)

131
go.sum Normal file
View File

@ -0,0 +1,131 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgraph-io/badger/v4 v4.2.0 h1:kJrlajbXXL9DFTNuhhu9yCx7JJa4qpYWxtE8BzuWsEs=
github.com/dgraph-io/badger/v4 v4.2.0/go.mod h1:qfCqhPoWDFJRx1gp5QwwyGo8xk1lbHUxvK9nK0OGAak=
github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=
github.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2 h1:tdlZCpZ/P9DhczCTSixgIKmwPv6+wP5DGjqLYw5SUiA=
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/glog v1.0.0 h1:nfP3RFugxnNRyKgeWd4oI1nYvXpxrx8ck8ZrcizshdQ=
github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 h1:ZgQEtGgCBiWRM39fZuwSd1LwSqqSW0hOdXCYYDX0R3I=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/flatbuffers v1.12.1 h1:MVlul7pQNoDzWRLTw5imwYsl+usrS1TXG2H4jg6ImGw=
github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.12.3 h1:G5AfA94pHPysR56qqrkO2pxEexdDzrpFJ6yt/VqWxVU=
github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.22.5 h1:dntmOdLpSpHlVqbW5Eay97DelsZHe+55D+xC6i0dDS0=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

701
main.go Normal file
View File

@ -0,0 +1,701 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net"
"net/http"
"os"
"os/signal"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
"time"
badger "github.com/dgraph-io/badger/v4"
"github.com/google/uuid"
"github.com/gorilla/mux"
"github.com/sirupsen/logrus"
"gopkg.in/yaml.v3"
)
// Core data structures
type StoredValue struct {
UUID string `json:"uuid"`
Timestamp int64 `json:"timestamp"`
Data json.RawMessage `json:"data"`
}
type Member struct {
ID string `json:"id"`
Address string `json:"address"`
LastSeen int64 `json:"last_seen"`
JoinedTimestamp int64 `json:"joined_timestamp"`
}
type JoinRequest struct {
ID string `json:"id"`
Address string `json:"address"`
JoinedTimestamp int64 `json:"joined_timestamp"`
}
type LeaveRequest struct {
ID string `json:"id"`
}
type PairsByTimeRequest struct {
StartTimestamp int64 `json:"start_timestamp"`
EndTimestamp int64 `json:"end_timestamp"`
Limit int `json:"limit"`
Prefix string `json:"prefix,omitempty"`
}
type PairsByTimeResponse struct {
Path string `json:"path"`
UUID string `json:"uuid"`
Timestamp int64 `json:"timestamp"`
}
type PutResponse struct {
UUID string `json:"uuid"`
Timestamp int64 `json:"timestamp"`
}
// Configuration
type Config struct {
NodeID string `yaml:"node_id"`
BindAddress string `yaml:"bind_address"`
Port int `yaml:"port"`
DataDir string `yaml:"data_dir"`
SeedNodes []string `yaml:"seed_nodes"`
ReadOnly bool `yaml:"read_only"`
LogLevel string `yaml:"log_level"`
GossipIntervalMin int `yaml:"gossip_interval_min"`
GossipIntervalMax int `yaml:"gossip_interval_max"`
SyncInterval int `yaml:"sync_interval"`
CatchupInterval int `yaml:"catchup_interval"`
BootstrapMaxAgeHours int `yaml:"bootstrap_max_age_hours"`
ThrottleDelayMs int `yaml:"throttle_delay_ms"`
FetchDelayMs int `yaml:"fetch_delay_ms"`
}
// Server represents the KVS node
type Server struct {
config *Config
db *badger.DB
members map[string]*Member
membersMu sync.RWMutex
mode string // "normal", "read-only", "syncing"
modeMu sync.RWMutex
logger *logrus.Logger
httpServer *http.Server
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
}
// Default configuration
func defaultConfig() *Config {
hostname, _ := os.Hostname()
return &Config{
NodeID: hostname,
BindAddress: "127.0.0.1",
Port: 8080,
DataDir: "./data",
SeedNodes: []string{},
ReadOnly: false,
LogLevel: "info",
GossipIntervalMin: 60, // 1 minute
GossipIntervalMax: 120, // 2 minutes
SyncInterval: 300, // 5 minutes
CatchupInterval: 120, // 2 minutes
BootstrapMaxAgeHours: 720, // 30 days
ThrottleDelayMs: 100,
FetchDelayMs: 50,
}
}
// Load configuration from file or create default
func loadConfig(configPath string) (*Config, error) {
config := defaultConfig()
if _, err := os.Stat(configPath); os.IsNotExist(err) {
// Create default config file
if err := os.MkdirAll(filepath.Dir(configPath), 0755); err != nil {
return nil, fmt.Errorf("failed to create config directory: %v", err)
}
data, err := yaml.Marshal(config)
if err != nil {
return nil, fmt.Errorf("failed to marshal default config: %v", err)
}
if err := os.WriteFile(configPath, data, 0644); err != nil {
return nil, fmt.Errorf("failed to write default config: %v", err)
}
fmt.Printf("Created default configuration at %s\n", configPath)
return config, nil
}
data, err := os.ReadFile(configPath)
if err != nil {
return nil, fmt.Errorf("failed to read config file: %v", err)
}
if err := yaml.Unmarshal(data, config); err != nil {
return nil, fmt.Errorf("failed to parse config file: %v", err)
}
return config, nil
}
// Initialize server
func NewServer(config *Config) (*Server, error) {
logger := logrus.New()
logger.SetFormatter(&logrus.JSONFormatter{})
level, err := logrus.ParseLevel(config.LogLevel)
if err != nil {
level = logrus.InfoLevel
}
logger.SetLevel(level)
// Create data directory
if err := os.MkdirAll(config.DataDir, 0755); err != nil {
return nil, fmt.Errorf("failed to create data directory: %v", err)
}
// Open BadgerDB
opts := badger.DefaultOptions(filepath.Join(config.DataDir, "badger"))
opts.Logger = nil // Disable badger's internal logging
db, err := badger.Open(opts)
if err != nil {
return nil, fmt.Errorf("failed to open BadgerDB: %v", err)
}
ctx, cancel := context.WithCancel(context.Background())
server := &Server{
config: config,
db: db,
members: make(map[string]*Member),
mode: "normal",
logger: logger,
ctx: ctx,
cancel: cancel,
}
if config.ReadOnly {
server.setMode("read-only")
}
return server, nil
}
// Mode management
func (s *Server) getMode() string {
s.modeMu.RLock()
defer s.modeMu.RUnlock()
return s.mode
}
func (s *Server) setMode(mode string) {
s.modeMu.Lock()
defer s.modeMu.Unlock()
oldMode := s.mode
s.mode = mode
s.logger.WithFields(logrus.Fields{
"old_mode": oldMode,
"new_mode": mode,
}).Info("Mode changed")
}
// Member management
func (s *Server) addMember(member *Member) {
s.membersMu.Lock()
defer s.membersMu.Unlock()
s.members[member.ID] = member
s.logger.WithFields(logrus.Fields{
"node_id": member.ID,
"address": member.Address,
}).Info("Member added")
}
func (s *Server) removeMember(nodeID string) {
s.membersMu.Lock()
defer s.membersMu.Unlock()
if member, exists := s.members[nodeID]; exists {
delete(s.members, nodeID)
s.logger.WithFields(logrus.Fields{
"node_id": member.ID,
"address": member.Address,
}).Info("Member removed")
}
}
func (s *Server) getMembers() []*Member {
s.membersMu.RLock()
defer s.membersMu.RUnlock()
members := make([]*Member, 0, len(s.members))
for _, member := range s.members {
members = append(members, member)
}
return members
}
// HTTP Handlers
func (s *Server) healthHandler(w http.ResponseWriter, r *http.Request) {
mode := s.getMode()
memberCount := len(s.getMembers())
health := map[string]interface{}{
"status": "ok",
"mode": mode,
"member_count": memberCount,
"node_id": s.config.NodeID,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(health)
}
func (s *Server) getKVHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
path := vars["path"]
var storedValue StoredValue
err := s.db.View(func(txn *badger.Txn) error {
item, err := txn.Get([]byte(path))
if err != nil {
return err
}
return item.Value(func(val []byte) error {
return json.Unmarshal(val, &storedValue)
})
})
if err == badger.ErrKeyNotFound {
http.Error(w, "Not Found", http.StatusNotFound)
return
}
if err != nil {
s.logger.WithError(err).WithField("path", path).Error("Failed to get value")
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
w.Write(storedValue.Data)
}
func (s *Server) putKVHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
path := vars["path"]
mode := s.getMode()
if mode == "syncing" {
http.Error(w, "Service Unavailable", http.StatusServiceUnavailable)
return
}
if mode == "read-only" && !s.isClusterMember(r.RemoteAddr) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
var data json.RawMessage
if err := json.NewDecoder(r.Body).Decode(&data); err != nil {
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
now := time.Now().UnixMilli()
newUUID := uuid.New().String()
storedValue := StoredValue{
UUID: newUUID,
Timestamp: now,
Data: data,
}
valueBytes, err := json.Marshal(storedValue)
if err != nil {
s.logger.WithError(err).Error("Failed to marshal stored value")
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
var isUpdate bool
err = s.db.Update(func(txn *badger.Txn) error {
// Check if key exists
_, err := txn.Get([]byte(path))
isUpdate = (err == nil)
// Store main data
if err := txn.Set([]byte(path), valueBytes); err != nil {
return err
}
// Store timestamp index
indexKey := fmt.Sprintf("_ts:%020d:%s", now, path)
return txn.Set([]byte(indexKey), []byte(newUUID))
})
if err != nil {
s.logger.WithError(err).WithField("path", path).Error("Failed to store value")
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
response := PutResponse{
UUID: newUUID,
Timestamp: now,
}
status := http.StatusCreated
if isUpdate {
status = http.StatusOK
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
json.NewEncoder(w).Encode(response)
s.logger.WithFields(logrus.Fields{
"path": path,
"uuid": newUUID,
"timestamp": now,
"is_update": isUpdate,
}).Info("Value stored")
}
func (s *Server) deleteKVHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
path := vars["path"]
mode := s.getMode()
if mode == "syncing" {
http.Error(w, "Service Unavailable", http.StatusServiceUnavailable)
return
}
if mode == "read-only" && !s.isClusterMember(r.RemoteAddr) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
var found bool
err := s.db.Update(func(txn *badger.Txn) error {
// Check if key exists and get timestamp for index cleanup
item, err := txn.Get([]byte(path))
if err == badger.ErrKeyNotFound {
return nil
}
if err != nil {
return err
}
found = true
var storedValue StoredValue
err = item.Value(func(val []byte) error {
return json.Unmarshal(val, &storedValue)
})
if err != nil {
return err
}
// Delete main data
if err := txn.Delete([]byte(path)); err != nil {
return err
}
// Delete timestamp index
indexKey := fmt.Sprintf("_ts:%020d:%s", storedValue.Timestamp, path)
return txn.Delete([]byte(indexKey))
})
if err != nil {
s.logger.WithError(err).WithField("path", path).Error("Failed to delete value")
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
if !found {
http.Error(w, "Not Found", http.StatusNotFound)
return
}
w.WriteHeader(http.StatusNoContent)
s.logger.WithField("path", path).Info("Value deleted")
}
func (s *Server) getMembersHandler(w http.ResponseWriter, r *http.Request) {
members := s.getMembers()
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(members)
}
func (s *Server) joinMemberHandler(w http.ResponseWriter, r *http.Request) {
var req JoinRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
now := time.Now().UnixMilli()
member := &Member{
ID: req.ID,
Address: req.Address,
LastSeen: now,
JoinedTimestamp: req.JoinedTimestamp,
}
s.addMember(member)
// Return current member list
members := s.getMembers()
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(members)
}
func (s *Server) leaveMemberHandler(w http.ResponseWriter, r *http.Request) {
var req LeaveRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
s.removeMember(req.ID)
w.WriteHeader(http.StatusNoContent)
}
func (s *Server) pairsByTimeHandler(w http.ResponseWriter, r *http.Request) {
var req PairsByTimeRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
// Default limit to 15 as per spec
if req.Limit <= 0 {
req.Limit = 15
}
var pairs []PairsByTimeResponse
err := s.db.View(func(txn *badger.Txn) error {
opts := badger.DefaultIteratorOptions
opts.PrefetchSize = req.Limit
it := txn.NewIterator(opts)
defer it.Close()
prefix := []byte("_ts:")
if req.Prefix != "" {
// We need to scan through timestamp entries and filter by path prefix
}
for it.Seek(prefix); it.ValidForPrefix(prefix) && len(pairs) < req.Limit; it.Next() {
item := it.Item()
key := string(item.Key())
// Parse timestamp index key: "_ts:{timestamp}:{path}"
parts := strings.SplitN(key, ":", 3)
if len(parts) != 3 {
continue
}
timestamp, err := strconv.ParseInt(parts[1], 10, 64)
if err != nil {
continue
}
// Filter by timestamp range
if req.StartTimestamp > 0 && timestamp < req.StartTimestamp {
continue
}
if req.EndTimestamp > 0 && timestamp >= req.EndTimestamp {
continue
}
path := parts[2]
// Filter by prefix if specified
if req.Prefix != "" && !strings.HasPrefix(path, req.Prefix) {
continue
}
var uuid string
err = item.Value(func(val []byte) error {
uuid = string(val)
return nil
})
if err != nil {
continue
}
pairs = append(pairs, PairsByTimeResponse{
Path: path,
UUID: uuid,
Timestamp: timestamp,
})
}
return nil
})
if err != nil {
s.logger.WithError(err).Error("Failed to query pairs by time")
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
if len(pairs) == 0 {
w.WriteHeader(http.StatusNoContent)
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(pairs)
}
// Utility function to check if request is from cluster member
func (s *Server) isClusterMember(remoteAddr string) bool {
host, _, err := net.SplitHostPort(remoteAddr)
if err != nil {
return false
}
s.membersMu.RLock()
defer s.membersMu.RUnlock()
for _, member := range s.members {
memberHost, _, err := net.SplitHostPort(member.Address)
if err == nil && memberHost == host {
return true
}
}
return false
}
// Setup HTTP routes
func (s *Server) setupRoutes() *mux.Router {
router := mux.NewRouter()
// Health endpoint
router.HandleFunc("/health", s.healthHandler).Methods("GET")
// KV endpoints
router.HandleFunc("/kv/{path:.+}", s.getKVHandler).Methods("GET")
router.HandleFunc("/kv/{path:.+}", s.putKVHandler).Methods("PUT")
router.HandleFunc("/kv/{path:.+}", s.deleteKVHandler).Methods("DELETE")
// Member endpoints
router.HandleFunc("/members/", s.getMembersHandler).Methods("GET")
router.HandleFunc("/members/join", s.joinMemberHandler).Methods("POST")
router.HandleFunc("/members/leave", s.leaveMemberHandler).Methods("DELETE")
router.HandleFunc("/members/pairs_by_time", s.pairsByTimeHandler).Methods("POST")
return router
}
// Start the server
func (s *Server) Start() error {
router := s.setupRoutes()
addr := fmt.Sprintf("%s:%d", s.config.BindAddress, s.config.Port)
s.httpServer = &http.Server{
Addr: addr,
Handler: router,
}
s.logger.WithFields(logrus.Fields{
"node_id": s.config.NodeID,
"address": addr,
}).Info("Starting KVS server")
// Start gossip and sync routines
s.startBackgroundTasks()
// Try to join cluster if seed nodes are configured
if len(s.config.SeedNodes) > 0 {
go s.bootstrap()
}
return s.httpServer.ListenAndServe()
}
// Stop the server gracefully
func (s *Server) Stop() error {
s.logger.Info("Shutting down KVS server")
s.cancel()
s.wg.Wait()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := s.httpServer.Shutdown(ctx); err != nil {
s.logger.WithError(err).Error("HTTP server shutdown error")
}
if err := s.db.Close(); err != nil {
s.logger.WithError(err).Error("BadgerDB close error")
}
return nil
}
// Background tasks placeholder (gossip, sync, etc.)
func (s *Server) startBackgroundTasks() {
// TODO: Implement gossip protocol
// TODO: Implement periodic sync
// TODO: Implement catch-up sync
}
// Bootstrap placeholder
func (s *Server) bootstrap() {
// TODO: Implement gradual bootstrapping
}
func main() {
configPath := "./config.yaml"
// Simple CLI argument parsing
if len(os.Args) > 1 {
configPath = os.Args[1]
}
config, err := loadConfig(configPath)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to load configuration: %v\n", err)
os.Exit(1)
}
server, err := NewServer(config)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to create server: %v\n", err)
os.Exit(1)
}
// Handle graceful shutdown
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigCh
server.Stop()
}()
if err := server.Start(); err != nil && err != http.ErrServerClosed {
fmt.Fprintf(os.Stderr, "Server error: %v\n", err)
os.Exit(1)
}
}