Compare commits

7 Commits

Author SHA1 Message Date
Kalzu Rekku
3287c38836 Fix merge left overs. 2025-10-05 22:53:20 +03:00
Kalzu Rekku
f7f956de47 Merge remote-tracking branch 'origin/risto' 2025-10-05 22:45:26 +03:00
Kalzu Rekku
fe45fa5254 Added commands to set meta data, like owner, group and permissions of key. 2025-10-05 22:38:25 +03:00
999deccb74 fix: JSON quote parsing 2025-10-05 22:27:26 +03:00
88a3b9824a fix: show friendlier error message for JSON parsing 2025-10-05 22:27:26 +03:00
8edbfacda5 Add Claude 2025-10-05 22:27:26 +03:00
Kalzu Rekku
ad17dff2fa Fix json quotation bug. 2025-10-05 21:39:09 +03:00
2 changed files with 429 additions and 18 deletions

160
CLAUDE.md Normal file
View File

@@ -0,0 +1,160 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**kvs-sh** is an interactive CLI shell for the KVS (Key-Value Store) distributed database. Think `mongosh`, `psql`, or `redis-cli` but for KVS. Single Go file implementation (~600 lines) with readline support for command history and auto-completion.
## KVS Server Reference
The complete KVS server source code is located at `../kvs/` relative to this repository. Reference it when:
- Understanding API contracts and response formats
- Verifying authentication/authorization behavior
- Checking endpoint paths and HTTP methods
- Understanding data structures (StoredValue, Member, User, Group types)
- Implementing new shell commands for backend features
Key server locations:
- `../kvs/server/handlers.go` - All API endpoint implementations
- `../kvs/server/routes.go` - Route definitions and HTTP methods
- `../kvs/types/types.go` - Canonical type definitions (StoredValue, etc.)
- `../kvs/auth/` - JWT authentication and permission system details
- `../kvs/README.md` - Complete API documentation with curl examples
- `../kvs/CLAUDE.md` - Server architecture and development guidance
When adding new commands to kvs-sh, always verify the actual backend implementation in `../kvs/` rather than making assumptions about API behavior.
## Build & Run
```bash
# Build
go build -o kvs-shell
# Run
./kvs-shell
# Install to system
go install
```
## Quick Start with Server
```bash
# 1. Start KVS server (in ../kvs/)
cd ../kvs && ./kvs config.yaml
# 2. Run shell
./kvs-shell
# 3. In shell:
# connect http://localhost:8080
# auth $KVS_ROOT_TOKEN
# put test/data '{"hello":"world"}'
```
**Important:** If auth fails with "token not found", the database has old data. Either:
- Create a new token via the API, or
- Wipe database: `rm -rf ../kvs/data && mkdir ../kvs/data` (restart server to generate root token)
## Testing
```bash
go test ./...
```
## Architecture
### Single-File Design
Everything lives in `main.go` - intentionally kept simple. The shell is a stateful REPL that maintains:
- HTTP client connection to KVS backend
- Current JWT auth token
- In-memory profile storage (username, token, server URL combinations)
- Active profile selection
### Core Components
**KVSClient (main.go:49)**: Central state container holding:
- `baseURL`: KVS server endpoint
- `currentToken`: JWT for authentication
- `profiles`: In-memory map of saved user profiles (NOT persisted to disk yet)
- `httpClient`: Reusable HTTP client with 30s timeout
**Command Router (main.go:465)**: Main loop uses switch statement on first word of input, delegates to `handle*` methods
**Command Parser (main.go:527)**: Custom parser respecting quotes - critical for JSON arguments like `put key '{"foo": "bar"}'`
**Auto-completion (main.go:503)**: Prefix tree using readline library for tab completion of commands
### HTTP Communication Pattern
All API calls go through `doRequest()` (main.go:85) which:
1. Serializes body to JSON if present
2. Sets `Authorization: Bearer <token>` header if token exists
3. Returns `(body, statusCode, error)` tuple
4. Delegates HTTP status interpretation to caller
### Key Backend API Mappings
- `GET /kv/<key>` → Returns `StoredValue` with UUID, timestamp, data
- `PUT /kv/<key>` → Accepts raw JSON, returns UUID + timestamp
- `DELETE /kv/<key>` → Returns 204 on success
- `GET /members/` → Returns array of cluster `Member` objects
- `GET /api/users/<uuid>` → Returns `User` object
- `GET /health` → Returns service status map
### Type Definitions (main.go:18-46)
Backend response structures are mirrored in the client:
- `StoredValue`: The core KV response with UUID, timestamp (int64 unix millis), and `json.RawMessage` data
- `Member`: Cluster member info with ID, address, last_seen timestamp
- `User`: User record with nickname hash, groups array, created/updated timestamps
- `Group`: Group definition (defined but not yet used in commands)
### Profile System
Profiles are in-memory only (see roadmap for persistence plan). When you `profile use <name>`, it swaps the client's token, userUUID, and baseURL atomically. The `*` marker in profile listings indicates active profile.
### Color Scheme
Uses fatih/color package with semantic coloring:
- Cyan: labels, headers, structured data keys
- Green: success messages, confirmations
- Yellow: warnings, "not found" states
- Red: errors, invalid usage
- Magenta: banner/title text
### Command Parsing - Critical Implementation Details
**parseCommand()** (main.go:527-570): Handles both single (`'`) and double (`"`) quotes intelligently.
**Key behavior:**
- Quotes are **only stripped** when they delimit an entire argument (start and end)
- Supports both `'` and `"` as quote delimiters
- Tracks which quote type started the argument to handle nesting correctly
- Preserves quotes that appear mid-argument (for unquoted JSON)
**Examples:**
```bash
put key '{"foo":"bar"}' # ✅ Strips outer ', preserves inner "
put key {"foo":"bar"} # ✅ Works (no spaces, compact JSON)
put key {"foo": "bar"} # ❌ Fails - spaces split into multiple args
put key "{\"foo\":\"bar\"}" # ✅ Works (escaped quotes in double-quoted arg)
```
**JSON Error Handling** (main.go:393-425):
- `printJSONError()` provides user-friendly error messages with examples
- Detects error type (invalid chars, incomplete JSON, etc.) and shows context-specific tips
- Shows what input was received to help debug quoting issues
## Backend Assumptions
The shell expects the KVS backend to:
1. Accept JWT tokens in `Authorization: Bearer <token>` header
2. Return timestamps as Unix milliseconds (int64)
3. Use standard REST semantics (404 for not found, 204 for delete, 200/201 for success)
4. Accept raw JSON payloads for PUT operations (not wrapped in metadata)
## Known Limitations & Roadmap Items
1. **No persistent profiles** - restart loses all `profile add` data (roadmap #1)
2. **No metadata commands** - backend supports owner/group/permissions but shell doesn't expose yet (roadmap #6)
3. **No group management** - `Group` type defined but no CRUD operations (roadmap #5)
4. **No TLS support** - HTTP only, no certificate handling (roadmap #8)
5. **No tab completion for keys** - only completes command names, not dynamic keys from server (roadmap #10)
6. **user create not implemented** - Listed in help but just shows "not implemented" message

287
main.go
View File

@@ -2,12 +2,14 @@
package main
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
@@ -228,22 +230,38 @@ func (c *KVSClient) handleGet(args []string) {
fmt.Println(cyan("UUID: "), stored.UUID)
fmt.Println(cyan("Timestamp:"), time.UnixMilli(stored.Timestamp).Format(time.RFC3339))
fmt.Println(cyan("Data:"))
var pretty bytes.Buffer
json.Indent(&pretty, stored.Data, "", " ")
fmt.Println(pretty.String())
}
func (c *KVSClient) handlePut(args []string) {
if len(args) < 2 {
if len(args) < 1 {
fmt.Println(red("Usage: put <key> <json-data>"))
fmt.Println("Or: put <key> (then paste JSON on next line)")
return
}
// Parse JSON data
var jsonStr string
if len(args) == 1 {
// Multiline mode - read from next input
fmt.Println(yellow("Enter JSON (Ctrl+D when done):"))
scanner := bufio.NewScanner(os.Stdin)
var lines []string
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
jsonStr = strings.Join(lines, "\n")
} else {
// Single line mode
jsonStr = strings.Join(args[1:], " ")
}
// Parse JSON data
var data json.RawMessage
if err := json.Unmarshal([]byte(args[1]), &data); err != nil {
fmt.Println(red("Invalid JSON:"), err)
if err := json.Unmarshal([]byte(jsonStr), &data); err != nil {
printJSONError(jsonStr, err)
return
}
@@ -295,6 +313,134 @@ func (c *KVSClient) handleDelete(args []string) {
}
}
// handleMeta is the dispatcher for "meta" sub-commands
func (c *KVSClient) handleMeta(args []string) {
if len(args) == 0 {
fmt.Println(red("Usage: meta <get|set> [options]"))
return
}
subCmd := args[0]
switch subCmd {
case "get":
c.handleMetaGet(args[1:])
case "set":
c.handleMetaSet(args[1:])
default:
fmt.Println(red("Unknown meta command:"), subCmd)
fmt.Println("Available commands: get, set")
}
}
// handleMetaGet fetches and displays metadata for a key
func (c *KVSClient) handleMetaGet(args []string) {
if len(args) < 1 {
fmt.Println(red("Usage: meta get <key>"))
return
}
key := args[0]
respBody, status, err := c.doRequest("GET", "/kv/"+key+"/metadata", nil)
if err != nil {
fmt.Println(red("Error:"), err)
return
}
if status == http.StatusNotFound {
fmt.Println(yellow("No metadata found for key:"), key)
return
}
if status != http.StatusOK {
fmt.Printf(red("Error getting metadata (status %d):\n"), status)
fmt.Println(string(respBody))
return
}
// Struct to parse the metadata response
type MetaResponse struct {
OwnerUUID string `json:"owner_uuid"`
GroupUUID string `json:"group_uuid"`
Permissions int `json:"permissions"`
CreatedAt int64 `json:"created_at"`
UpdatedAt int64 `json:"updated_at"`
}
var meta MetaResponse
if err := json.Unmarshal(respBody, &meta); err != nil {
fmt.Println(red("Error parsing metadata response:"), err)
return
}
fmt.Println(cyan("Metadata for key:"), key)
fmt.Printf(" Owner UUID: %s\n", meta.OwnerUUID)
fmt.Printf(" Group UUID: %s\n", meta.GroupUUID)
fmt.Printf(" Permissions: %d\n", meta.Permissions)
fmt.Printf(" Created At: %s\n", time.Unix(meta.CreatedAt, 0).Format(time.RFC3339))
fmt.Printf(" Updated At: %s\n", time.Unix(meta.UpdatedAt, 0).Format(time.RFC3339))
}
// handleMetaSet updates metadata for a key using flags
func (c *KVSClient) handleMetaSet(args []string) {
if len(args) < 3 {
fmt.Println(red("Usage: meta set <key> --owner <uuid> | --group <uuid> | --permissions <number>"))
return
}
key := args[0]
// Use a map to build the JSON payload for partial updates
payload := make(map[string]interface{})
// Parse command-line flags
for i := 1; i < len(args); i++ {
switch args[i] {
case "--owner":
if i+1 < len(args) {
payload["owner_uuid"] = args[i+1]
i++ // Skip the value
}
case "--group":
if i+1 < len(args) {
payload["group_uuid"] = args[i+1]
i++ // Skip the value
}
case "--permissions":
if i+1 < len(args) {
perms, err := strconv.Atoi(args[i+1])
if err != nil {
fmt.Println(red("Invalid permissions value:"), args[i+1])
return
}
payload["permissions"] = perms
i++ // Skip the value
}
}
}
if len(payload) == 0 {
fmt.Println(red("No valid metadata fields provided to set."))
return
}
respBody, status, err := c.doRequest("PUT", "/kv/"+key+"/metadata", payload)
if err != nil {
fmt.Println(red("Error:"), err)
return
}
if status != http.StatusOK {
fmt.Printf(red("Error setting metadata (status %d):\n"), status)
fmt.Println(string(respBody))
return
}
fmt.Println(green("Metadata updated successfully for key:"), key)
// The response body contains the full updated metadata, so we can print it for confirmation
var pretty bytes.Buffer
json.Indent(&pretty, respBody, "", " ")
fmt.Println(pretty.String())
}
func (c *KVSClient) handleMembers(args []string) {
respBody, status, err := c.doRequest("GET", "/members/", nil)
if err != nil {
@@ -390,6 +536,77 @@ func (c *KVSClient) handleUserGet(args []string) {
fmt.Printf(" Updated: %s\n", time.Unix(user.UpdatedAt, 0).Format(time.RFC3339))
}
func (c *KVSClient) handleUserCreate(args []string) {
if len(args) < 1 {
fmt.Println(red("Usage: user create <nickname>"))
return
}
// The request body requires a JSON object like {"nickname": "..."}
requestPayload := map[string]string{
"nickname": args[0],
}
respBody, status, err := c.doRequest("POST", "/api/users", requestPayload)
if err != nil {
fmt.Println(red("Error:"), err)
return
}
// The backend returns 200 OK on success
if status != http.StatusOK {
fmt.Printf(red("Error creating user (status %d):\n"), status)
fmt.Println(string(respBody))
return
}
// Parse the successful response to get the new user's UUID
var response struct {
UUID string `json:"uuid"`
}
if err := json.Unmarshal(respBody, &response); err != nil {
fmt.Println(red("Error parsing successful response:"), err)
return
}
fmt.Println(green("User created successfully"))
fmt.Println(cyan("UUID:"), response.UUID)
}
// printJSONError displays a user-friendly JSON parsing error with helpful hints
func printJSONError(input string, err error) {
fmt.Println(red("❌ Invalid JSON:"), err)
fmt.Println()
// Show what was received
if len(input) > 100 {
fmt.Println(yellow("Received:"), input[:100]+"...")
} else {
fmt.Println(yellow("Received:"), input)
}
fmt.Println()
// Provide helpful hints based on common errors
errMsg := err.Error()
if strings.Contains(errMsg, "looking for beginning of") ||
strings.Contains(errMsg, "invalid character") {
fmt.Println(cyan("💡 Tip:"), "JSON with spaces needs quotes:")
fmt.Println(green(" ✓ Correct:"), "put key '{\"hello\":\"world\"}'")
fmt.Println(green(" ✓ Correct:"), "put key '{\"a\":1,\"b\":2}'")
fmt.Println(green(" ✓ Compact:"), "put key {\"a\":1,\"b\":2} (no spaces)")
fmt.Println(red(" ✗ Wrong: "), "put key {\"a\": 1, \"b\": 2} (spaces without quotes)")
} else if strings.Contains(errMsg, "unexpected end of JSON") {
fmt.Println(cyan("💡 Tip:"), "JSON appears incomplete or cut off")
fmt.Println(green(" Check:"), "Are all brackets/braces matched? {}, []")
fmt.Println(green(" Check:"), "Did spaces split your JSON into multiple arguments?")
} else {
fmt.Println(cyan("💡 Tip:"), "Wrap JSON in quotes for complex values:")
fmt.Println(green(" Single quotes:"), "put key '{\"your\":\"json\"}'")
fmt.Println(green(" Double quotes:"), "put key \"{\\\"your\\\":\\\"json\\\"}\"")
}
}
func (c *KVSClient) handleHelp(args []string) {
help := `
KVS Interactive Shell - Available Commands:
@@ -407,6 +624,11 @@ Key-Value Operations:
put <key> <json> - Store JSON value at key
delete <key> - Delete key
Resource Metadata:
meta get <key> - Get metadata (owner, group) for a key
meta set <key> [flags] - Set metadata for a key
Flags: --owner <uuid>, --group <uuid>, --permissions <number>
Cluster Management:
members - List cluster members
health - Check service health
@@ -482,14 +704,25 @@ func main() {
client.handlePut(args)
case "delete":
client.handleDelete(args)
case "meta":
client.handleMeta(args)
case "members":
client.handleMembers(args)
case "health":
client.handleHealth(args)
case "user":
if len(args) > 0 && args[0] == "get" {
client.handleUserGet(args[1:])
if len(args) > 0 {
switch args[0] {
case "get":
client.handleUserGet(args[1:])
case "create":
client.handleUserCreate(args[1:])
default:
fmt.Println(red("Unknown user command:"), args[0])
fmt.Println("Type 'help' for available commands")
}
} else {
// This was the original behavior, which notes that listing users is not implemented.
client.handleUserList(args)
}
default:
@@ -511,6 +744,14 @@ var completer = readline.NewPrefixCompleter(
readline.PcItem("get"),
readline.PcItem("put"),
readline.PcItem("delete"),
readline.PcItem("meta",
readline.PcItem("get"),
readline.PcItem("set",
readline.PcItem("--owner"),
readline.PcItem("--group"),
readline.PcItem("--permissions"),
),
),
readline.PcItem("members"),
readline.PcItem("health"),
readline.PcItem("user",
@@ -527,25 +768,35 @@ var completer = readline.NewPrefixCompleter(
func parseCommand(line string) []string {
var parts []string
var current strings.Builder
inQuotes := false
var quoteChar rune = 0 // Use a rune to store the active quote character (' or ")
for i := 0; i < len(line); i++ {
ch := line[i]
switch ch {
case '"':
inQuotes = !inQuotes
case ' ', '\t':
if inQuotes {
current.WriteByte(ch)
} else if current.Len() > 0 {
for _, ch := range line {
switch {
// If we see a quote character
case ch == '\'' || ch == '"':
if quoteChar == 0 {
// Not currently inside a quoted string, so start one
quoteChar = ch
} else if quoteChar == ch {
// Inside a quoted string and found the matching quote, so end it
quoteChar = 0
} else {
// Inside a quoted string but found the *other* type of quote, treat it as a literal character
current.WriteRune(ch)
}
// If we see a space or tab and are NOT inside a quoted string
case (ch == ' ' || ch == '\t') && quoteChar == 0:
if current.Len() > 0 {
parts = append(parts, current.String())
current.Reset()
}
// Any other character
default:
current.WriteByte(ch)
current.WriteRune(ch)
}
}
// Add the last part if it exists
if current.Len() > 0 {
parts = append(parts, current.String())
}