#7 Add _ls and _tree Endpoints for Hierarchical Key Listing Using Merkle Tree
-----------------------------------------

KVS supports hierarchical keys (e.g., /home/room/closet/socks), which is great for organizing data like a file system. However, there's currently no built-in way for clients to discover or list subkeys under a given prefix/path. This makes it hard to build intuitive tools or UIs that need to navigate the keyspace, such as a web-based explorer or CLI client.

Add two new read-only endpoints that leverage the existing Merkle tree infrastructure for efficient prefix-based key listing. This aligns with KVS's modular design, eventual consistency model, and Merkle-based sync (no need for full DB scans—traverse the tree to identify relevant leaf nodes in O(log N) time).
Proposed Endpoints

    Direct Children Listing (_ls or _list):
        Endpoint: GET /kv/{path}/_ls (or GET /kv/{path}/_list for clarity).
        Purpose: Returns a sorted list of direct subkeys under the given path/prefix (non-recursive).
        Query Params (optional):
            limit: Max number of keys to return (default: 100, max: 1000).
            include_metadata: If true, include basic metadata like timestamps (default: false).
        Response (JSON):

        {
          "path": "/home/room",
          "children": [
            { "subkey": "closet", "timestamp": 1695280000000 },
            { "subkey": "bed", "timestamp": 1695279000000 }
          ],
          "total": 2,
          "truncated": false
        }

    Behavior:
        Treat {path} as a prefix (e.g., /home/room/ → keys starting with /home/room/ but not /home/room/sub/).
        Use the Merkle tree to find leaf nodes in the prefix range [prefix, prefix~] (where ~ is the next lexicographical prefix).
        Skip index keys (e.g., _ts:*).
        Respect auth: Use existing middleware (e.g., read scope if auth_enabled: true).
        In read-only/syncing modes: Allow if not modifying data.

Recursive Tree View (_tree):

    Endpoint: GET /kv/{path}/_tree.
    Purpose: Returns a recursive tree structure of all subkeys under the given path (depth-first or breadth-first, configurable).
    Query Params (optional):
        depth: Max recursion depth (default: unlimited, but suggest 5 for safety).
        limit: Max total keys (default: 500, max: 5000).
        include_metadata: Include timestamps/UUIDs (default: false).
        format: json (default) or nested (tree-like JSON).
    Response (JSON, nested format):

    {
      "path": "/home/room",
      "children": [
        {
          "subkey": "closet",
          "children": [
            { "subkey": "socks", "timestamp": 1695281000000 }
          ],
          "timestamp": 1695280000000
        },
        {
          "subkey": "bed",
          "timestamp": 1695279000000
        }
      ],
      "total": 3,
      "truncated": false
    }

        Behavior:
            Build on _ls logic: Recursively query sub-prefixes via Merkle tree traversal.
            Prune at depth or limit to avoid overload.
            Same auth and mode rules as _ls.

Integration with Existing Systems

    Merkle Tree Usage: Extend cluster/merkle.go (e.g., add GetKeysInRange(startKey, endKey) []string method) to traverse nodes covering the prefix range without fetching full values. Reuse buildMerkleTreeFromPairs and filterPairsByRange from handlers.go.
    Range Query Reuse: Build on existing KVRangeRequest/KVRangeResponse in types.go and getKVRangeHandler (strip values to return just keys for efficiency).
    Auth & Permissions: Apply via authService.Middleware (e.g., read scope). Respect allow_anonymous_read.
    Config Toggle: Add key_listing_enabled: true to types.Config for optional disable (e.g., for security in public clusters).
    Distributed Consistency: Since Merkle trees are synced, listings will be eventually consistent across nodes. Add a consistent: true query param to force a quick Merkle refresh if needed.


#12 Missing API Endpoints for Resource Metadata Management (Ownership & Permissions)
-----------------------------------------

The KVS system currently lacks API endpoints to manage ResourceMetadata for key-value paths (/kv/{path}). While the AuthService and permissions.go implement robust permission checking based on OwnerUUID, GroupUUID, and Permissions, there are no exposed routes to:

Assign group-level permissions: Users cannot grant read/write access to specific groups for a given key-value path.

Change resource ownership: Users cannot transfer ownership of a key-value entry to another user.

This prevents administrators from fully leveraging the existing authentication and authorization framework for fine-grained access control over stored data.

Impact:

Limited administrative control over data access.

Inability to implement granular, group-based access policies for KV data.

Difficulty in reassigning data ownership when users or roles change.

Proposed Solution:
Implement new API endpoints (e.g., /kv/{path}/metadata) to allow authenticated and authorized users to:

Set/update the OwnerUUID for a given path.

Set/update the GroupUUID for a given path.

Set/update the Permissions bitmask for a given path.

Relevant Files:

server/routes.go (for new API routes)

server/handlers.go (for implementing new handlers)

auth/auth.go (for AuthService methods to interact with ResourceMetadata)

auth/permissions.go (existing logic for permission checks)

types/types.go (for ResourceMetadata structure)