Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
485 lines (356 loc) · 9.45 KB

CONTRIBUTING.md

File metadata and controls

485 lines (356 loc) · 9.45 KB

Contributing to SochDB Node.js SDK

Thank you for your interest in contributing to the SochDB Node.js SDK! This guide provides all the information you need to build, test, and contribute to the project.

Table of Contents

Development Setup

Prerequisites

  • Node.js 18+ or Bun 1.0+
  • TypeScript 5.0+
  • Rust toolchain (for building server)
  • Protocol Buffers compiler (protoc)
  • Git

Clone and Install

# Clone the repository
git clone https://github.com/sochdb/sochdb-nodejs-sdk.git
cd sochdb-nodejs-sdk

# Install dependencies
npm install

# Build TypeScript
npm run build

# Run tests
npm test

With Bun

# Install with Bun
bun install

# Build
bun run build

# Test
bun test

Building from Source

TypeScript Compilation

# Compile TypeScript to JavaScript
npm run build

# Watch mode for development
npm run build:watch

With Protocol Buffers

If you need to regenerate gRPC stubs:

# Install protoc-gen-ts
npm install -g protoc-gen-ts

# Generate from proto files
cd sochdb/proto
protoc --ts_out=. --grpc_out=. *.proto

Running Tests

Unit Tests

# Run all tests
npm test

# Run with coverage
npm run test:coverage

# Run specific test file
npm test -- grpc-client.test.ts

Integration Tests

# Start SochDB server first
cd sochdb
cargo run -p sochdb-grpc

# In another terminal, run integration tests
cd sochdb-nodejs-sdk
npm run test:integration

Type Checking

# Run TypeScript type checker
npm run typecheck

# Watch mode
npm run typecheck:watch

Server Setup for Development

Starting the Server

# Development mode
cd sochdb
cargo run -p sochdb-grpc

# Production mode (optimized)
cargo build --release -p sochdb-grpc
./target/release/sochdb-grpc --host 0.0.0.0 --port 50051

Server Configuration

The server runs all business logic including:

  • ✅ HNSW vector indexing (15x faster than ChromaDB)
  • ✅ SQL query parsing and execution
  • ✅ Graph traversal algorithms
  • ✅ Policy evaluation
  • ✅ Multi-tenant namespace isolation
  • ✅ Collection management

Configuration File

Create sochdb-server-config.toml:

[server]
host = "0.0.0.0"
port = 50051

[storage]
data_dir = "./data"

[logging]
level = "info"

Code Style

TypeScript

We follow TypeScript best practices:

# Format code
npm run format

# Lint
npm run lint

# Fix lint issues
npm run lint:fix

ESLint Configuration

{
  "extends": [
    "eslint:recommended",
    "plugin:@typescript-eslint/recommended"
  ],
  "rules": {
    "no-console": "warn",
    "@typescript-eslint/explicit-function-return-type": "error"
  }
}

Commit Messages

Follow conventional commits:

feat: Add temporal graph support
fix: Handle connection timeout
docs: Update API reference
test: Add integration tests for graphs

Code Review Checklist

  • All tests pass
  • Code follows TypeScript style guidelines
  • Documentation updated (TSDoc)
  • Examples added/updated if needed
  • No breaking changes (or documented in CHANGELOG)
  • Type definitions exported correctly

Pull Request Process

  1. Fork and Clone

    git clone https://github.com/YOUR_USERNAME/sochdb-nodejs-sdk.git
cd sochdb-nodejs-sdk

  2. Create Feature Branch

    git checkout -b feature/your-feature-name

  3. Make Changes

    • Write code
    • Add tests
    • Update documentation

  4. Test Locally

    npm test
npm run lint
npm run typecheck

  5. Commit and Push

    git add .
git commit -m "feat: Your feature description"
git push origin feature/your-feature-name

  6. Create Pull Request

    • Go to GitHub
    • Create PR from your branch
    • Fill out PR template
    • Wait for review

Architecture Overview

Thin Client Architecture

┌────────────────────────────────────────────────┐
│         Rust Server (sochdb-grpc)              │
├────────────────────────────────────────────────┤
│  • All business logic (Graph, Policy, Search)  │
│  • Vector operations (HNSW)                    │
│  • SQL parsing & execution                     │
│  • Collections & Namespaces                    │
│  • Single source of truth                      │
└────────────────────────────────────────────────┘
                       │ gRPC/IPC
                       ▼
            ┌─────────────────────┐
            │   Node.js SDK       │
            │   (~1,282 LOC)      │
            ├─────────────────────┤
            │ • Transport layer   │
            │ • Type definitions  │
            │ • Zero logic        │
            └─────────────────────┘

Key Components

grpc-client.ts

  • SochDBClient class for gRPC
  • All server operations
  • Connection management
  • Error handling

format.ts (188 lines)

  • WireFormat enum (TOON, JSON, Columnar)
  • ContextFormat enum (TOON, JSON, Markdown)
  • FormatCapabilities utilities

types.ts

  • TypeScript type definitions
  • SearchResult, Document, GraphNode, GraphEdge
  • Request/response interfaces

Comparison with Old Architecture

Feature Old (Fat Client) New (Thin Client)
SDK Size 5,038 LOC 1,282 LOC (-75%)
Business Logic In SDK (TypeScript) In Server (Rust)
Bug Fixes Per language Once in server
Semantic Drift High risk Zero risk
Performance FFI overhead Network call
Maintenance 3x effort 1x effort

Migration Guide

From v0.3.3 to v0.3.4

Key Changes:

  • Removed embedded Database class
  • All operations now go through SochDBClient
  • Server must be running for all operations
  • FFI bindings removed

Old Code:

import { Database } from '@sochdb/sochdb';

const db = await Database.open('./data');
await db.put(Buffer.from('key'), Buffer.from('value'));
await db.close();

New Code:

import { SochDBClient } from '@sochdb/sochdb';

// Start server first: cargo run -p sochdb-grpc
const client = new SochDBClient({ address: 'localhost:50051' });
await client.putKv('key', Buffer.from('value'));
await client.close();

Migration Checklist:

  • Start SochDB server (cargo run -p sochdb-grpc)
  • Replace Database.open() with new SochDBClient()
  • Update connection strings to point to server
  • Add error handling for all operations
  • Remove any FFI-related code

Release Process

Version Bumping

# Update version in package.json
npm version patch  # or minor, major

# Update CHANGELOG.md
vim CHANGELOG.md

Building Distribution

# Clean build
npm run clean
npm run build

# Check bundle size
npm run build:analyze

Publishing to npm

# Test locally
npm pack

# Dry run
npm publish --dry-run

# Publish (requires npm login)
npm publish --access public

Testing Checklist

Before submitting a PR, ensure:

  • All unit tests pass: npm test
  • Integration tests pass (with server): npm run test:integration
  • Type checking passes: npm run typecheck
  • Linting passes: npm run lint
  • Code formatted: npm run format
  • Documentation updated (TSDoc comments)
  • CHANGELOG.md updated
  • No console.log statements in production code

Performance Testing

Benchmarks

# Run benchmarks
npm run benchmark

# Run specific benchmark
npm run benchmark -- vector-search

Load Testing

# Start server
cd sochdb
cargo run -p sochdb-grpc --release

# Run load test
cd sochdb-nodejs-sdk
npm run test:load

Documentation

TSDoc Comments

Use TSDoc for all public APIs:

/**
 * Search for vectors in an index
 * 
 * @param indexName - Name of the vector index
 * @param query - Query vector
 * @param k - Number of results to return
 * @returns Array of search results with IDs and distances
 * 
 * @example
 * ```typescript
 * const results = await client.search('embeddings', queryVector, 10);
 * console.log(results);
 * ```
 */
async search(indexName: string, query: number[], k: number): Promise<SearchResult[]>

Generate Documentation

# Generate TypeDoc documentation
npm run docs

# View locally
open docs/index.html

Getting Help

License

By contributing to SochDB Node.js SDK, you agree that your contributions will be licensed under the Apache License 2.0.