Skip to content

Optimize Vector Math for Drift Detection #86

Description

@junyeong0619

Title: perf: Accelerate vector similarity calculations using Rust/SIMD

Body:

Problem
Vector operations such as cosine similarity and Euclidean distance calculation in src/vectorwave/database/db_search.py (check_semantic_drift) and src/vectorwave/utils/replayer_semantic.py are currently implemented in pure Python. These operations become a bottleneck when processing high-dimensional vectors (e.g., 1536 dims) or large batches of logs.

Proposed Solution
Offload heavy mathematical computations to Rust.

Implement vector distance functions using Rust libraries like ndarray or utilize SIMD instructions for maximum performance.

Expose these functions to Python via PyO3.

Expected Outcome
Faster execution of Semantic Drift detection and Semantic Replay tests.

Reduced CPU usage during vector comparisons.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions