Skip to content

split pinecone-client.ts with 3 sub files#86

Merged
wpak-ai merged 6 commits into
cppalliance:mainfrom
jonathanMLDev:split-pinecone-client-rebased-branch
May 14, 2026
Merged

split pinecone-client.ts with 3 sub files#86
wpak-ai merged 6 commits into
cppalliance:mainfrom
jonathanMLDev:split-pinecone-client-rebased-branch

Conversation

@jonathanMLDev
Copy link
Copy Markdown
Collaborator

@jonathanMLDev jonathanMLDev commented May 14, 2026

Summary

src/pinecone-client.ts had grown to ~605 lines, mixing lazy SDK init, namespace discovery, single-index search, hybrid Promise.allSettled orchestration, dense/sparse merge, reranking, keywordSearch, and count. A change to any one of those required scrolling past hundreds of lines of unrelated code, and the two as unknown SDK-bypass casts (one undocumented, one with a six-line rationale) were buried inside the file.

Changes

  • New src/pinecone/indexes.ts (177 lines) — PineconeIndexSession owns the lazy Pinecone client and dense/sparse SearchableIndex handles. Holds the previously inline pc.index(name) as unknown as SearchableIndex casts and the inferMetadataFieldType helper used by listNamespacesWithMetadata.
  • New src/pinecone/search.ts (176 lines) — pure functions: searchIndex (preferred Pinecone search API + backward-compatible searchRecords fallback), mergeResults (dedupe by _id, keep higher score, sort desc), plus sliceMergedHitsToSearchResults, mapSparseHitsToSearchResults, and countUniqueDocumentsFromHits extracted from the facade so each module stays under the 250-line budget.
  • New src/pinecone/rerank.ts (59 lines) — rerankResults(pc, rerankModel, query, results, topN) with the original six-line as unknown rationale comment preserved verbatim, plus the unranked-slice fallback on inference error.
  • Refactored src/pinecone-client.ts (~211 lines after prettier) — composes a PineconeIndexSession and delegates. Hybrid Promise.allSettled, error logging, merge, rerank/no-rerank branch, and the chunk_text-injection rule for reranked queries stay in the facade.
  • as unknown casts moved verbatim with their existing comments. as unknown no longer appears in pinecone-client.ts; both sites are easy to find in the new modules.
  • ensureIndexes and searchIndex remain private async instance methods that delegate to the modules, so src/pinecone-client.test.ts continues to patch them via PineconeClientTestDouble without any test edits.
  • No public API changes. All callers (server/tools/{count,query,keyword-search,query-documents,guided-query}-tool.ts, server/namespaces-cache.ts, server/client-context.ts, index.ts, scripts/test-search.ts) keep importing PineconeClient from ./pinecone-client.js.

Related Issue

close #82

Summary by CodeRabbit

Release Notes

  • Breaking Changes

    • listNamespacesFromKeywordIndex() return type updated to Array<{ namespace: string; recordCount: number }>
  • Refactor

    • Reorganized Pinecone client internals to use specialized, modular components for search operations, reranking, and index management
  • Tests

    • Expanded test coverage for search helpers, reranking functionality, and index session management

Review Change Stack

# Conflicts:
#	src/pinecone-client.ts
@jonathanMLDev jonathanMLDev self-assigned this May 14, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Warning

Rate limit exceeded

@jonathanMLDev has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 17 minutes and 33 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 38f383e6-6b7b-42b0-84b8-ec6cde2ca679

📥 Commits

Reviewing files that changed from the base of the PR and between 81924ef and f0ce85a.

📒 Files selected for processing (4)
  • src/pinecone-client.test.ts
  • src/pinecone-client.ts
  • src/pinecone/indexes.test.ts
  • src/pinecone/indexes.ts
📝 Walkthrough

Walkthrough

This PR refactors PineconeClient from a 605-line monolith into focused modules following the facade pattern. PineconeIndexSession manages lazy initialization and index access; new search.ts module exports query execution and result post-processing helpers; rerank.ts encapsulates reranking logic. The client facade delegates to these modules, preserving the public API. Comprehensive test coverage is added for each new module.

Changes

Pinecone Client Monolith Refactoring

Layer / File(s) Summary
Index Management & Initialization
src/pinecone/indexes.ts, src/pinecone/indexes.test.ts
New PineconeIndexSession class manages lazy SDK initialization, cached index handles, namespace listing with record counts, metadata type inference from sampled records, and index health checks with graceful error aggregation. Tests cover success/failure paths, metadata sampling, and index reachability verification.
Search Operations & Result Processing
src/pinecone/search.ts, src/pinecone/search.test.ts
New module exports searchIndex (query with fallback compatibility), mergeResults (deduplication by score), hit-to-result conversion helpers, sparse hit mapping, and countUniqueDocumentsFromHits with identifier priority fallback and truncation detection. Tests verify dense/sparse fallbacks, merge precedence, and count deduplication.
Reranking with Fallback
src/pinecone/rerank.ts, src/pinecone/rerank.test.ts
New module exports rerankResults function invoking inference.rerank on chunk_text, mapping results to SearchResult with reranked: true, and falling back to unreranked top-N with reranked: false on error. Tests cover empty input, successful rerank response mapping, and error-path fallback.
Client Facade & Test Updates
src/pinecone-client.ts, src/pinecone-client.test.ts
PineconeClient refactored to delegate initialization, namespace listing, and health checks to PineconeIndexSession; query() and keywordSearch() now call imported search helpers; count() delegates to countUniqueDocumentsFromHits. Client test suite updated with stubPineconeClient helper, environment cleanup hooks, and extended coverage for fallback behaviors, reranking paths, sparse indexing, and sparse-to-result mapping.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • wpak-ai

Poem

🐰 A monolith once sprawled five hundred lines deep,
Now split into modules, where responsibilities creep.
Indexes manage themselves, searches flow clean,
Reranking just reranks—the cleanest you've seen!
The facade holds the seams, while tests guard the dream. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: splitting a large monolithic file into three smaller focused modules.
Linked Issues check ✅ Passed All coding objectives from #82 are met: PineconeClient split into 3+ modules (indexes.ts, search.ts, rerank.ts), all modules under 250 lines, public API unchanged, as unknown casts preserved with comments, tests refactored to pass, and no circular dependencies introduced.
Out of Scope Changes check ✅ Passed All changes directly support the split requirement: refactored pinecone-client.ts, added new modules for indexes/search/reranking, updated tests, and improved module organization per #82 specifications.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

❌ Patch coverage is 82.90598% with 40 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@335bda1). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/pinecone/indexes.ts 71.68% 32 Missing ⚠️
src/pinecone-client.ts 70.00% 6 Missing ⚠️
src/pinecone/search.ts 97.53% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #86   +/-   ##
=======================================
  Coverage        ?   66.01%           
=======================================
  Files           ?       33           
  Lines           ?     1121           
  Branches        ?      366           
=======================================
  Hits            ?      740           
  Misses          ?      381           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
src/pinecone-client.test.ts (1)

34-37: 💤 Low value

afterEach doesn't clean PINECONE_TOP_K.

The constructor also reads process.env['PINECONE_TOP_K'] (pinecone-client.ts Line 44). Any future test that sets it will leak across tests. Cheap insurance to add it here alongside the other two.

♻️ Proposed cleanup
   afterEach(() => {
     delete process.env['PINECONE_INDEX_NAME'];
     delete process.env['PINECONE_RERANK_MODEL'];
+    delete process.env['PINECONE_TOP_K'];
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/pinecone-client.test.ts` around lines 34 - 37, The test teardown in
afterEach currently deletes PINECONE_INDEX_NAME and PINECONE_RERANK_MODEL but
misses PINECONE_TOP_K, which the PineconeClient constructor reads from
process.env; update the afterEach in src/pinecone-client.test.ts to also delete
process.env['PINECONE_TOP_K'] so tests don't leak environment state between runs
(look for the afterEach block and add deletion of PINECONE_TOP_K).
src/pinecone-client.ts (1)

152-163: 💤 Low value

Encapsulation concern: Consider wrapping reranking in a session method instead of exposing the raw client.

ensureClient() is confirmed as a public method on PineconeIndexSession (at src/pinecone/indexes.ts:45) that returns a Pinecone SDK instance. The method is correctly used by rerankResultsImpl, which calls .inference.rerank on the client (as seen in `src/pinecone/rerank.ts:24).

However, exposing the raw Pinecone client through a public method widens the encapsulation boundary that the session facade was intended to maintain. Consider refactoring rerankResultsImpl to accept the session and call a narrower reranking method (e.g., indexSession.rerank(...)) instead, keeping SDK access fully encapsulated within the session class.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/pinecone-client.ts` around lines 152 - 163, The code currently passes a
raw SDK client into rerankResultsImpl by calling
this.indexSession.ensureClient(), which breaks the PineconeIndexSession
encapsulation; add a narrow rerank method on PineconeIndexSession (e.g.,
PineconeIndexSession.rerank or similar) that internally calls the SDK's
inference.rerank, change rerankResultsImpl to accept a PineconeIndexSession (or
call the new session.rerank) instead of a Pinecone client, and update the call
site in pinecone-client.ts to pass this.indexSession (and invoke the
session.rerank wrapper) so the SDK client never leaves the session class and
rerank logic remains encapsulated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/pinecone-client.test.ts`:
- Line 1: Prettier style violations in src/pinecone-client.test.ts are causing
CI failures; run the code formatter on this file (e.g., run `prettier --write
src/pinecone-client.test.ts`) to fix formatting, then stage the updated file and
re-run tests; ensure imports at the top (describe, it, expect, beforeEach,
afterEach, vi) and the rest of the test file adhere to your project's
Prettier/ESLint rules before committing.

In `@src/pinecone-client.ts`:
- Around line 43-45: The assignment to this.defaultTopK should use nullish
coalescing and a radix-aware parse with NaN guarding: read the env var via
process.env['PINECONE_TOP_K'] ?? String(DEFAULT_TOP_K), call parseInt(..., 10),
then check Number.isFinite and fallback to DEFAULT_TOP_K if parseInt returns NaN
or a non-finite value; update the initialization in the constructor where
this.defaultTopK is set and ensure clampTopK will receive a valid finite integer
(refer to this.defaultTopK, clampTopK, PINECONE_TOP_K, and DEFAULT_TOP_K).
- Around line 73-78: The method listNamespacesFromKeywordIndex currently returns
Promise<Array<{ namespace: string; recordCount: number }>> which breaks the API
contract; change its return type to KeywordIndexNamespacesResult and update the
corresponding implementation in src/pinecone/indexes.ts so that
indexSession.listNamespacesFromKeywordIndex also returns the discriminated union
{ ok: true; namespaces: Array<{ namespace: string; recordCount: number }> } | {
ok: false; error: string }; ensure both listNamespacesFromKeywordIndex (in
pinecone-client.ts) and the implementation in indexes.ts preserve the
.ok/.namespaces or .ok/.error shape expected by scripts/test-search.ts.

In `@src/pinecone/indexes.test.ts`:
- Around line 1-164: The test file formatting fails Prettier; run the formatter
on the file (e.g., prettier --write src/pinecone/indexes.test.ts) or apply your
project's configured formatter to fix whitespace/quote/indentation issues so CI
passes; ensure you don't change test logic around the
PineconeIndexSessionTestDouble and ThrowingEnsureSession classes or the
listNamespaces... / checkIndexes test cases—only update formatting.

---

Nitpick comments:
In `@src/pinecone-client.test.ts`:
- Around line 34-37: The test teardown in afterEach currently deletes
PINECONE_INDEX_NAME and PINECONE_RERANK_MODEL but misses PINECONE_TOP_K, which
the PineconeClient constructor reads from process.env; update the afterEach in
src/pinecone-client.test.ts to also delete process.env['PINECONE_TOP_K'] so
tests don't leak environment state between runs (look for the afterEach block
and add deletion of PINECONE_TOP_K).

In `@src/pinecone-client.ts`:
- Around line 152-163: The code currently passes a raw SDK client into
rerankResultsImpl by calling this.indexSession.ensureClient(), which breaks the
PineconeIndexSession encapsulation; add a narrow rerank method on
PineconeIndexSession (e.g., PineconeIndexSession.rerank or similar) that
internally calls the SDK's inference.rerank, change rerankResultsImpl to accept
a PineconeIndexSession (or call the new session.rerank) instead of a Pinecone
client, and update the call site in pinecone-client.ts to pass this.indexSession
(and invoke the session.rerank wrapper) so the SDK client never leaves the
session class and rerank logic remains encapsulated.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0cd654d9-49b3-4cfc-961d-c430f4a74d4e

📥 Commits

Reviewing files that changed from the base of the PR and between 335bda1 and 81924ef.

📒 Files selected for processing (8)
  • src/pinecone-client.test.ts
  • src/pinecone-client.ts
  • src/pinecone/indexes.test.ts
  • src/pinecone/indexes.ts
  • src/pinecone/rerank.test.ts
  • src/pinecone/rerank.ts
  • src/pinecone/search.test.ts
  • src/pinecone/search.ts

Comment thread src/pinecone-client.test.ts
Comment thread src/pinecone-client.ts
Comment thread src/pinecone-client.ts
Comment thread src/pinecone/indexes.test.ts
@jonathanMLDev jonathanMLDev requested a review from wpak-ai May 14, 2026 18:14
@wpak-ai wpak-ai merged commit 32c5ac1 into cppalliance:main May 14, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split pinecone-client.ts Monolith

2 participants