Skip to content

feat(providers): validate retriever settings keys to catch typos early (#116)#172

Merged
himanshu231204 merged 2 commits into
OpenAgentHQ:mainfrom
Nitjsefnie:feat/116-retriever-settings-validation
Jul 17, 2026
Merged

feat(providers): validate retriever settings keys to catch typos early (#116)#172
himanshu231204 merged 2 commits into
OpenAgentHQ:mainfrom
Nitjsefnie:feat/116-retriever-settings-validation

Conversation

@Nitjsefnie

Copy link
Copy Markdown
Contributor

Fixes #116.

Adds early validation of the free-form retriever settings dict so a typo like collectoin_name is caught at provider-build time (with a "did you mean 'collection_name'?" suggestion) instead of either being silently swallowed by a **kwargs sink (qdrant) or blowing up with an opaque TypeError deep in a constructor (chroma).

Design — I matched your existing precedent rather than inventing one. config/loader.py (~L139) already handles unknown/misspelled metric names by dropping them and emitting a logging.warning naming them, not by raising. That's the same problem shape (a typo in a user-supplied free-form name field), so the new _validation.py helper does the same: warn naming the bad key + a difflib suggestion, then drop it. Providers opt in by declaring a SETTINGS_KEYS attribute, so it's backward-compatible and, per the issue's "start with one or two," only chroma and qdrant opt in for now (trivially extensible to the rest).

Allowed keys are derived from each retriever's actual constructor, not the issue text:

  • chroma → {collection_name, persist_directory, distance_fn}
  • qdrant → {collection_name, embedder, url, api_key, prefer_grpc, distance}

One divergence worth flagging: the issue lists k as a chroma setting, but k is a retrieve() argument, not a constructor setting — so a config with settings.k will now warn. I trusted the code over the issue here; say the word if you actually want k accepted as a settings key.

One decision I want to leave to you (one-line change either way — the helper already returns the unknown-key list): I chose warn-and-drop to match your metrics handling, but it does change chroma's failure mode from a hard TypeError to a warning + falling back to the default collection. If you'd rather a typo hard-fail (raise ConfigurationError) so a mistyped collection_name can't silently evaluate against the wrong collection, I'm happy to flip it — arguably closer to the issue's "discover early" motivation. I went with the precedent-consistent choice by default.

Tests (tests/unit/test_providers/test_retriever_settings_validation.py): unknown key caught with the bad key + suggestion named (both at the helper level and through the full get_retriever factory path), known keys pass clean, and the no-close-match case. Mutation-verified: neutering the factory validation fails the typo test; restored, it passes. uv run pytest tests/unit is green aside from one pre-existing, unrelated BERTScore float-precision failure in bertscore.py (reproduces on clean main, untouched here); ruff/mypy clean on the new files.

Disclosure: this contribution was made with AI assistance (Claude), reviewed and mutation-tested before submission.

Nitjsefnie and others added 2 commits July 17, 2026 05:29
Retriever providers accept a free-form `settings` dict, so a typo like
`collectoin_name` was either silently swallowed by a provider's `**kwargs`
(qdrant) or raised an opaque `TypeError` deep in a constructor (chroma) --
and only at runtime.

Add a validation helper that, at provider-build time, warns on unknown
setting keys and suggests the closest known key (`difflib`), then drops the
key. This mirrors how unknown/misspelled metric names are surfaced in
`config.loader` (warn + drop, not hard-fail). Providers opt in by declaring
a `SETTINGS_KEYS` set; chroma and qdrant declare theirs, derived from their
actual constructor parameters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@himanshu231204
himanshu231204 merged commit 51c79f3 into OpenAgentHQ:main Jul 17, 2026
9 checks passed
@github-actions

Copy link
Copy Markdown

🎉 Congratulations @Nitjsefnie!

Your pull request has been successfully merged into main. 🚀

Thank you for contributing to OpenAgentHQ and helping improve the project.

We truly appreciate your contribution and hope to see you back with more amazing PRs!

Happy Open Sourcing! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add validation for retriever settings field to catch typos early

2 participants