feat(providers): validate retriever settings keys to catch typos early (#116)#172
Merged
himanshu231204 merged 2 commits intoJul 17, 2026
Conversation
Retriever providers accept a free-form `settings` dict, so a typo like `collectoin_name` was either silently swallowed by a provider's `**kwargs` (qdrant) or raised an opaque `TypeError` deep in a constructor (chroma) -- and only at runtime. Add a validation helper that, at provider-build time, warns on unknown setting keys and suggests the closest known key (`difflib`), then drops the key. This mirrors how unknown/misspelled metric names are surfaced in `config.loader` (warn + drop, not hard-fail). Providers opt in by declaring a `SETTINGS_KEYS` set; chroma and qdrant declare theirs, derived from their actual constructor parameters. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
🎉 Congratulations @Nitjsefnie! Your pull request has been successfully merged into main. 🚀 Thank you for contributing to OpenAgentHQ and helping improve the project. We truly appreciate your contribution and hope to see you back with more amazing PRs! Happy Open Sourcing! ❤️ |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #116.
Adds early validation of the free-form retriever
settingsdict so a typo likecollectoin_nameis caught at provider-build time (with a "did you mean 'collection_name'?" suggestion) instead of either being silently swallowed by a**kwargssink (qdrant) or blowing up with an opaqueTypeErrordeep in a constructor (chroma).Design — I matched your existing precedent rather than inventing one.
config/loader.py(~L139) already handles unknown/misspelled metric names by dropping them and emitting alogging.warningnaming them, not by raising. That's the same problem shape (a typo in a user-supplied free-form name field), so the new_validation.pyhelper does the same: warn naming the bad key + adifflibsuggestion, then drop it. Providers opt in by declaring aSETTINGS_KEYSattribute, so it's backward-compatible and, per the issue's "start with one or two," onlychromaandqdrantopt in for now (trivially extensible to the rest).Allowed keys are derived from each retriever's actual constructor, not the issue text:
{collection_name, persist_directory, distance_fn}{collection_name, embedder, url, api_key, prefer_grpc, distance}One divergence worth flagging: the issue lists
kas a chroma setting, butkis aretrieve()argument, not a constructor setting — so a config withsettings.kwill now warn. I trusted the code over the issue here; say the word if you actually wantkaccepted as a settings key.One decision I want to leave to you (one-line change either way — the helper already returns the unknown-key list): I chose warn-and-drop to match your metrics handling, but it does change chroma's failure mode from a hard
TypeErrorto a warning + falling back to the default collection. If you'd rather a typo hard-fail (raise ConfigurationError) so a mistypedcollection_namecan't silently evaluate against the wrong collection, I'm happy to flip it — arguably closer to the issue's "discover early" motivation. I went with the precedent-consistent choice by default.Tests (
tests/unit/test_providers/test_retriever_settings_validation.py): unknown key caught with the bad key + suggestion named (both at the helper level and through the fullget_retrieverfactory path), known keys pass clean, and the no-close-match case. Mutation-verified: neutering the factory validation fails the typo test; restored, it passes.uv run pytest tests/unitis green aside from one pre-existing, unrelated BERTScore float-precision failure inbertscore.py(reproduces on cleanmain, untouched here); ruff/mypy clean on the new files.Disclosure: this contribution was made with AI assistance (Claude), reviewed and mutation-tested before submission.