feature(ehi): Adding high volume EHI example#540
feature(ehi): Adding high volume EHI example#540fivetran-sahilkhirwal wants to merge 11 commits intomainfrom
Conversation
🧹 Python Code Quality Check✅ No issues found in Python Files. This comment is auto-updated with every commit. |
There was a problem hiding this comment.
Pull request overview
This pull request adds two new EHI (Microsoft SQL Server / Caboodle-style) Connector SDK examples—one “simple” connector and one “high volume” connector—and updates the repository’s top-level connector index to reference both variants.
Changes:
- Update the top-level
README.mdconnector list to point to two EHI example variants. - Add a
simple_ehiconnector example usingpython-tdswith basic schema discovery, batch fetching, and incremental sync based on a modified-date column. - Add a
high_volume_ehiconnector example usingpyodbcwith parallel schema discovery, keyset/offset pagination readers, and threaded table syncing.
Reviewed changes
Copilot reviewed 9 out of 13 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Splits the EHI entry into simple_ehi and high_volume_ehi references. |
| connectors/ehi/simple_ehi/requirements.txt | Adds the python-tds dependency for the simple connector. |
| connectors/ehi/simple_ehi/README.md | Adds documentation for the simple EHI connector example. |
| connectors/ehi/simple_ehi/connector.py | Implements the simple EHI connector (schema discovery + sync loop + state/checkpointing). |
| connectors/ehi/simple_ehi/configuration.json | Adds placeholder configuration for the simple connector. |
| connectors/ehi/high_volume_ehi/requirements.txt | Adds dependencies for the high-volume connector (currently includes requests). |
| connectors/ehi/high_volume_ehi/README.md | Adds documentation for the high-volume EHI connector example. |
| connectors/ehi/high_volume_ehi/readers.py | Adds keyset/offset/incremental reader implementations. |
| connectors/ehi/high_volume_ehi/models.py | Adds schema detection + replication key detection logic. |
| connectors/ehi/high_volume_ehi/constants.py | Adds performance/retry/tuning constants. |
| connectors/ehi/high_volume_ehi/connector.py | Implements the high-volume connector orchestration and threaded table syncing. |
| connectors/ehi/high_volume_ehi/configuration.json | Adds placeholder configuration for the high-volume connector. |
| connectors/ehi/high_volume_ehi/client.py | Adds a pyodbc connection wrapper + pooled acquire/retry behavior. |
|
@codex review |
481c421 to
c43b8f6
Compare
fivetran-JenasVimal
left a comment
There was a problem hiding this comment.
Please address the comments
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6c6bc312b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fivetran-satvikpatil
left a comment
There was a problem hiding this comment.
Please address one comment, else LGTM. Also please add testing details again after the changes.
1eac13e
Jira ticket
Closes https://fivetran.atlassian.net/browse/RD-1172756
Description of Change
Testing
Checklist
Some tips and links to help validate your PR:
fivetran debugcommand.