Skip to content

Conversation

@JackieTien97
Copy link
Contributor

This pull request introduces a new streaming DataFrame interface to the IoTDB Python client, enabling users to efficiently fetch large query results in manageable chunks. The main changes add support for iterating over result sets by DataFrame, buffering data internally, and updating example scripts to demonstrate the new API.

Streaming DataFrame API:

  • Added has_next_df() and next_df() methods to SessionDataSet to allow users to check for and retrieve the next chunk of results as a DataFrame.
  • Implemented internal buffering in IoTDBRpcDataSet with __df_buffer, and provided methods _has_buffered_data() and next_dataframe() to manage and return DataFrames of size fetch_size. [1] [2]
  • Refactored result set processing logic into _process_buffer() and _build_dataframe() to support both streaming and batch DataFrame construction.

Example and usability improvements:

  • Updated session_example.py and table_model_session_example.py to demonstrate usage of the new streaming DataFrame API. [1] [2]
  • Added missing type import for Optional to support new method signatures.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds a streaming DataFrame interface to the IoTDB Python client, enabling efficient retrieval of large query results in configurable chunks.

Changes:

  • Introduced has_next_df() and next_df() methods to SessionDataSet for iterating over query results as DataFrames
  • Implemented internal buffering mechanism in IoTDBRpcDataSet with next_dataframe() method that returns chunks of exactly fetch_size rows
  • Refactored result_set_to_pandas() into _process_buffer() and _build_dataframe() helper methods to support both batch and streaming DataFrame operations

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
iotdb-client/client-py/iotdb/utils/iotdb_rpc_dataset.py Added streaming buffer (__df_buffer), implemented next_dataframe() method for chunked DataFrame retrieval, refactored DataFrame construction into reusable helper methods, added Optional type import
iotdb-client/client-py/iotdb/utils/SessionDataSet.py Added public API methods has_next_df() and next_df() that delegate to underlying RPC dataset streaming implementation
iotdb-client/client-py/table_model_session_example.py Demonstrated usage of new streaming API with has_next_df()/next_df() pattern
iotdb-client/client-py/session_example.py Demonstrated usage of new streaming API with has_next_df()/next_df() pattern

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov
Copy link

codecov bot commented Jan 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 39.28%. Comparing base (30eebf2) to head (a453860).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17035      +/-   ##
============================================
- Coverage     39.28%   39.28%   -0.01%     
  Complexity      212      212              
============================================
  Files          5104     5104              
  Lines        341484   341490       +6     
  Branches      43520    43522       +2     
============================================
- Hits         134149   134148       -1     
- Misses       207335   207342       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

HTHou

This comment was marked as outdated.

@HTHou HTHou self-requested a review January 19, 2026 02:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants