Skip to content

Feature/add limit lazy implementation - #4

Merged
Edwardvaneechoud merged 5 commits into
mainfrom
feature/add-limit-lazy-implementation
Mar 20, 2026
Merged

Feature/add limit lazy implementation#4
Edwardvaneechoud merged 5 commits into
mainfrom
feature/add-limit-lazy-implementation

Conversation

@Edwardvaneechoud

Copy link
Copy Markdown
Owner

This pull request introduces several improvements and fixes across the codebase, focusing on enhancing test coverage (especially for doctests), improving documentation with more realistic and runnable examples, and refining the core API for better usability and consistency. The most notable changes are grouped below.

Testing and Build Improvements:

  • Added a global pytest fixture in conftest.py to automatically set up the doctest environment, including copying test data and preparing a Parquet file if possible. This ensures doctests can run reliably and with the correct context.
  • Updated pyproject.toml to include stub files and test sources in builds, and enabled module-level doctests by default.
  • Modified GitHub Actions workflows to run pytest from the project root (not just the tests directory), ensuring doctests in source files are executed. [1] [2]

API and Core Logic Enhancements:

  • Improved the flexibility of the LazyFrame constructor to accept any iterable as input, not just lists or dicts, and clarified error messages. [1] [2]
  • Refactored the head method to return a lazy LimitNode in the query plan instead of materializing data immediately, and removed the optimize parameter for simplicity. Updated the corresponding method in TypedLazyFrame as well. [1] [2]
  • Changed the implementation of __len__ on LazyFrame to materialize the data if necessary, returning the length of the collected data, improving usability.

Documentation and Example Improvements:

  • Rewrote and expanded doctest examples in src/pyfloe/expr.py and src/pyfloe/core.py to use realistic, fully runnable code that exercises the API and produces actual output, instead of relying on doctest: +SKIP. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

Internal and Minor Improvements:

  • Added Iterable to imports in src/pyfloe/core.py and included LimitNode in the plan node imports. [1] [2]

These changes collectively improve the reliability of the test suite, make the documentation more useful and accurate for users, and simplify the core data frame API for easier usage and extension.

@Edwardvaneechoud
Edwardvaneechoud merged commit c1e7a45 into main Mar 20, 2026
4 checks passed
@Edwardvaneechoud
Edwardvaneechoud deleted the feature/add-limit-lazy-implementation branch March 20, 2026 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant