Feature/add limit lazy implementation - #4
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several improvements and fixes across the codebase, focusing on enhancing test coverage (especially for doctests), improving documentation with more realistic and runnable examples, and refining the core API for better usability and consistency. The most notable changes are grouped below.
Testing and Build Improvements:
conftest.pyto automatically set up the doctest environment, including copying test data and preparing a Parquet file if possible. This ensures doctests can run reliably and with the correct context.pyproject.tomlto include stub files and test sources in builds, and enabled module-level doctests by default.pytestfrom the project root (not just thetestsdirectory), ensuring doctests in source files are executed. [1] [2]API and Core Logic Enhancements:
LazyFrameconstructor to accept any iterable as input, not just lists or dicts, and clarified error messages. [1] [2]headmethod to return a lazyLimitNodein the query plan instead of materializing data immediately, and removed theoptimizeparameter for simplicity. Updated the corresponding method inTypedLazyFrameas well. [1] [2]__len__onLazyFrameto materialize the data if necessary, returning the length of the collected data, improving usability.Documentation and Example Improvements:
src/pyfloe/expr.pyandsrc/pyfloe/core.pyto use realistic, fully runnable code that exercises the API and produces actual output, instead of relying ondoctest: +SKIP. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]Internal and Minor Improvements:
Iterableto imports insrc/pyfloe/core.pyand includedLimitNodein the plan node imports. [1] [2]These changes collectively improve the reliability of the test suite, make the documentation more useful and accurate for users, and simplify the core data frame API for easier usage and extension.