TST: run the EnvironmentAnalysis tests in the default suite - #1176
Open
ting-hong-shieh wants to merge 1 commit into
Open
TST: run the EnvironmentAnalysis tests in the default suite#1176ting-hong-shieh wants to merge 1 commit into
ting-hong-shieh wants to merge 1 commit into
Conversation
Nine of the ten EnvironmentAnalysis tests carried @pytest.mark.slow, so test_pytest.yaml never ran them and the subsystem measured 7.0%, 39.7% and 19.4% in the run that gates every pull request. They were not slow because of the work they do. The env_analysis fixture reads two committed files under data/weather/, touches no network, and was declared at function scope, so the nine tests each rebuilt it and twenty years of hourly reanalysis data was parsed nine times. Roughly 212 of the 247 seconds the two files took were that repeated setup. Scope the fixture to the session and drop the nine marks. No test rebinds its attributes; test_exports already loads into a copy.deepcopy rather than into the fixture. Non-slow coverage goes from 84.2757% to 89.6674%, +945 statements, for about 74 seconds across the five steps.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1176 +/- ##
===========================================
+ Coverage 84.57% 89.96% +5.39%
===========================================
Files 131 131
Lines 17527 17527
===========================================
+ Hits 14824 15769 +945
+ Misses 2703 1758 -945 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TST: run the EnvironmentAnalysis tests in the default suite
Closes part of #709.
What this changes
EnvironmentAnalysishas ten tests. Nine of them carry@pytest.mark.slow, sotest_pytest.yamlnever runs them and the subsystem measures 7.0% (plots/environment_analysis_plots.py), 39.7% (environment/environment_analysis.py) and 19.4% (prints/environment_analysis_prints.py) in the run that gates every pull request.They are not slow because of the work they do. The
env_analysisfixture reads two committed files underdata/weather/, touches no network, and is declared at function scope, so nine tests each rebuild it and twenty years of hourly reanalysis data is parsed nine times.This pull request sets the fixture to
scope="session"and removes the nine slow marks. No production code changes.Effect on the default suite
Measured by running the five steps of
test_pytest.yamlin order — unit, doctests, integration with the three animation deselects, the VTK animation tests, acceptance — each--cov-append, at4263fa95d7fe6f63d9593f01e4ff7a088369e195, before and after this branch.Per module:
plots/environment_analysis_plots.pyenvironment/environment_analysis.pyprints/environment_analysis_prints.pyThat is +945 statements and +5.39 points, which leaves the suite 6 statements short of the 90% target in #709. Codecov reports the same pair for this branch, 84.57% to 89.96%.
Effect on runtime
Per stage, same five commands:
About 74 seconds added in total. The fixture is built once per pytest invocation, so the unit and integration steps pay for it separately.
That cost is what is left after the scope change. The nine tests on their own, both timings with a warm page cache so the comparison is fair:
The repeated setups were roughly 212 of the 247 seconds this file took on a cold cache.
Holding the analysis for the session costs about 84 MB of resident memory, measured as the RSS delta across constructing the fixture object.
Why the session scope is safe here
Every one of the ten tests reads from the fixture and none rebinds its attributes.
test_exportsis the only one that loads state into anEnvironmentAnalysis, and it already does that on acopy.deepcopyrather than on the fixture.create_environment_objectreturns a newEnvironment.What is not in scope
Twenty slow marks remain, twelve of them in
tests/integration/environment/test_environment.py, the rest across the Monte Carlo, flight, encoding and Open-Meteo tests. Those are slow for reasons a fixture scope does not address, and the scheduledtest-pytest-slow.yamljob still runs them with-m slow --runslow.No CHANGELOG entry: the file asks for tests to be left out.
Corrected after opening: the first version of this table was measured in an environment without
contextily, which skipstest_monte_carlo_plots_background.pyand understated both columns by 53 statements. The change itself, +945, was reported correctly and matches Codecov. #1179 fixes the packaging gap that caused it.