Skip to content

TST: run the EnvironmentAnalysis tests in the default suite - #1176

Open
ting-hong-shieh wants to merge 1 commit into
RocketPy-Team:developfrom
ting-hong-shieh:tst/env-analysis-non-slow
Open

TST: run the EnvironmentAnalysis tests in the default suite#1176
ting-hong-shieh wants to merge 1 commit into
RocketPy-Team:developfrom
ting-hong-shieh:tst/env-analysis-non-slow

Conversation

@ting-hong-shieh

@ting-hong-shieh ting-hong-shieh commented Aug 16, 2026

Copy link
Copy Markdown

TST: run the EnvironmentAnalysis tests in the default suite

Closes part of #709.

What this changes

EnvironmentAnalysis has ten tests. Nine of them carry @pytest.mark.slow, so test_pytest.yaml never runs them and the subsystem measures 7.0% (plots/environment_analysis_plots.py), 39.7% (environment/environment_analysis.py) and 19.4% (prints/environment_analysis_prints.py) in the run that gates every pull request.

They are not slow because of the work they do. The env_analysis fixture reads two committed files under data/weather/, touches no network, and is declared at function scope, so nine tests each rebuild it and twenty years of hourly reanalysis data is parsed nine times.

This pull request sets the fixture to scope="session" and removes the nine slow marks. No production code changes.

Effect on the default suite

Before After
Statements 17,527 17,527
Covered 14,824 15,769
Missed 2,703 1,758
Coverage 84.5781% 89.9583%

Measured by running the five steps of test_pytest.yaml in order — unit, doctests, integration with the three animation deselects, the VTK animation tests, acceptance — each --cov-append, at 4263fa95d7fe6f63d9593f01e4ff7a088369e195, before and after this branch.

Per module:

Module Before After
plots/environment_analysis_plots.py 7.0% 98.4%
environment/environment_analysis.py 39.7% 94.2%
prints/environment_analysis_prints.py 19.4% 100.0%

That is +945 statements and +5.39 points, which leaves the suite 6 statements short of the 90% target in #709. Codecov reports the same pair for this branch, 84.57% to 89.96%.

Effect on runtime

Per stage, same five commands:

Stage Before After
Unit 2,167 passed, 13 skipped, 126.70s 2,173 passed, 7 skipped, 154.04s
Doctests 48 passed, 4.41s 48 passed, 4.29s
Integration 154 passed, 40 skipped, 83.61s 157 passed, 37 skipped, 130.02s
VTK animation 3 passed, 5.93s 3 passed, 4.51s
Acceptance 18 passed, 12.51s 18 passed, 12.56s

About 74 seconds added in total. The fixture is built once per pytest invocation, so the unit and integration steps pay for it separately.

That cost is what is left after the scope change. The nine tests on their own, both timings with a warm page cache so the comparison is fair:

$ pytest tests/unit/environment/test_environment_analysis.py \
         tests/integration/environment/test_environment_analysis.py --runslow -q
10 passed in 106.70s     # function-scoped fixture
10 passed in  30.11s     # session-scoped fixture

The repeated setups were roughly 212 of the 247 seconds this file took on a cold cache.

Holding the analysis for the session costs about 84 MB of resident memory, measured as the RSS delta across constructing the fixture object.

Why the session scope is safe here

Every one of the ten tests reads from the fixture and none rebinds its attributes. test_exports is the only one that loads state into an EnvironmentAnalysis, and it already does that on a copy.deepcopy rather than on the fixture. create_environment_object returns a new Environment.

What is not in scope

Twenty slow marks remain, twelve of them in tests/integration/environment/test_environment.py, the rest across the Monte Carlo, flight, encoding and Open-Meteo tests. Those are slow for reasons a fixture scope does not address, and the scheduled test-pytest-slow.yaml job still runs them with -m slow --runslow.

No CHANGELOG entry: the file asks for tests to be left out.


Corrected after opening: the first version of this table was measured in an environment without contextily, which skips test_monte_carlo_plots_background.py and understated both columns by 53 statements. The change itself, +945, was reported correctly and matches Codecov. #1179 fixes the packaging gap that caused it.

Nine of the ten EnvironmentAnalysis tests carried @pytest.mark.slow, so
test_pytest.yaml never ran them and the subsystem measured 7.0%, 39.7%
and 19.4% in the run that gates every pull request.

They were not slow because of the work they do. The env_analysis fixture
reads two committed files under data/weather/, touches no network, and
was declared at function scope, so the nine tests each rebuilt it and
twenty years of hourly reanalysis data was parsed nine times. Roughly 212
of the 247 seconds the two files took were that repeated setup.

Scope the fixture to the session and drop the nine marks. No test rebinds
its attributes; test_exports already loads into a copy.deepcopy rather
than into the fixture.

Non-slow coverage goes from 84.2757% to 89.6674%, +945 statements, for
about 74 seconds across the five steps.
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.96%. Comparing base (4263fa9) to head (5d4343f).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1176      +/-   ##
===========================================
+ Coverage    84.57%   89.96%   +5.39%     
===========================================
  Files          131      131              
  Lines        17527    17527              
===========================================
+ Hits         14824    15769     +945     
+ Misses        2703     1758     -945     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant