Skip to content

BUG: keep stochastic nominal values stable across reseeds - #1169

Open
thc1006 wants to merge 16 commits into
RocketPy-Team:developfrom
thc1006:bug/stable-nominal-across-reseeding
Open

BUG: keep stochastic nominal values stable across reseeds#1169
thc1006 wants to merge 16 commits into
RocketPy-Team:developfrom
thc1006:bug/stable-nominal-across-reseeding

Conversation

@thc1006

@thc1006 thc1006 commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

119 lines of this are production code. Most of it is stochastic_model.py; the other three files get a line or two each, hooking the same recording call. The other 600 are the tests that each of the problems below cost, and a note in the user documentation. Reading the four hunks in stochastic_model.py is enough to judge the change.

Problem

_set_stochastic(seed) re-validates every declared input, and validation reads the nominal off the wrapped object. create_object writes the sampled value back onto that same object on purpose, so re-reading it on a reseed takes one simulation's output as the next one's nominal.

Four reseeds of one model, all with seed 12345, wind_velocity_x_factor=(1.0, 0.1) on a nominal 10 m/s:

develop    8.576175   7.355078   6.307843   5.409717
this PR    8.576175   8.576175   8.576175   8.576175

Each value on develop is the one before it multiplied by the same factor again. A plain scalar spec drifts the same way, since _validate_scalar and the (std, "distribution") tuple both take their centre from the object.

Invariant established by this PR

A model samples around the value the wrapped object held when that input was configured, however many times it is reseeded and whatever ran before. For the types that get copied, nothing downstream can move it: not the generated object, not last_rnd_dict, and not the next draw. A Function or a callable is held by reference and its own mutators still reach the baseline, which docs/user/stochastic.rst says.

Configured, not built, because an add_* method installs its input after __init__ and is read then. docs/user/stochastic.rst states the rule and the four cases that sit outside it: a late add_* input, a component position, an ensemble wind factor under a selected member, and anything held by reference.

Changes

  • read each nominal once and keep it, instead of re-reading the wrapped object on every reseed;
  • keep that value private, and hand out a copy. It reaches the model attribute, last_rnd_dict and the object create_object returns, and a write through any of those used to land on the kept one;
  • copy through the built-in containers rather than stopping at the outside, since an array inside an airfoil tuple would otherwise stay shared;
  • read the nominal again when a late input is configured again, and replace a pair such as x and y together or not at all. An axis the caller left out keeps everything it had, since an omitted argument and a written None arrive the same way; removing one deliberately is BUG: setting a declared eccentricity back to None leaves the old distribution in place #1171;
  • give every draw its own copy of a mutable value. A list-valued input handed back the candidate itself, so on develop a whole batch of simulations shares one outline, and that outline is the deterministic fin the caller passed in:
five create_object calls, no reset in between
develop   all five share one shape_points object
          writing through the first one reaches calisto_free_form_fins itself
          the next simulation reads (9.9, 9.9) where (0.08, 0.1) was drawn
this PR   each call gets its own, the deterministic fin is untouched

A serial MonteCarlo run never resets between simulations, so this holds for the whole study rather than for adjacent pairs;

  • record a draw through _record_draw, after any subclass that adjusts a value. StochasticFreeFormFins corrects a perturbed outline in dict_generator and StochasticParachute derives its pressure noise seed in create_object, both once the base class has already recorded;
  • drop _declare_stochastic_input and the _MISSING sentinel, which the grouped reconfiguration left without callers;
  • leave a component position uncached: it arrives through an injected getter, reads an attribute nothing writes back to, and shares the one name position across every component;
  • say in docs/user/stochastic.rst where a nominal comes from, since three places there said only that it is taken from the deterministic object.

Non-goals

Breaking change

Yes, for a model whose nominal was drifting. Nothing is drawn differently and no draw is added or removed, so the stream position is untouched: stochastic_calisto under seed 42 reads mass=14.906007947 on develop and the same here.

Verification

Focused first, then the suites the CI jobs run:

pytest tests/unit/stochastic/test_stochastic_model.py       36 passed
pytest tests/unit                                         2188 passed
pytest rocketpy --doctest-modules                           48 passed
pytest tests/integration                                   154 passed
pytest tests/acceptance                                     18 passed
ruff check .  /  ruff format --check .                     clean
pylint rocketpy/ tests/ docs/                           10.00/10, exit 0

The four tests/unit/test_sensitivity.py failures on this machine are a missing statsmodels, and they fail the same way on an unmodified develop.

Each mechanism is pinned by a mutation, and each mutation leaves a control standing:

undone goes red still passes
the nominal is not kept seven _snapshot_of's own test
the kept value is handed out rather than copied the two alias tests the drift tests
_snapshot_of copies nothing the two container tests the drift tests
_snapshot_of stops at a tuple the nested-array test the rest
the cache is keyed by seed rather than by model the three seed-history tests the rest
a late input is not read again when configured again the reconfiguration test the rest
an omitted axis joins the replacement the two axis tests, before the reset as well as after the rest
each input is committed as it validates, one at a time the pair test the rest
a list draw hands back the candidate itself the consecutive-create_object test the rest
last_rnd_dict points at what was built from the record test the rest
the outline correction happens after the record the two free-form record tests, and the source scan the rest
the noise seed is derived after the record the two parachute record tests, and the source scan the rest
the previous nominal is not restored when validation raises the rollback test the rest
the injected getter is ignored test_visualize_attributes[stochastic_calisto] the rest
the getter's result is kept under position too the component-position test the rest

Two of those exist because a weaker fix passes everything else. Keying the cache by seed satisfies "the same seed four times agrees" and still re-reads a moved nominal whenever the seed changes. Collapsing _snapshot_of to a shallow copy satisfies every drift test, because those use scalars.

Together with #1170

Both touch the stochastic package, and green on separate bases says nothing about the merge, so they are merged into a throwaway tree on develop and run there: pylint exit 0, ruff clean, the unit suite, doctests, integration and acceptance. The mutations that undo either change still fail their own tests in the combined tree, so neither is masking the other. stochastic_calisto under seed 42 reads mass=14.906007947 there too.

Related to #1171, which stays open. The reasoning is recorded there.

Extracted from

#1054, as the first of the split you asked for in #1054 (comment).

The idea is that branch's. The code is not: copy-on-read, the recursive snapshot and the reconfiguration path are all new here, and its older one-argument _declare_stochastic_input is deliberately left behind, since #1167 replaced it on develop with a version that keeps the value.

@thc1006
thc1006 requested a review from a team as a code owner August 15, 2026 17:16
@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.62%. Comparing base (4263fa9) to head (302fc8b).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1169      +/-   ##
===========================================
+ Coverage    84.57%   84.62%   +0.04%     
===========================================
  Files          131      131              
  Lines        17527    17557      +30     
===========================================
+ Hits         14824    14857      +33     
+ Misses        2703     2700       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thc1006

thc1006 commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Since this went up I found three things wrong with it and fixed them. The description above is current, so this is only the record of what moved.

The kept nominal was reachable from outside. _nominal handed back the cached object itself, and on the empty-spec path that one object was also the model attribute, the entry in last_rnd_dict, and the outline on the FreeFormFins that create_object returns. A write through any of them moved what the next reseed sampled around:

after generated.shape_points[1] = (9.9, 9.9)
kept nominal = [(0, 0), (9.9, 9.9), (0.12, 0.1), (0.12, 0)]

It copies on the way out now, so what it hands over is disposable.

_snapshot_of stopped at a tuple, which left an array inside an airfoil pair shared with the object it came from. It walks lists, tuples, sets and dicts now, and rebuilds a namedtuple through its own type rather than flattening it.

Configuring a late input twice kept the first nominal. A second add_cp_eccentricity went on sampling around the value the rocket held at the first call, 0.5 where 0.8 had been asked for. Reproducible, and around the wrong centre, which is harder to notice than a value that moves. Late configuration drops the kept value before validation reads one, and puts it back if validation raises.

Two I left alone. #1171 is the other half of the same lifecycle, where setting a declared eccentricity back to None leaves the earlier distribution in place; that is develop's behaviour rather than this branch's. And a Function nominal is still held by reference: set_source on a drag curve does move the baseline, so the boundary has a test now instead of only a line in the docs.

Head is 1f0e516f. Nothing here changes what is drawn or how many draws are made.

@thc1006 thc1006 changed the title BUG: sample around the nominal a stochastic model was built with BUG: keep stochastic nominal values stable across reseeds Aug 15, 2026
_set_stochastic re-validates every declared input, and validation reads the
nominal off the wrapped object. create_object writes the sampled value back
onto that same object on purpose, so re-reading it on a reseed took one
simulation's output as the next one's nominal: a wind factor compounded
10 -> 8.576 -> 7.355 -> 6.308 under a single fixed seed, and a plain scalar
spec drifted the same way.

Read the nominal once and keep it. Containers are copied on the way in, so
writing through the wrapped object cannot reach it either. A component
position arrives through an injected getter, reads an attribute nothing
writes back to, and shares one name across every component, so those are
read live rather than cached.

Extracted from RocketPy-Team#1054.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Copying on the way in was not enough. _nominal handed back the kept object
itself, and on the empty-spec path that one object reached the model
attribute, last_rnd_dict and the FreeFormFins create_object returns, so a
write through any of them moved what the next reseed sampled around.

_snapshot_of stopped at a tuple as well, which left an array inside an
airfoil pair shared with the object it came from.

Copy on the way out too, and recurse through the built-in containers. The
documented contract now names the four cases that stay outside it: an input
added after construction, a component position, an ensemble wind factor, and
anything that is not an array or a built-in container.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
An add_* input is configured after __init__, so its nominal is read then. The
new test writes the rocket's eccentricity before add_cp_eccentricity and again
after it, and only the first one may reach the draw.

The (std, distribution) form now runs its own seed histories rather than
repeating one seed, which is what a cache keyed by the seed instead of by the
model actually fails.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
The documented exception said a Function is held as it was given. Nothing
enforced it, so closing the hole later would have gone unnoticed and the
documentation would have quietly become wrong.

Measured: set_source on the rocket's drag curve moves the drawn value from
0.377 to 0.890, and deepcopy of that curve costs 6 microseconds. Cost is not
the reason to leave it. _snapshot_of cannot raise today, and deepcopying
whatever a user passed, on a path that runs on every reseed, would make it
able to.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Keeping the nominal gave the second add_cp_eccentricity nothing to replace,
so it went on sampling around the value the rocket held at the first call:
0.5 where 0.8 was asked for. Reproducible, and around the wrong centre, which
is harder to notice than a value that moves.

Late configuration drops the kept nominal before validation reads one, and
puts it back if validation raises, so a refused call leaves the previous
configuration standing.

Only the reconfiguration path. Passing None still leaves the earlier
declaration in place, which is develop's behaviour and not this branch's.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Measured on the current implementation: a cycle recurses until Python stops
it, two references to one list come back as two lists, and the elements of an
object-dtype array stay shared. None of those reach a supported nominal, but
the docstring read like a general deep copy and should not.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
None is a configuration too. The nominal was refreshed but the earlier
distribution stayed declared, so the next reseed validated it again and drew
an uncertainty the caller had asked to remove. Filed as RocketPy-Team#1171 while the
removal lived elsewhere; it belongs in the replacement helper this branch
added, so it is here rather than in a second PR that owns the other half of
one state transition.

None still means an axis that was never given, and removing what was never
declared stays a no-op. Both meanings have a test.

The snapshot test asserted a dict entry was not None, which held whether or
not anything had been copied, and no test reached the set branch at all.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
add_cp_eccentricity takes x and y in one call, so a y that will not validate
left x already replaced and declared. Validation happens for the whole group
before anything is committed now.

The test gives only y first, so x is undeclared going in and a partial commit
shows up as an eccentricity the caller never successfully asked for. Asserting
the nominal alone did not catch it, since x's nominal was restored either way.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
@thc1006
thc1006 force-pushed the bug/stable-nominal-across-reseeding branch from 9766211 to 798c0c1 Compare August 16, 2026 00:00
Copying on the way out of the kept nominal was not the last boundary. The list
branch handed back the candidate itself, which for an empty spec is the model's
own working value, and FreeFormFins keeps shape_points by reference. Writing
through the first generated fins reached the second ones:

    first = stochastic.create_object()
    first.shape_points[1] = (9.9, 9.9)
    second = stochastic.create_object()   # (9.9, 9.9) as well

No reseed in between, which is how create_object is documented to be used and
how a serial Monte Carlo runs it.

last_rnd_dict was the same dictionary the values were built from, so it moved
with them too. It records what was drawn now, which matters because a Monte
Carlo writes it out after the flight rather than before.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Recording in the base generator put the record before StochasticFreeFormFins
had pulled the fin root back onto the body line. Under seed 7 the two root
points drifted to 0.000299 and 0.001340, the correction returned them to zero,
and the record kept the outline the fins were never built from. The rocket
copies each component's record into its own, so the Monte Carlo input log
carried it too.

That is the failure class RocketPy-Team#1090 was about, arriving from the other side.

_record_draw is the one place a model publishes what it drew, and a subclass
that changes a value calls it again. A source scan holds the next subclass to
the same rule, since the one that gets it wrong is the one nobody wrote a
fixture for.

_declare_stochastic_input and the _MISSING sentinel had no callers left after
the grouped reconfiguration landed, and the first still carried the None
handling that RocketPy-Team#1171 was about, so they are gone rather than left as a second
lifecycle for someone to reach for.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
The docstring still promised values itself when there are no candidates,
which stopped being true when the draw started handing back a copy. Nothing
reaches that branch through a validated input, since an empty list validates
to the object's own value, so it is a guard against integers(0) rather than a
path with a caller. It has a test now, which is also the one line of this
change Codecov had no coverage for.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
StochasticParachute.create_object derives the pressure noise seed after the
draw, and RocketPy-Team#1134 relied on last_rnd_dict being the same dictionary to carry it
into the record. Snapshotting the draw broke that link: the parachute is still
built with the seed, but the record loses it, so the Monte Carlo inputs stop
describing the parachute that flew.

    develop   recorded 37773913418288439290323614982376424810
    before    recorded <absent>

The source scan missed it because it only read dict_generator overrides. It
reads create_object too now, and tracks the names a method binds from a draw
rather than guessing at a variable name, so a local a method fills in for its
own use is not mistaken for a record.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
StochasticMotorModel is a StochasticModel subclass that rocketpy.stochastic
does not export, so the scan could not see it. It overrides neither method
today, which is why nothing was wrong, and which is also why the gap would
have gone unnoticed until something did.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Removing on None looked like one line inside the new replacement helper, and
it is not. add_cp_eccentricity(x=..., y=...) defaults both to None, so an
omitted axis and an explicit None read identically, and the removal took away
an axis the caller never mentioned:

    add_cp_eccentricity(x=0.001, y=0.002)
    add_cp_eccentricity(x=0.005)     # y quietly gone

develop keeps y here, and so does this again. Removing an earlier declaration
needs an argument omission cannot supply, which is a signature change and its
own decision, so it stays in RocketPy-Team#1171 rather than arriving inside a change about
nominal ownership.

The test that asked for removal is replaced by one that holds the omitted axis
in place, since that is the behaviour anything already written depends on.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Both arguments read as optional and nothing said what happens when the method
is called again, which is the whole of the question behind RocketPy-Team#1171. Each public
docstring now states it: a later call replaces what was configured, an omitted
axis keeps what it had, and taking one away is not supported.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Keeping its declaration was not enough. The omitted axis still went through
the whole replacement: its kept nominal was dropped, None was validated again
into a lone nominal, and that was written back over its distribution. The
private side then said the axis was random while the attribute dict_generator
reads said it was not, so it stopped varying:

    add_cp_eccentricity(x=0.001, y=0.002)
    add_cp_eccentricity(x=0.005)
    eight draws of y -> one distinct value

A serial Monte Carlo never resets, so a whole study would have run with that
axis switched off and nothing raised.

Dropping the nominal also moved the centre. With the rocket's own y changed
between the two calls, the next reset centred the old distribution on 9.0
rather than the 0.0 it was configured around.

An axis given as None that already has a configuration is now left out of the
transaction: not revalidated, its nominal not re-read, its attribute not
rewritten. The test covers both eccentricity methods and looks before the
reset as well as after, which is where the previous one missed it.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant