Interior refactor: Aragog, Zalmoxis, atmodeller, dummy modules, testing overhaul, version pinning, oxygen accounting#678
Open
timlichtenberg wants to merge 1082 commits into
Open
Interior refactor: Aragog, Zalmoxis, atmodeller, dummy modules, testing overhaul, version pinning, oxygen accounting#678timlichtenberg wants to merge 1082 commits into
timlichtenberg wants to merge 1082 commits into
Conversation
timlichtenberg
changed the title
Member
|
Exciting! Looking forward to reviewing this PR when it's ready :) |
timlichtenberg
added a commit
that referenced
this pull request
May 14, 2026
PR #678 has been a draft for weeks and CI was effectively silent on every push because GitHub stopped firing pull_request:synchronize events for draft pull requests in September 2022. There is no workflow-level flag to opt back in; the event is filtered out at the event-routing layer before the workflow sees it. Manual workflow_dispatch after each push works but is tedious and gets skipped in practice. Adds `tl/**` to the push trigger in ci-pr-checks.yml and code-style.yaml so my long-running draft branches get CI on every push regardless of draft state. The pattern is narrow enough that nobody else's branches are affected. Adds a concurrency group keyed on the commit SHA in both workflows. When a tl/** PR eventually transitions out of draft, the same commit will fire BOTH the push event AND pull_request:synchronize. The concurrency group cancels the older run when the newer one fires so the matrix only executes once per commit, preserving the lesson from the aragog publish-workflow double-fire incident. Drops the `if: github.event.pull_request.draft == false` filter from code-style.yaml's codestyle job. It was redundant: GitHub's default draft-block on pull_request already prevents that path; the filter also evaluated to false on push events (because github.event.pull_request is null), which would have blocked the new push trigger from working.
timlichtenberg
added a commit
that referenced
this pull request
May 14, 2026
Adds @pytest.mark.skip with FIXME reasons to every test that surfaced as failing once the push trigger started actually exercising the suite. All failures trace to environment issues in the CI Docker image, not code defects: - input/minimal.toml does not validate against the post-merge config schema (3 tests) - SPIDER/Aragog P-S EOS lookup tables (Zenodo 19473625) are not present in the Docker image (1 test) - fwl_data/planet_reference/Exoplanets/DACE_PlanetS.csv is not present in the Docker image (6 smoke tests) - The inference smoke fixture invokes proteus start which exits code 1 inside the CI container (4 smoke tests) Full inventory, root causes, and the re-enable workflow are tracked in claude-config/memory/projects/proteus/ ci_skipped_tests_2026_05_14.md so we can pick them back up during the test infrastructure rework phase before PR #678 moves out of draft.
timlichtenberg
added a commit
that referenced
this pull request
May 14, 2026
src/proteus/outgas/calliope.py imports equilibrium_atmosphere_authoritative_O at module load. That entry point exists only on the tl/fo2-source-framework branch of CALLIOPE and has not yet shipped to PyPI; with the previous version pin CI collects tests against a CALLIOPE that lacks the symbol and the unit + smoke tiers both fail at import. This is a temporary cross-repo coupling. The right end state is the CALLIOPE branch merged into main, a 26.05.14 release published to PyPI, and this dependency reverted to a normal version pin. Until then the git URL keeps PR #678 testable.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #678 +/- ##
===========================================
+ Coverage 70.56% 91.20% +20.64%
===========================================
Files 100 108 +8
Lines 13675 14310 +635
Branches 2241 2582 +341
===========================================
+ Hits 9650 13052 +3402
+ Misses 3875 1258 -2617
+ Partials 150 0 -150
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
timlichtenberg
added a commit
that referenced
this pull request
May 16, 2026
The cache warmup workflow only clones main and runs the setup-proteus composite. It produces no commits, no comments, and no artifacts, so the GITHUB_TOKEN never needs write access. Set permissions to the minimal contents:read. Addresses the CodeQL workflow-permissions advisory on PR #678.
timlichtenberg
added a commit
that referenced
this pull request
May 16, 2026
The PyPI-URL assertion in test_python_package_latest_version_returns_pypi_value used a substring check (`'pypi.org' in url`) that accepts a URL where 'pypi.org' appears in the query string or path of an attacker-controlled host (e.g. https://attacker.example/?host=pypi.org/foo). Switch to urlparse so the hostname is matched exactly, and check the package name on the parsed path instead of the raw URL. Strengthens the discrimination guard against URL-spoofing regressions and clears the CodeQL py/incomplete-url-substring-sanitization alert on PR #678.
timlichtenberg
added a commit
that referenced
this pull request
May 17, 2026
The four tests in tests/tools/test_chili_compare_mappings.py loaded tests/validation/chili/compare_to_chili.py via importlib, asserting on its CHILI_TO_PROTEUS dict mappings. The previous commit moved compare_to_chili.py into the dev attic, which left this test with no target on disk; the four assertions raised FileNotFoundError and broke both Linux and macOS unit tests on PR #678. The test is now in the attic alongside the script it guarded. Both files were added on this branch and the regression they together catch is a property of the comparison workflow (CHILI Table 3 column mapping), which lives in the attic now.
Six new unit tests target the previously-untested paths in proteus/orbit/wrapper.py: - init_orbit early-return on module='None' (string sentinel, not Python None). Discriminating against a regression that compared against the Python literal. - init_orbit lovepy import branch, with the heat_tidal=False warning. Patches lovepy.import_lovepy so the test does not require the Julia runtime. - update_period sanity warning on unphysical (sub-kg) total mass. - run_orbit dummy module dispatch: Imk2 is set via run_dummy_orbit. - run_orbit no-module branch: Imk2 = 0.0 and no submodule called. - run_orbit Roche-limit warning when separation < roche_limit. Each new test seeds hf_row with semimajorax_sat upstream of update_separation because the source order populates that field only inside the satellite=False branch (which runs after update_separation). Tests are dispatch-focused so the lovepy and Julia paths stay mocked.
Six tests cover the previously-untested branches of proteus/star/phoenix.py: - Teff and radius pulled from a spada track at the configured age, with an age-unit guard that rejects Gyr-vs-Myr confusion. - Teff and radius pulled from a Baraffe track when tracks='baraffe', asserting the spada Value entry point stays uncalled. - log g computed from G * M / R**2 in cgs. The closed-form pin catches the units bug class that would forget the SI->cgs factor of 100 and land 2 dex low. - Stellar mass outside the spada range raises ValueError and writes status code 23 via UpdateStatusfile. - Explicit overrides for Teff / radius / log g in config bypass the track-driven branch. - get_phoenix_modern_spectrum raises ValueError when any of Teff / radius / log g remain None. Each test sits inside the anti-happy-path discipline: edge case plus a discrimination guard against the most plausible regression.
Five new unit tests reach the previously-untested Baraffe dispatch paths in update_stellar_radius, update_stellar_temperature, and update_instellation, plus the closed-form Stefan-Boltzmann equilibrium-temperature helper. Each Baraffe test asserts the Baraffe entry point was called and the spada Value method was NOT, ruling out a dispatch-swap regression. The equilibrium temperature test pins T_eqm = 254 K for Earth-like insolation and albedo and adds exponent guards against the most plausible Stefan-Boltzmann regressions (T^3 -> 720 K, T^5 -> 96 K). The dummy-star instellation branch is covered with a mocked calc_instellation: the wrapper passes Teff + R_star + separation in SI units verbatim, and F_xuv is set explicitly to 0.0.
Four new unit tests target the previously-untested branches in proteus/outgas/atmodeller.py::calc_surface_pressures_atmodeller: - Solubility model name configured but not in atmodeller's registry: a warning fires naming the missing model; dispatch continues. - Real-gas EOS name configured but not in the eos registry: the warning mentions the ideal-gas fallback alongside the gas name. - T_magma below the configured T_floor: the wrapper logs a warning and returns before model.solve runs; hf_row stays untouched. - model.solve raising RuntimeError: the exception bubbles up to the caller; the wrapper logs 'Atmodeller solve failed' first. Each test reuses the existing _make_path_c_config + _earth_hf_row + _fake_atmodeller_output fixtures so the species-network construction runs through the real registry while the solve is stubbed.
Six new unit tests close the previously-untested branches in proteus/utils/coupler.py: - _get_git_revision when chdir fails: returns 'unknown' and the caller's CWD is preserved (catches a missing finally-block restore). - _get_git_revision when subprocess raises FileNotFoundError (git absent): the catch-all except still produces 'unknown'. - _get_git_revision when subprocess raises TimeoutExpired: pin the timeout path explicitly so a future narrowing of the except clause surfaces here. - validate_module_versions when a module is older than the pinned minimum: raises EnvironmentError AND writes status code 20 via UpdateStatusfile. Pin both side effects. - validate_module_versions when the installed version is above the expected minimum: returns None without touching the status file. - validate_module_versions when the dependency string has no entry for the module: the closure-resolved expected version is None and the 'no pin' guard short-circuits the comparison. Each test patches importlib.metadata.requires() to drive the closure helper _get_expver from outside the function, keeping the mocks at the narrowest viable scope.
Eight follow-up corrections surfaced by an independent review pass
over the Track 0 + Wave A commits.
Critical fix:
- Register the four new AGNI diagnostic columns (tau_atm_TOA,
tau_atm_surface, agni_Ra_max, agni_t_conv_over_t_rad) in
GetHelpfileKeys(). atmos_clim/wrapper.py copies output keys into
hf_row only when they already exist there, so without this
registration the diagnostics never reach runtime_helpfile.csv.
Marker discipline:
- tests/atmos_clim/test_agni.py: pytestmark is now a list with
timeout(30), matching every other unit-test file in the repo.
- tests/utils/test_coupler.py: same fix.
- tests/star/test_wrapper.py: remove physics_invariant from the
module-level marker; apply per-function only to the closed-form
formula tests and the inverse-square pins. Structural dispatch
tests (Baraffe entry-point + Numpy shape check + zero-flux
identity) no longer carry the marker.
- tests/atmos_clim/test_agni.py: physics_invariant added to the
optical-depth monotonicity test and the RCB ratio test.
- tests/star/test_phoenix.py: physics_invariant added to the
closed-form log g test.
- tests/escape/test_escape.py: physics_invariant added to the
mass-conservation tests (bulk-reservoir partitioning,
negative-mass clamp, baseline-snapshot accounting).
Source hygiene:
- src/proteus/atmos_clim/agni.py: replace two historical-narrative
comments ('added in AGNI 1.10.2', 'AGNI 1.10.2 diagnostics') with
plain-language descriptions of what the code does.
Test docstring fix:
- tests/atmos_clim/test_agni.py: correct the discrimination guard
description for test_summarise_diagnostics_picks_top_convective
level_for_rcb. The previously-cited regression (argmax over all
booleans landing at index 3) is wrong because np.argmax returns
the first True. The correct alternative regression
(np.where(mask_c)[0][-1]) is the one that lands at index 3.
All 280 tests across the touched files pass.
Six new unit tests target the previously-untested branches in proteus/interior_energetics/boundary.py: - BoundaryRunner constructor T_surf_0 <= T_p_0 ordering guarantee (clamp when user-supplied T_surf > T_p). Temperatures stay positive. - boundary_layer_thickness tiny-delta fallback at T_p == T_surf (limit-input case that would otherwise divide by zero). - boundary_layer_thickness sign and bounded-range invariant for representative magma-ocean and partially-cooled cases. - dT_pdt denominator switch in the fully-solid regime (T_p below solidus). Asserts the cooling-rate sign without pinning a closed-form value the parameterised viscosity model controls. - run_solver CSV logging path: when logging is enabled, a CSV header and at least one data row land at the configured output directory. All six tests use the existing SimpleNamespace-based fixtures and patch solve_ivp at the in-module binding so no real ODE is integrated. The physics_invariant marker is applied per-function only where the assertion exercises a real physics invariant.
Seven new unit tests target the previously-untested error paths in proteus/atmos_clim/common.py: - read_ncdf_profile returns None and logs an error when the NetCDF file is absent. The main loop relies on this contract. - read_atmosphere_data returns None when any timestep profile is unreadable AND emits the 'extract archived data' hint when data.tar exists in the output folder. - Albedo_t logs an error and stays self.ok=False when the lookup CSV is missing. - Albedo_t logs an error and stays self.ok=False when pd.read_csv raises on a corrupt file. - Albedo_t logs an error and stays unloaded when the CSV lacks the required 'tmp' or 'albedo' columns. - Albedo_t.evaluate returns None and logs an error when called on an unloaded instance. - Albedo_t.evaluate clamps an out-of-range interpolator output to [0, 1] AND logs a warning. Marked physics_invariant because the boundedness assertion holds for every input. Each test pins both the side-effect (state flag or log line) and the return value to discriminate against partial regressions that would flip only one of the two.
Five new unit tests for the grid-level helpers in proteus/utils/plot_offchem.py: - offchem_read_grid raises when the target folder contains no case_* subfolders. - offchem_read_grid aggregates per-year pkl snapshots across two fake case folders, sorts the years per case, and returns the expected 2-D shape. - offchem_slice_grid keeps only grid points whose options match every cvar_filter entry. - offchem_slice_grid warns and returns empty arrays when the filter excludes every point. - offchem_slice_grid warns when a filter key is absent from a grid point's options dict; the point is still included because the source's `continue` skips the exclusion check for unknown keys. read_config is stubbed via monkeypatch so the test pkls don't have to round-trip the full PROTEUS config schema. The module-level pytestmark is upgraded to a list including timeout(30) to match the project convention.
Three corrections from the independent review pass over commits 90aa8ff / e81e291 / f1f0e9a: - test_boundary_runner_init_clamps_surface_temperature_below_potential: rewrite to actually exercise the clamp. The previous shape never forced T_surf_0 > T_p_0, so the test passed for the wrong reason. Now drives the zalmoxis-style init path (interior_o.ic == 2) with hf_row['T_surf'] > hf_row['T_magma'] and pins T_surf_0 == T_p_0 - 1 exactly. The physics_invariant marker is removed because this is a structural ordering check, not a physics-invariant assertion. - test_dT_pdt_uses_silicate_heat_capacity_for_fully_solid_mantle: upgrade from a sign-only assertion to a closed-form numeric pin. The test now re-derives numerator and denominator at the parameters under test (phi == 0, r_s == planet_radius) and pins dT_pdt to numerator / (silicate_heat_capacity * mantle_mass). Discrimination guard: re-derives the partial-melt-branch denominator and confirms the alternate ratio differs from the captured value by at least half its magnitude. - test_offchem_read_grid_aggregates_year_data_across_cases: replace the cardinality-only check with a set-membership assertion on the case names AND a per-row name-keyed year-sum check. The previous shape could not catch a regression that joined the wrong case folder to the wrong opts dict, because glob ordering is not stable across platforms.
Move the 50-line AGNI-vs-interior deadlock-detection block out of Proteus.start() into a new _check_atmosphere_deadlock method on Proteus. The inline block becomes a single call. No behaviour change. Six new unit tests on the helper: - Converged solve resets the counter to zero, regardless of any pre-loaded near-trip value. - First iteration (hf_all is None) leaves the counter at zero because there is no previous row to compare against. - AGNI failure with a still-moving interior resets the counter: transient non-convergence is not a deadlock. - AGNI failure with frozen interior state (bit-exact T_magma and Phi_global, F_atm relative change below 1e-6) increments the counter and logs the warning naming both current and max. - Counter at threshold raises RuntimeError AND writes status code 22 via UpdateStatusfile, in that order. - F_atm 1e-6 relative tolerance boundary: a 5e-7 change classifies as frozen (increments); a step to 2x F_atm classifies as moving (resets). Guards against a flipped comparator.
Four new unit tests target the previously-untested postprocessing methods on Proteus (lines 1055-1098): - observe() reads the helpfile, takes the last row, and dispatches to run_observe with (hf_row, output_dir, config). Pin the hf_row type as dict (catches a regression that passed the full DataFrame) and pin the output_dir as the second positional arg. - observe() raises with 'too short to be postprocessed' on an empty helpfile; run_observe must NOT be called. - offline_chemistry() dispatches to run_chemistry and returns its result identity-equal (no wrapping, no copying). - offline_chemistry() raises the same way on an empty helpfile. Each test extracts the archives via patch.object so the file-system side-effect path stays mocked. Discrimination guards pin both the dispatch args and the return propagation.
Three corrections covering the Wave B commits and adjacent files: Discrimination guards on observe/offline_chemistry tests: - Replace the one-row helpfile fixture with a three-row variant with distinct T_magma, F_atm, Phi_global values across rows. - The single-row fixture could not distinguish iloc[-1] from iloc[0] or iloc[middle], so a row-index regression would have passed undetected. The new fixture pins last-row values and adds != guards against the first-row and middle-row values. Drop stale internal-analysis paths from comments: - proteus.py:572 and :860 referenced output_files/analysis_1EO_M1_profiles/DEEP_ANALYSIS_FINDINGS.md. That folder is gitignored and the analysis run is not part of the repo; the path adds no value for a cold reader. The remaining comments still explain the failure mode in current-state terms. Adjacent test-file hygiene: - tests/atmos_clim/test_agni_deadlock.py:33 was using the bare pytestmark = pytest.mark.unit form; promoted to the list form with timeout(30) to match every other unit-test file. - The "first-iteration" test docstring was rephrased to describe the defect mechanism rather than how it was found.
Six new integration-tier tests pin the (atmos_clim=agni,
interior_energetics=aragog) production coupling:
- atmos_clim schema round-trip for 'agni' and 'janus' (the two
real backends), the cross-validator-aware 'dummy' branch with
rayleigh disabled, and rejection of an invalid backend name.
- interior_energetics schema round-trip for module='aragog' and
both Aragog backends ('jax' production default, 'numpy' fallback),
plus rejection of an invalid backend.
- AGNI _summarise_tau_band on a four-level, three-band profile that
grows monotonically with depth: pin tau_atm_TOA at 0.0117 and
tau_atm_surface at 5.67 (band-mean per level), assert the
matrix design invariant tau_atm_TOA < 0.5 * tau_atm_surface.
- _summarise_tau_band on a fully-transparent (all-zero) tau_band:
both endpoints land exactly at zero. Limit-input case.
- GetHelpfileKeys registers the four AGNI 1.10.2 diagnostics so
the atmos_clim wrapper merge guard propagates them to hf_row.
- The two diagnostic summarisers emit finite values on a
well-formed input and NaN on a degenerate input, never raising.
The slow-tier two-timestep coupled run with real Julia + real CVode
sits at ~30 min on Linux GHA and is out of scope at the integration
tier; the existing test_smoke_modules.py chain exercises the
real-binary AGNI path in nightly.
…lback Two corrections targeting nightly CI and the AGNI x aragog test. AGNI build order: AGNI 1.10.2 added Glob to its Project.toml. The setup-proteus action previously ran `julia deps/build.jl` first, which (via SOCRATES's clean_wrappers.jl) does `using Glob` against Julia's default environment. The default env carries no AGNI deps, so the build script failed with "Package Glob not found in current path." Swap the order: activate AGNI's project and Pkg.instantiate first so every dep listed in the manifest (including Glob) is available when deps/build.jl runs. Comment explains the order constraint. Diagnostics aggregator NaN contract: The AGNI x aragog test was missing a degenerate-input check for _summarise_diagnostics. Add a purely-radiative atmosphere case (mask_c all False): Ra_max stays finite at the array max, and the timescale ratio is NaN because there is no convective level to anchor at. Pin both: Ra_max == 3.0 and math.isnan(ratio).
Six new integration-tier tests pin the (atmos_clim=agni, outgas=calliope) Stage-1b Paper-3 fiducial coupling: - outgas schema round-trip for 'calliope', 'atmodeller', 'dummy' and rejection of an invalid module name. - CALLIOPE solver parameters nguess and nsolve enforce strict positivity at the attrs validator. Pin zero AND negative to reject a regression that swapped gt(0) for ge(0). - Calliope.is_included covers every gas in the documented ten-species set (H2O, CO2, N2, S2, SO2, H2S, NH3, H2, CH4, CO); an undocumented attribute (Ar) raises rather than silently returning False. - Optical-depth monotonicity from TOA to surface on a five-level, three-band profile representative of a wet-greenhouse atmosphere. Pin tau_atm_TOA = 0.00117 and tau_atm_surface = 12.67 from the closed-form row means; assert the matrix design invariant tau_atm_TOA < 0.5 * tau_atm_surface (~4 orders of magnitude margin in this regime). - Per-band monotonicity check: every band's depth profile must be non-decreasing. Stronger than the mean-only check; catches a regression that flipped only one band's indexing. - Wrapper merge contract: GetHelpfileKeys registers BOTH the four AGNI 1.10.2 diagnostic keys AND the CALLIOPE per-gas partial pressure keys (H2O_bar, CO2_bar, N2_bar, H2_bar, CO_bar).
rdc49
reviewed
Jun 5, 2026
Contributor
There was a problem hiding this comment.
This pull request looks, looking forward to trying out some of the new features. I can confirm that all the tutorials ran successfully on my machine.
In response to your query regarding the heat fluxes in the boundary module, the module should run as you describe. F_int should be set to F_atm: q_m describes the heat exchange between the mantle and the surface, and should only be seen by the backend. In short, the model runs as intended.
I have a small query about the .toml files that were removed from the input/planets folder. Isn't it useful to have these files so that we have some frame of reference when we want to model well-known planets?
Member
Author
Yes, I am aware that this would be remarked. I can see the ups of this. However, I personally think that maintaining all of these is a massive burden. Whenever there is a config change in principle all of these TOML files need to be tested by hand right now. In my opininon, we should only maintain TOML files that are incorporated into some sort of automated workflow. The current CHILI TOMLs are exceptions as they are incorporated into the tutorial. However, I think we should also incroprate them automatically. One of my next actions (after fwl-io though) will be to try to install an automated execution of some of these TOMLs on a nightly runner to resolve this. This would provide a permanent live-execution of the code to see how the main version under all defaults performs. TOMLs for individual planets should live on the Zenodo archives of finalised data for papers imo, together with the exact PROTEUS version (github SHA) they were executed on. |
Member
|
I don't think I've run with PALEOS before, so not sure if this is related to this PR, but is it expected that the first PALEOS steps take very long to run or is this an issue on my end? When running with all_options.toml, I get the following output somewhere near the beginning: and on the last couple of lines it hangs for many minutes. Later on, similar long waits. Just wondering if this is expected or whether perhaps this PR causes it (or something that went wrong with my install). @maraattia |
stuitje
reviewed
Jun 5, 2026
Contributor
There was a problem hiding this comment.
This file should now also mention the Validation/ directory inside docs/.
egpbos
reviewed
Jun 5, 2026
Member
egpbos
left a comment
egpbos
left a comment
There was a problem hiding this comment.
Fixes for installing AGNI (follow up from the AGNI PR).
| # under socrates/ by tools/get_socrates.sh; the build output is what | ||
| # the AGNI / JANUS atmosphere modules link against at runtime. | ||
| url = "https://github.com/FormingWorlds/SOCRATES.git" | ||
| ref = "e7133ff7388847c7939b38572c6e91cd05d5b755" |
Member
There was a problem hiding this comment.
This particular ref seems to be causing issues with the AGNI install, because it still uses Glob (also see discussion here nichollsh/AGNI#204 (comment)). One commit later, the Glob dependency was removed: FormingWorlds/SOCRATES@f77be10
Which SOCRATES commit shall we use here to fix the AGNI install? We should at least upgrade to f77be10, but maybe we can also just use the latest one.
Member
There was a problem hiding this comment.
We should update to the latest one fe0c4a486f1dbf9addf6f01a03c644d6bd5e1d1e.
I've suggested this in my review (below).
stuitje
reviewed
Jun 5, 2026
Contributor
There was a problem hiding this comment.
In all the other API references I use the table style for docstrings which is in my opinion much better readable.
Suggested change
| docstring_section_style: table |
nichollsh
requested changes
Jun 5, 2026
Member
nichollsh
left a comment
nichollsh
left a comment
There was a problem hiding this comment.
Thank you for these changes, @timlichtenberg, since I understand that this PR reflects an enormous amount of effort on your part over several months, across multiple modules. Fantastic work all around!
I am overall happy with the direction and intention of these changes. They include important physics that we sorely need to model. I do have a few questions about the choices regarding simulating (im)miscibility with the Rogers2025 parametrisation.
There are also various 'wrappers' in this PR which aim to mitigate some weird behaviours arising from AGNI. I shall try to address these on the AGNI side too.
Unfortunately, I have not (yet) had time to try the new ./install.sh script. This script seems like a good approach and I have some suggestions here for the resolving the issues that others have identified. I will update this comment with any other issues I find when following this new install script.
I have made several other comments and suggestions below.
| # Tracks nichollsh/AGNI main. Bump after every AGNI release that | ||
| # PROTEUS depends on. | ||
| url = "https://github.com/nichollsh/AGNI.git" | ||
| ref = "b06a3fed51e0f1610556634d5b5a5e0425428f0e" |
Member
There was a problem hiding this comment.
Suggested change
| ref = "b06a3fed51e0f1610556634d5b5a5e0425428f0e" | |
| ref = "d6fe2d0f7c0b0251a960d11db149ec8708704f09" |
| # under socrates/ by tools/get_socrates.sh; the build output is what | ||
| # the AGNI / JANUS atmosphere modules link against at runtime. | ||
| url = "https://github.com/FormingWorlds/SOCRATES.git" | ||
| ref = "e7133ff7388847c7939b38572c6e91cd05d5b755" |
Member
There was a problem hiding this comment.
This hash needs to be updated because it is causing problems when installing SOCRATES. See discussion here: nichollsh/AGNI#204 (comment)
Suggested change
| ref = "e7133ff7388847c7939b38572c6e91cd05d5b755" | |
| ref = "fe0c4a486f1dbf9addf6f01a03c644d6bd5e1d1e" |
Comment on lines
+15
to
+17
| description: Julia version (AGNI requires 1.11.x) | ||
| required: false | ||
| default: '1.11.2' |
Member
There was a problem hiding this comment.
Suggested change
| description: Julia version (AGNI requires 1.11.x) | |
| required: false | |
| default: '1.11.2' | |
| description: Julia version | |
| required: false | |
| default: '1.12.6' |
AGNI now supports Julia 1.12 and the 1.13 RC.
| set -euo pipefail | ||
|
|
||
| if ! command -v julia >/dev/null 2>&1; then | ||
| echo "ERROR: julia is not on PATH. Install Julia first (see docs/How-to/installation.md)." >&2 |
Member
There was a problem hiding this comment.
Suggested change
| echo "ERROR: julia is not on PATH. Install Julia first (see docs/How-to/installation.md)." >&2 | |
| echo "ERROR: julia is not on PATH. Install Julia first (see https://www.h-nicholls.space/AGNI/dev/howto/getting_started/)." >&2 |
Member
There was a problem hiding this comment.
Why remove this file? It's an important part of the CHILI simulations for generating output files in the format specified by the protocol.
Comment on lines
+72
to
+75
| Phi_global = float(hf_row.get('Phi_global', 1.0)) | ||
| gravity = float(hf_row.get('gravity', 9.81)) | ||
| R_int = float(hf_row.get('R_int', 6.371e6)) | ||
| area = 4.0 * np.pi * R_int**2 |
Member
There was a problem hiding this comment.
Should not need default fallbacks here since these variables should always be set.
| kg_atm = f_atm * kg_total | ||
| kg_liquid = f_dissolved * kg_total | ||
| kg_solid = 0.0 | ||
| mmw = _MMW.get(species, 0.028) |
Member
There was a problem hiding this comment.
Weird to need a default here? If we are missing a species' MMW value, it should be added to the dictionary. Alternatively, we can calculate MMW from the molecule's atoms dynamically (AGNI does this already, for example).
Comment on lines
+140
to
+144
| if P_total > 0: | ||
| mmw_sum = sum(hf_row.get(f'{s}_vmr', 0.0) * _MMW.get(s, 0.028) for s in gas_list) | ||
| hf_row['atm_kg_per_mol'] = mmw_sum | ||
| else: | ||
| hf_row['atm_kg_per_mol'] = 0.028 # default ~N2 |
Member
There was a problem hiding this comment.
More strange behaviour defaulting to N2 here. Doesn't seem necessary or safe. Suggest supporting all species dynamically or throwing an error.
| _exc, | ||
| ) | ||
|
|
||
| global _zalmoxis_fail_count |
Member
There was a problem hiding this comment.
I strongly caution against the use of global variables where possible. They can easily cause problems, for a variety of reasons, such as when running multiple instances of PROTEUS at the same time. Could you somehow rework this to be a local variable that is passed around appropriately?
| _output_zalmoxis_path = os.path.join(outdir, 'data', 'zalmoxis_output.dat') | ||
| if os.path.isfile(_output_zalmoxis_path): | ||
| try: | ||
| shutil.copy2(_output_zalmoxis_path, _output_zalmoxis_path + '.prev') |
Member
There was a problem hiding this comment.
Is it necessary to pass information to/from Zalmoxis via file operations? Both PROTEUS and Zalmoxis are Python-based codes, so it seems that the arrays could probably be exchanged in memory.
Performing these file operations is going to add a lot of overhead, and reduce performance.
@egpbos was doing some work on this recently.
stuitje
reviewed
Jun 5, 2026
| This run takes 30 minutes to several hours depending on hardware. | ||
| The initial Zalmoxis structure solve (~10-20 min) is the slowest | ||
| phase. Monitor progress with | ||
| `tail -f output/tutorial_earth/proteus_00.log` (the log appears |
Contributor
There was a problem hiding this comment.
I would move this tail -f output/tutorial_earth/proteus_00.log out of the warning box and clearly in the normal text. You can easily read over it if it's in the warning box (I did...)
stuitje
reviewed
Jun 5, 2026
Contributor
There was a problem hiding this comment.
I got an error for this tutorial:
RuntimeError: Zalmoxis did not converge: pressure=True, density=True, mass=False. Final M=1.94e+24 kg, R=6.43e+06 m. EOS: core=PALEOS:iron, mantle=PALEOS-2phase:MgSiO3.
When I did some digging I saw many of these lines:
Error with tabulated EOS for melted_mantle at P=2.48e+06 Pa, T=5066.354867712648: /home6/s4339150/FW/FWL_DATA/zalmoxis_eos/EOS_PALEOS_MgSiO3/paleos_mgsio3_tables_pt_proteus_liquid.dat not found.
Error with tabulated EOS for melted_mantle at P=2.48e+06 Pa, T=5066.354867709451: /home6/s4339150/FW/FWL_DATA/zalmoxis_eos/EOS_PALEOS_MgSiO3/paleos_mgsio3_tables_pt_proteus_liquid.dat not found.
On a quick look, I don't see a CLI command (proteus get) for the PALEOS eos tables; should they be downloaded manually from zenodo?
Perhaps this should this be mentioned e.g. in the prerequisites? + perhaps it would be a good idea to add this to the CLI?
Unless I missed something of course, I didn't have time to dig into this completely.
stuitje
reviewed
Jun 5, 2026
| ```bash | ||
| conda activate proteus | ||
| mkdir -p output/tutorial_earth | ||
| nohup proteus start --offline -c input/tutorials/tutorial_earth.toml \ |
stuitje
reviewed
Jun 5, 2026
|
|
||
| <figure markdown="span"> | ||
| { width="100%" } | ||
| <figcaption>Multi-panel overview of the PROTEUS Earth analogue tutorial run. |
Contributor
There was a problem hiding this comment.
For clarity (very long caption):
Suggested change
| <figcaption>Multi-panel overview of the PROTEUS Earth analogue tutorial run. | |
| <b><figcaption>Multi-panel overview of the PROTEUS Earth analogue tutorial run.</b> |
Contributor
|
First of all, @timlichtenberg thank you for all the work you put into this PR!! I have looked only at the documentation so far, and I have run the installation instructions + two tutorials. I can review more this weekend but today I have sadly run out of time. I already added some comments above, but here are my general impressions. DocumentationGeneral
Home
How-to guides
Explanations
Reference
Tutorials
|
stuitje
reviewed
Jun 5, 2026
|
|
||
| <figure markdown="span"> | ||
| { width="100%" } | ||
| <figcaption>All-dummy tutorial output. (a) Heat fluxes: the interior and |
Contributor
There was a problem hiding this comment.
For clarity:
Suggested change
| <figcaption>All-dummy tutorial output. (a) Heat fluxes: the interior and | |
| <figcaption><b>All-dummy tutorial output.</b> (a) Heat fluxes: the interior and |
BOREAS is no longer a mandatory dependency of PROTEUS. It is only needed when a configuration sets escape.module = "boreas", so installing it for every user is unnecessary. What changed: - Drop the boreas git dependency from [project] dependencies and record the pin in [tool.proteus.modules.boreas] instead (full commit SHA). - Add tools/get_boreas.sh to clone and install it on demand, following the pattern used for the other optional modules. - Install BOREAS explicitly in the CI setup so the escape tests keep running. - Point the config validator and the runtime guard at the install script when escape.module = "boreas" and the package is missing. The package is not published on PyPI under a usable name (the PyPI "boreas" name belongs to an unrelated project), so a direct git URL was previously the only way to pull it. That URL also blocks publishing fwl-proteus to PyPI, which the module pin avoids. BOREAS is no longer listed in the user documentation. The config option and the install script stay in the source for anyone who needs them.
VULCAN and atmodeller are alternative physics backends that a standard PROTEUS run does not need: the default outgassing module is calliope and atmospheric chemistry is off by default. Both are GPL-3.0 licensed, while the PROTEUS core is Apache-2.0, so installing them for every user is both unnecessary and a licensing burden. VULCAN and atmodeller: - Move fwl-vulcan and atmodeller out of the mandatory dependencies into [project.optional-dependencies] extras, keeping their version pins. - Install on demand with pip install "fwl-proteus[vulcan]" or "fwl-proteus[atmodeller]". JAX stays mandatory; the Aragog interior solver, which is the default, needs it. - When a config selects one of these backends but the package is missing, the loader now explains how to install it and states the GPL-3.0 license. - CI installs both extras so their tests still run, and proteus doctor no longer reports fwl-vulcan as missing when it is not installed. - Both remain documented as optional modules. BOREAS installer: - Add error handling to tools/get_boreas.sh so a failed clone, checkout, or install aborts loudly instead of printing "Done!". - Drop the unpinned git fallback from the install hint; the script reads the pinned commit from pyproject.toml. - Fix the optional-dependency example in the testing guide to name a Python-importable package.
Moving fwl-vulcan and atmodeller to optional extras left two ways a default `pip install fwl-proteus` could fail, neither visible in CI (which installs the extras). This restores a working default install. Default interior solver: - The default Aragog interior backend runs on JAX and its modules are equinox objects, so equinox is required by a standard run. It used to arrive only as a transitive dependency of atmodeller; once atmodeller became optional, a default install no longer had equinox and the default interior solver would fail to import. Declare equinox==0.13.2 as a mandatory dependency and correct the comment: the jax<0.10 ceiling exists for the default Aragog stack, not only for atmodeller. Reference configuration: - input/all_options.toml selected atmos_chem.module = "vulcan", so loading it required the optional VULCAN package. Since `proteus start`, `proteus install-all`, and `proteus update-all` all read this file, a default install hit an ImportError. Default atmos_chem to "none"; VULCAN stays documented and selectable. License wording: - The optional-backend notices no longer imply that declining VULCAN and atmodeller yields a license-clean install: the default install already includes other GPL-3.0 dependencies (such as fwl-aragog). The notices state each package's GPL-3.0 license without the misleading contrast. Robustness: - Add a check that the extras named in the CI setup action are real pyproject extras, so a typo cannot silently skip the optional installs. - Add a check that doctor's mandatory package list excludes the optional backends and keeps the framework packages. - Drop vulcan from the schema cross-product enumeration, matching how the other optional backends are handled. - Harden tools/get_vulcan.sh so a failed clone, checkout, or build aborts loudly, matching tools/get_boreas.sh. - Derive the optional version badges from the pyproject extras so a version bump updates the docs from a single source.
VULCAN previously had two independent version sources: the fwl-vulcan floor in the optional extra, and a separate commit SHA in [tool.proteus.modules]. The SHA tracked the default branch (main), not the released tag, so the editable checkout that tools/get_vulcan.sh produced could diverge from the PyPI release a default install resolves. Bring VULCAN onto the same single-source pattern as fwl-aragog and fwl-zalmoxis: remove the [tool.proteus.modules.vulcan] entry, and have tools/get_vulcan.sh check out the git tag matching the fwl-vulcan floor in pyproject.toml. The editable checkout and the PyPI release now stay in lock-step, and there is one place to bump the version. A test guards that VULCAN keeps no module-level SHA pin.
The documentation is built and served with zensical, which manages file watching itself. The leftover `watch:` list is a plain-mkdocs serve-time directive that the zensical build does not use, so drop it.
The `watch:` block is the zensical serve-time live-reload directive that every PROTEUS-ecosystem docs config keeps (aragog, CALLIOPE, Zalmoxis, JANUS and the rest carry the same src/README/CONTRIBUTING/CODE_OF_CONDUCT list). Dropping it left PROTEUS as the only repo without one, for no gain: `zensical build` ignores the block, so it is build-neutral, and it lists no `docs/` entry, so it does not trigger the empty-build issue that only a `docs/`-under-watch path caused (that was addressed separately). Restore it so local `zensical serve` live-reloads on source and root-document edits, consistent with the rest of the ecosystem.
The PR coverage check posted a "Coverage Notice" comment on every run where the estimated-total coverage was below target, creating a fresh comment each time rather than updating one, so a single PR accumulated a long string of duplicates. The number is also routinely understated: when the nightly coverage artifact is unavailable the estimated total collapses to unit-only coverage, which cannot reach the full-gate target, so the notice fired as a non-blocking warning that did not reflect real coverage. Remove the comment-posting step. The coverage gate itself is unchanged: the job still computes the estimated total, still fails a ready-for-review PR when coverage genuinely drops, and still writes the numbers to the run summary. A real shortfall therefore shows up as a failed check instead of a repeated PR comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is ready for review. It is the larger, integrating half of a coupled change with FormingWorlds/CALLIOPE#20 (chemistry). CALLIOPE#20 has merged and shipped as
fwl-calliope26.06.01, which this PR pins, so the dependency is satisfied; see the review guidance and timeline below.Review timeline. I would like the review to take about a week. Please send feedback by Friday 5 June; I will incorporate comments and aim to merge by Monday 8 June.
Reviewers: @maraattia, @nichollsh, @rdc49, @planetmariana, @EmmaPostolec, @stuitje, @egpbos, @MarijnJ0, @Emeline0110 @IKisvardai. General comments are welcome, not just on the points I flag below.
How to review. This PR is too large to read line by line, and most of it is documentation and tests. Please do not try to read every diff. Instead, serve the docs locally (
zensical serve, seedocs/How-to/documentation.md) and follow a few of the tutorials end to end to confirm they work for you. For the implementation itself, please focus only on the specific places where I tag you in the text below; that is where I want eyes on the actual code.Paired PR and merge order. This PR depends on FormingWorlds/CALLIOPE#20, the chemistry half of the oxygen accounting. CALLIOPE#20 has merged and is released as
fwl-calliope26.06.01; this PR pins that release (fwl-calliope>=26.06.01), so the dependency is satisfied and this PR is ready to merge on its own.Description
tl/interior-refactorhas been in development for a few months. The change the branch is named for is the incorporation of two new Python interior backends, Aragog (thermal evolution) and Zalmoxis (interior structure), and the reorganisation of the interior into two clean axes around them. Built on top of that are a one-command automatic installer, central module version pinning, a new config format version, a documentation restructure, a test-framework overhaul, a full set of dummy module backends, atmodeller as a fully supported outgassing backend, and the #677 whole-planet oxygen accounting fix. The branch also merges in the ten PRs that landed onmainin the meantime, so it stays current withmainrather than diverging from it; that is catch-up, not new capability. Most of the change is the docs overhaul, the migrated config files, and the expanded test suite, not new physics.Interior backends: Aragog and Zalmoxis
This is the change the branch is named for. The interior is two clean axes:
interior_energetics/for thermal evolution (spider,aragog,boundary,dummy) andinterior_struct/for radial structure (zalmoxis,dummy, plus a SPIDER-internal option). Energy bookkeeping uses frozen-mass conservation (E_residual_cons_J). Architecture overview:docs/Explanations/code_architecture.md.Aragog (interior energetics, the default). Aragog is a pure-Python interior thermal-evolution model in the entropy formulation: the same magma-ocean physics as the compiled C SPIDER, reimplemented around an entropy solver. The wrapper in
src/proteus/interior_energetics/aragog.pydrives Aragog'sEntropySolverover an entropy-based equation of state (EntropyEOS), with the mantle EOS tables materialised underdata/spider_eosand cached by content fingerprint so repeated solves in one process reuse them. Aragog integrates the entropy evolution with a stiff ODE solver (SUNDIALS CVODE) and a JAX-assembled Jacobian, and derives the core density from the mesh. It is the defaultinterior_energetics.module, withspider, the boundary backend, anddummyas the alternatives. The main practical gain is that Aragog needs no PETSc and no C build, so the common interior path is pure Python; SPIDER stays fully supported, and the two can be run at the same initial condition as a cross-implementation check. Aragog installs as an editable sibling checkout (aragog/) inside the PROTEUS root, withfwl-aragogpinned in[project] dependenciesas a fallback. Config:[interior_energetics], seedocs/Reference/config/interior.md.Zalmoxis (interior structure, the default). Zalmoxis solves the planet's radial interior structure (density, pressure, gravity, radius) self-consistently. The wrapper in
src/proteus/interior_struct/zalmoxis.pycalls Zalmoxis' solver over its EOS tables and melting curves and iterates the density profile to convergence with a Picard scheme. To keep that iteration cheap across the many structure solves in a run, the wrapper seeds each solve from the previous converged density profile, keyed on planet identity (mass_tot,core_frac,mantle_mass_fraction) so a multi-planet driver never seeds one planet from another; the seed only accelerates convergence and never changes the converged answer. Zalmoxis is the defaultinterior_struct.module; the alternativedummystructure uses the Noack & Lasbleis (2020) analytic scaling laws (calibrated for 0.8 to 2 M_Earth), and the energetics solver can either take the Zalmoxis structure as its external mesh or use an Adams-Williamson density profile. The interior radius and density profile Zalmoxis returns feed both the energetics solve and the dry-mass target that the volatile and oxygen accounting is computed against. Zalmoxis installs as an editable sibling checkout (Zalmoxis/), withfwl-zalmoxispinned as a fallback. Config:[interior_struct], seedocs/Reference/config/interior.md. @maraattia, please review the Zalmoxis implementation specifically: go through the[interior_struct]parameters ininput/all_options.tomland tell me whether each one is understandable and useful as named and documented, or should be renamed, defaulted differently, or dropped.Outgassing backend: atmodeller
atmodeller is a fully supported alternative to CALLIOPE for the outgassing and equilibrium-chemistry step, selectable with
outgas.module = "atmodeller". It wraps the atmodeller package (Bower et al. 2025, ApJ 995:59), a JAX-based solver for thermodynamically consistent magma-ocean and atmosphere equilibrium with real-gas equations of state. Both outgassing backends honour the sameplanet.fO2_sourcemodes, including the authoritative-Ofrom_O_budgetpath, so the whole-planet oxygen accounting behaves identically whichever you select. Config lives under[outgas](shared solver parameters) and[outgas.atmodeller]. @Emeline0110, please run the Earth analogue tutorial twice, once withoutgas.module = "atmodeller"and once withoutgas.module = "calliope", and tell me whether both work for you. Seedocs/Reference/config/escape_outgas.md.Automatic install path
bash install.shis a single-command installer for the whole ecosystem, in place of the manual sequence of clone, build, and pip-install steps. From a fresh conda environment it runs idempotently through pre-flight checks (OS, disk, Python 3.12, system libraries), Julia setup with a version pin (1.11), theFWL_DATAandRAD_DIRenvironment variables, the SOCRATES build, AGNI and FastChem, editable installs of every Python submodule and PROTEUS itself, reference-data downloads, and a finalproteus doctorverification. It is safe to re-run: completed stages are skipped. Data scope is selectable (--all-data,--no-data, default essential set) and-iruns it interactively. For an environment where PROTEUS already imports,proteus install-allperforms the same setup from the CLI (--export-envpersists the environment variables);proteus doctordiagnoses an existing install (environment variables, submodule presence and editable git hashes, SOCRATES and AGNI,FWL_DATA); andproteus update-allrefreshes the submodules, reference data, and PROTEUS itself in place. @egpbos and @nichollsh, please stress-test the installer,install-all,doctor, andupdate-allon a clean machine and on a cluster and flag anything that breaks. Seedocs/How-to/installation.mdanddocs/How-to/doctor.md.Editable install layout
Aragog, Zalmoxis, and VULCAN install as editable sibling checkouts inside the PROTEUS root, matching AGNI / MORS / JANUS / CALLIOPE / ZEPHYRUS. The PyPI pins (
fwl-aragog,fwl-zalmoxis,fwl-vulcan) stay as a fallback forpip install fwl-proteuswithout cloning.proteus doctorreports the editable git hash and dirty state next to the installed version, and a new CI gate verifies the editable copy takes precedence over the PyPI fallback on every PR. New setup scripts:tools/get_aragog.sh,tools/get_zalmoxis.sh. Seedocs/How-to/installation.mdanddocs/How-to/doctor.md.Module version pinning
External module versions are pinned in one place and checked at runtime.
pyproject.tomlcarries a[tool.proteus.modules]table that pins every non-PyPI module PROTEUS clones or builds (AGNI, SOCRATES, SPIDER, VULCAN, LovePy, PETSc) to aurland aref(commit SHA, tag, or branch). The CI composite action and thetools/get_*.shscripts both read this table (viatools/_module_pins.py), so bumping a module is a one-linerefedit and the same SHA always reproduces the same external state, which keeps branch CI deterministic. The Python submodules (fwl-aragog,fwl-zalmoxis,fwl-janus,fwl-calliope, and the rest) carry minimum-version pins in[project] dependencies. At startup,validate_module_versions(src/proteus/utils/coupler.py) compares the installed version of each active module against its required minimum, with a comparison that handles both semver and CalVer (year/month/day), and refuses to run on an out-of-date module. PROTEUS itself is versioned with setuptools-scm CalVer (version_scheme = "no-guess-dev"), matching the Aragog / Zalmoxis / CALLIOPE ecosystem convention. Seedocs/Reference/module_versions.md.Config format and defaults
PROTEUS stamps a
config_versionon every config. The current format is3.0(CURRENT_CONFIG_VERSIONinsrc/proteus/config/_config.py); it defaults to3.0, and a config that explicitly declares a different version is rejected with an actionable error that points atinput/all_options.toml. Version 3.0 covers the current layout: the interior split into[interior_energetics]and[interior_struct], the volatile inventory under[planet.elements], and the current section names. The shippedinput/configs and the test configs are all on 3.0.Every config parameter has a default. The config dataclasses in
src/proteus/config/define a safe Earth-like or dummy value for each field, so a minimal config loads to a known state and only the fields you want to change need to appear (tests/config/test_defaults.pyguards this). The per-parameter defaults are documented in thedocs/Reference/config/pages, andinput/all_options.tomlis the full enumeration of every option with recommended starting values. Seedocs/How-to/config.md.Config migration tool
tools/migrate_config_v2_to_v3.pyconverts an existing 2.0 config to the 3.0 format. It moves every field to its new location and pins any default that changed between the versions, so a converted config reproduces the original run instead of silently picking up new 3.0 defaults.--gridhandles grid configs, andtools/migrate_equivalence_check.pyplustests/tools/verify that the shipped 2.0 configs convert faithfully.@nichollsh @IKisvardai, this is the migration tool Harrison suggested at today's dev meeting. Please run it on one of your own 2.0 configs and let me know if anything looks off.
Documentation restructure
The documentation follows a Diataxis layout: How-to guides (install,
proteus doctor, configure, run, clusters, development), Tutorials (all-dummy quick start, Earth analogue, parameter-grid sweep, Solar System CHILI intercomparison), Explanations (model description, coupling loop, code architecture reflecting the interior two-axis split, dummy modules, test framework), and Reference (per-section config reference, melting curves, module versions, output format, API, and a new per-source Validation tree). The Validation pages anchor each physics source to a published benchmark or analytical limit and are wired into the nav. Installation, local-machine, and diagnose/update guides are included. The docs build with Zensical (zensical serve/zensical build). Entry points:docs/index.md,docs/getting_started.md.Test framework and coverage
The branch brings PROTEUS onto the ecosystem test standard. Every test file carries a module-level tier marker (
unit,smoke,integration,slow) with a wall-time budget; physics tests additionally carryphysics_invariant(asserts conservation, positivity, boundedness, monotonicity, or symmetry) andreference_pinned(pins against a published benchmark, an analytical limit, or a SPIDER-versus-Aragog cross-check). Markers are strict (--strict-markers), and an AST linter (tools/check_test_quality.py) blocks single-assert tests, weakis not Nonechecks, missing docstrings, and float==comparisons against a one-way baseline. CI runs a fast PR gate (unit + smoke on Linux and macOS on every push) and a nightly full gate (adds integration and slow tiers). Line coverage auto-ratchets toward the 90% ecosystem ceiling and is never manually decreased (tools/update_coverage_threshold.py). The suite mirrorssrc/proteus/one-to-one, validated bytools/validate_test_structure.sh. A test-count badge on the test framework page reports the live total, unit, and combined integration-tier counts, regenerated frompytest --collect-onlyduring the documentation deploy and served from the published docs site. Seedocs/How-to/testing.mdanddocs/Explanations/test_framework.md.Dummy modules
Every module slot has a
dummybackend: interior energetics and structure, atmosphere climate and chemistry, outgassing, orbit, star, and escape. Each replaces the full physics with a minimal parameterisation that captures the qualitative behaviour (a planet cools, volatiles outgas, the atmosphere radiates) without external solvers, compiled code, or reference data. They serve two purposes. First, testing: an all-dummy run exercises the entire coupling architecture (the helpfile data bus, timestep control, convergence checks, output pipeline) in well under a minute, which is what the fast unit tests and the quick-start tutorial use, and what keeps coupling bugs separable from solver bugs. Second, physics grounding: the dummy backends give analytical end-member behaviour that the production modules should reproduce in the appropriate limits, a zeroth-order sanity check on any full-solver result. They are not meant for quantitative science. Seedocs/Explanations/dummy_modules.mdand the all-dummy quick-start tutorial.Incorporated
mainPRsEach incorporated PR keeps the original author's physics and tests. Please check that your code still behaves as you intended on this branch, and tell me if you have questions or spot anything off.
@nichollsh (seven PRs): #661 (get_socrates script), #659 (CHILI intercomparison data and scripts), #665 (aerosol support and chemistry plot rework), #669 (doctor command and SOCRATES version check), #671 (VULCAN as a Python package), #673 (environment and install simplification), and #675 (BayesOpt rewrite, AGNI grey-gas RT, timestep-overshoot fix,
prevent_warmingboundary-layer fix, config tolerance migration). #675 is staged into eight atomic commits so each sub-feature is independently verifiable. Two things to confirm: (a) in #665 the aerosol schema is present but the mass mixing ratio is pinned at 0.0 with no source term plumbed, so aerosols are radiatively inert even withaerosols_enabled=True; is that the intended interim state? (b) in #659 the xfail marker describes a cattrs silent-drop, but that case currently passes; can you confirm the original intent? I also deliberately left out a few items from #675 (thedt.adaptive/dt.proportionalnested-class refactor, the parent-to-child schema move forp_top/p_obs/spectral_*/num_levels, and several renames and removals that postdate the fork) because they would either rewrite every[params.dt]block or silently revert load-bearing features; happy to walk through that.@planetmariana: #658 (library of solidus/liquidus parameterizations, selected via
melting_dir). Please confirm the parameterization selection behaves as you expect under the new interior split.@rdc49: #668 (boundary interior module, the fourth energetics backend). It implements the Schaefer et al. (2016) parameterised-convection magma-ocean model: the Rayleigh-Nusselt mantle flux
q_m, the mantle potential-temperature evolution (their Eq. 15), and the surface energy balance4 pi R^2 (q_m - F_atm)(their Eq. 20), all of which match the published equations. One small reporting point to confirm: the helpfileF_intcolumn is set toF_atm, while the convective fluxq_mthat drives the evolution is written to the backend's own log; is that the reporting you want, or shouldF_intcarryq_m? Otherwise, please confirm it runs as intended on this branch.(#662, FastChem install docs, is mine.)
Whole-planet oxygen accounting (closes #677)
#677 reported
M_atm > M_planetfor volatile-rich cases (highH_ppmw), with the summed volatile inventory also inconsistent with the bulk mass.Oxygen is a tracked element in PROTEUS-side accounting alongside H/C/N/S. The chemistry step is unchanged (CALLIOPE and atmodeller equilibrate against the fO2 buffer), and the atmospheric and dissolved O mass they produce is counted in
M_ele, subtracted from the Zalmoxis dry-mass target, and included in the proportional escape distribution. Aplanet.elements.O_modefield selects how the O budget is set:"ppmw","kg","FeO_mantle_wt_pct"(a petrology-friendly unit, sets the volatile O budget only, does not change the PALEOS EOS density), or"ic_chemistry", which defers the IC budget to CALLIOPE's first equilibrium and is the default, so a config that says nothing about oxygen keeps the buffered-mode behaviour.tools/migrate_oxygen_mode.pywrites an explicitO_modeinto the[planet.elements]blocks underinput/andtests/. A runtime invariant enforcesM_atm <= M_planet, and an IC consistency check hard-fails when the user O budget diverges from CALLIOPE's equilibrium value by more than 50%. The chemistry half lives in FormingWorlds/CALLIOPE#20. Config reference:docs/Reference/config/planet.md.One caveat worth a comment.
FeO_mantle_wt_pctsets the volatile O budget only; it does not change the mantle EOS density, since PALEOS still uses its built-in FeO content. That makes it a leaky unit for now, and I would like a view on whether to keep it that way or make it strict once PALEOS density responds to user-set mantle composition.Please try this once in an extreme volatile-rich regime (water-dominated, high H budget): @IKisvardai, @nichollsh, @EmmaPostolec, @planetmariana. And @IKisvardai, @EmmaPostolec, @planetmariana: please send me one representative TOML from your current work, so I can convert your v2 configs to the v3 layout and run the new accounting on your own planet scenarios.
Radial fO2 via Fe3+/Fe2+ tracking (#653)
@planetmariana, the interface for your radial ferric/ferrous framework is already scaffolded.
planet.fO2_sourceis an enum:"user_constant"(the fO2-buffered source, the default),"from_O_budget"(authoritative O, fO2 derived), and a reserved"from_mantle_redox"member that currently raises a clear "reserved for issue #653, not yet wired into the runtime" error (src/proteus/config/_planet.py). To wire in your branch you add thefrom_mantle_redoxruntime path to the outgas dispatch insrc/proteus/outgas/calliope.py(andatmodeller.py), keyed onconfig.planet.fO2_source, alongside the two existing source branches. Because O is tracked end-to-end, the bookkeeping prerequisite for a self-consistent Fe-derived fO2 is in place. Could you check that this hook works for your approach in principle?Bug fixes
src/proteus/orbit/satellite.py):Ltotuses the satellite mass in the orbital square-root, following Korenaga 2023 (Icarus 400, Eq. 60); the planet-mass form would inflate L byM_planet / M_satellite(~81 for Earth-Moon) and corrupt the dω/dt and da/dt evolution. The Earth-Moon value is pinned against Korenaga 2023. @MarijnJ0, this is your module; please confirm the form is right. Seedocs/Validation/orbit/satellite.md.T_magma, so the mantle skin layer stays consistent across the resume boundary; residual ~0.9 K.Validation of changes
Unit and smoke tests green on Linux and macOS (Python 3.12) on every push. The full nightly (unit, integration, and slow tiers, including the zalmoxis-coupled structure run on both platforms) is green on the current head, with combined coverage above the 90% ecosystem target.
ruff checkandruff format --checkclean. End-to-end check of the #677 fix atH_ppmw = 2e5, Earth mass, fO2 = +4:M_planet = mass_tot * M_earthexactly,M_atm / M_planet = 0.7737, mass-conservation invariant holds.All four tutorials run end to end, in particular the Solar System CHILI intercomparison, which exercises the full interior, structure, outgassing, and atmosphere coupling across the terrestrial planets. Beyond the tutorials, the branch has been run on a range of super-Earth configurations. It has not been stress-tested across every possible configuration combination.
Issues closed
Closing keywords auto-close the PROTEUS issues on merge. GitHub does not auto-close across repositories, so I will close the two Zalmoxis issues by hand once this PR merges.
M_atm <= M_planet. See the oxygen-accounting section above.interior_struct.module, solving the radial structure at runtime.core_frac_mode = "mass"the core/mantle mass split follows self-consistently from the structure and the EOS.Phi_global_vol) directly into the helpfile, so the column carries the true volumetric melt fraction.F_atm, refreshed each coupling iteration, so the integrated surface flux equals the atmosphere's F_atm. Per-layer fluxes still differ from the surface value, as they physically should.codecov.ymlcarries separateunit-testsandnightlyflags withcarryforward: true, so a fast-suite PR upload is compared on equal footing with a base that also carries nightly coverage. Both codecov checks pass on this PR.interior_struct.zalmoxis.update_interval > 0, PROTEUS recomputes the Zalmoxis structure as the evolution proceeds, throughupdate_structure_from_interior, gated by elapsed time and changes inT_magmaand melt fraction.input/all_options.tomluses the built-in PALEOS EOS, so a fresh install needs no external RTPress100TPa tables, and the file runs as a standard integration test (tests/integration/test_integration_std_config.py) that is green in the nightly.Related
Next up
The next major effort is a shared input/output layer for the ecosystem (working name fwl-io): a single library for reading and writing PROTEUS data products against a central data manifest, so I/O, reference-data resolution, and provenance live in one place across all modules. I am deliberately not addressing that here. This PR lays the basis for it: the helpfile data bus, the central module-pin manifest in
pyproject.toml, the config-version stamp, and the editable-install layout are the pieces fwl-io will build on. It comes next, as its own PR, starting with #605 (universal downloader utilities).This also needs to land on
mainand be stress-tested against the wider set of configurations the group runs. I expect some issues to surface in configs not covered here, and I will work through them as fast as I can over the next few weeks.Escape modeling needs further development, and that is in progress: a BSc group project and @EmmaPostolec's PhD thesis are tackling it more closely, including a more physical, element-fractionated treatment.
Feature requests in this review are welcome. I will weigh each against scope: some can fold into this PR, others I will defer to a follow-up so this one can land on the timeline above.
Checklist