Mode A pre-build sweep runner#135
Open
erikfilias wants to merge 1 commit into
Open
Conversation
…_Cases) run(cases, solver_name, mode='pre-build', backend=...) drives many cases through openTEPES_run, one independent build-and-solve per case, over serial, multiprocessing, or joblib backends. A Case points at one input source (CSV dir or .duckdb) plus an optional output dir and label. The runner reads back each case's openTEPES_run_status_*.json and returns one summary dict per case in input order, so nothing has to pickle the Pyomo model across workers; a failing case is captured as status='error' instead of aborting the sweep. Additive only. New tests cover serial parity, multi-case order/isolation and a captured error, the multiprocessing backend, and the mode/overlay/backend guards.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Mode A of the parallelisation plan in the architecture:
a small, additive sweep runner over the existing
openTEPES_run.What it adds
openTEPES_Runner.py—run(cases, solver_name, backend=...)runs many cases throughopenTEPES_run, one independent build-and-solve per case, over aserial(default, no extra dependency),multiprocessing, orjoblibbackend.openTEPES_Cases.py—Casenames one input source (CSV directory or.duckdb) plus an optional output directory and label.Design notes
openTEPES_runreturns the Pyomo model, which cannot cross a worker process boundary, so the runner reads back each case'sopenTEPES_run_status_*.jsonand returns one summary dict per case, in input order, for every backend.status="error"so one bad pathway does not abort the rest of the sweep.openTEPES_runcall.Tests
New tests in
tests/test_run.py: serial parity vs a direct run, a multi-case sweep (order, output isolation, captured error), the multiprocessing backend, and the mode/overlay/backend guards.