Replace custom validators with attrs.validators - #835
Conversation
* first pass at EAF performance model * updated to eaf model based on peer programming * current state of eaf scrap perf * Adding in compressed gas storage capex (not complete) * scrap only performance model with unit test * update outputs to be based on steel produced * change output to slag_out * add notes * docstrings * cmu eaf dri wip * Capex finished * Opex added, compressor model crashing * wip dri * Temporarily using pysam wind for speed * Calculating storage cost in example, need to fix tests * First test passing * Changed compressor types, capex_per_kg test failing * Storage capex per keg test fixed * Reverting example tech_config * Docs + changing h2 storage constants to uninstalled (installed is calculated) * Fixing test values * Moving images * Changelog update * Update commodity from pig iron to sponge iron (NatLabRockies#670) * update from pig iron to sponge iron * changelog * update lco naming in test * bugfix when calling check_inputs when using round trip efficiency * Added a quick test for the bugfix * add nuclear to electricity producing tech and update electricity commodity error message and add error message tests * update changelog * Updating test_hydrogen_storage * Fixing h2_storage_cost * Fixing double CEPCI inflation in tank cost * unit conversion correction * unit test scrap * cmu eaf with dri model and tests * cost model * cost model test * more unit tests * clean up and changelog * Added feedstock consumption profiles as outputs to electrolyzer (NatLabRockies#690) * added feedstock consumption outputs to electrolyzer * changed kg H2O to galUS * Changed many demands to set_points * Updated changelog * update capex value * change dri comp input * removed commodity_name from h2_storage cost config and updated formatting for doc string * desc * cost model default steel capacity * change name to capacity * error for production and capacity * added inputs * adjust molar mass constants * Add PLM optimized dispatch controller, example 34, and tests * Make sure the example runs * WIP: Addressing minor PR comments * Adding CMU examples * Peak load management heuristic control (NatLabRockies#641) * add peak load management control * add time series generation method as a utility function * added common compute checks and parameters to baseclass for openloop controllers Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: kbrunik <102193481+kbrunik@users.noreply.github.com> * Added steel CMU doc page * Added iron CMU examples * Updating test value after fixing cold DRI logic * more sig figs * doc mods * Add docs * minor cleanup to pose_optimization * updated call to make_unique_case_name * Addressing Jasa comments * Update docs and add figure * Fully removed DV counting method * removed get_number_of_design_variables method * Edit docs page * Edit docstrings to include raises * Updating example * Adding more flexible pricing schemes * Add integartion tests * Add PLM optimized dispatch controller, example 34, and tests * Make sure the example runs * Add docs * Add docs * Update docs and add figure * Edit docs page * Edit docstrings to include raises * FIX formatting in supported modesl * Fix formatting in test_all_ex * Addressing PR comments * Fix formatting in supported_models * More formatting fixes * Updated example test values * Change code to handle varibale timesteps * Reverting errants changes * Ability to have a user-defined resource model (NatLabRockies#698) * added ability to specify custom resource model * updated to remove nicknaming of custom models * updated resource index to show setting resource data with set_val --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * add setpoint to fuel cell * docs and changelog * Pre-commit fixes * Remove notebook * Add min separation input and make sure methods run correctly * addressing PR feedback * Fix bug in peak window start time definition in example * Change user defined param fromn_max)hrs to n_max_events * rename n_control_window to n_control_window_hours * update changelog * replace outdated static n2 with dynamic and interactive n2 diagram * update changelog * remove unused image files * Apply suggestions from code review Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Changed buy/sell price mode to user-set instead of inferred * Address reviewer comments * Make description in tech config more specific * Move standard methds to the top and delete demand csv * Add figure in docs * Make sure tests run * Made shared variable within fuel cell to simplify and clarify calcs * Make sure test_all_examples runs * Run pre-commit on all files * Address Elenya's comments * Change n_control_window to n_control_window_hours * Use methods and existing utils for example * added full logic to check technology interconnections * minor cleanups to new methods * renamed new method * updated changelog * updated error handling to aggregate error messages * minor cleanup to new method * Solve for optimal power rather than assuming P_max * Updates to the technology connections check * Give user option to either pick inc/event or /kwh * Add comments for inc/event conversion * bump miniconda action for upgraded node usage * Edit docs to include new definitions * Edit config comments * Minor edits to configs * Add tests for storageperformancemodel * Raise Valueerror if min peak separation < event dur * Remove Valueerror if min peak separation < event dur * Update h2integrate/control/control_strategies/storage/plm_optimized_storage_controller.py Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Update h2integrate/control/control_strategies/storage/plm_optimized_storage_controller.py Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Update h2integrate/control/control_strategies/storage/plm_optimized_storage_controller.py Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Apply pre-commit formatting * update pyomo n2 * Updated pyomo N2 in docs * move H2Integrate model to top-level import (cherry picked from commit 62a574e) * add basic file i/o as top-level imports (cherry picked from commit 12ab0fb) * consolidate docs imports for updates (cherry picked from commit 06abe72) * update changelog (cherry picked from commit 7ba1a0f) * update PR # * fix typo * update hierarchy * move erroneously corrected imports out of code block and back to executed block * add validated yaml loading functions to top-level * Update docs/control/pyomo_controllers.md Co-authored-by: Jared Thomas <jaredthomas68@users.noreply.github.com> * Minor change to trigger CI * Fix SOC min max notation * Change round to ceil in steps_per_event calc * Add subtests * Ability to use timeseries profiles for finance calcs (NatLabRockies#725) * added adjusted capacity factor comp, still need to update connections to it * attempted to add connection but work-in-progress * updated example 17 and added subtests, finished integration * Added docstring to new class * added test for new component * added doc string * Apply suggestion from @johnjasa * added test for error * added more subtests for example 17 * renamed variable to commodity_stream_output * updated documentation * updated test --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: John Jasa <john.jasa@nrel.gov> * Minor changes to comments and subtests * Change round to ceil when computing n_days * Update h2integrate/control/control_strategies/storage/plm_optimized_storage_controller.py Co-authored-by: Jared Thomas <jaredthomas68@users.noreply.github.com> * Add _time_step_bound and make sure init_sc is within bounds * Typo fixes based on PR feedback * bugfix so electrolyzer outputs are real numbers if electrolyzer is never operated * added tests for bugfix * removed out of date TODO statement * Bugfix on storage so storage can't charge with unavailable input commodity (NatLabRockies#664) * added test to highlight error * fixed bug of charging with unavailable commodity * added subtests for charging less than available * added test for technology naming and updated logic for using_feedback_control in storage baseclass setup() * other fixes from merge * fixed example 33 * updated example 34 but tests still failing, doesnt converge * fixed example 34 w help from sanjana * updated example 34 to use electricity feedstock instead of grid --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * added error if technology interconnections is missing (NatLabRockies#744) * Allow for sell and buy price mode of "constant" in grid cost model (NatLabRockies#764) * updated grid cost config and added logic to allow for shape of 1 costs * Added to grid test and changelog --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Added purge gas as a multivariable output (NatLabRockies#760) * Added purge gas as a multivariable output * Added to changelog * Defer heavy imports for faster package load times (NatLabRockies#762) * Working on just-in-time importing * Updated changelog * Updates based on PR feedback * Update h2integrate/core/supported_models.py Co-authored-by: Rob Hammond <13874373+RHammond2@users.noreply.github.com> --------- Co-authored-by: Rob Hammond <13874373+RHammond2@users.noreply.github.com> * Feature: EIA Natural Gas Price Retrieval for Feedstocks (NatLabRockies#719) * add unintegrated, working EIA API data querier * probably working integration into H2I * ensure data are always in monthly format * add openmdao * move feedstocks to its own folder * move natural gas pricing to feedstocks and convert config for compliance * update long comment and start building out NG cost model * update compute docstring and include feedstocks folder in api docs * update inline comment usage * convert price to hourly over full year and update comments/docstrings * remove placeholder for lint ignore * ensure price timeseries matches n_timesteps * add leap year handling for clean 8760s * update changelog * fix typos, make api_key_file consistent, and enable environment variables * update docstrings and spacing * add the EIA model to the docs page * add requests instead of relying on other packages requiring it * add to supported models * update class hierarchy * fix import * fix typo * better column handling for price * add compliant performance model to accompany the cost model * update changelog * update supported models * remove lingering resource reference * update column filtering from previous modification * remove deprecated arg * fix miscellaneous data handling issues * ensure saved data is compatible with reload * fix final data handling issue * update changelog for pr number * add latitude and longitude, and handle no api key file input * add reverse geocoding * update docs for reverse geocoding * add back correct annual reindex logic * add test for all but feedstock cost model and fix minor issues * add missing docstring * add test for configuration class and fix class issues * move the base functionality to separate preprocessing modules * reinstate more descriptive todo comment * remove custom performance model * remove deprecated model * add ability for setup to be used by subclass even if it uses its own config * continue to split out functionality to preprocessing * split out most of data gathering * fix control flow and extra comma * add argument to function * make monthly conversion robust enough for multi-index and add state correction * check for state being kept * move monthly to 8760 conversion to eia preprocessing * remove duplicitous code * change file name and update single site compatibility * update module name in docs * update state retrieval * add docstring * move geospatial tests to preprocessing * move eia api testing to preprocessing and add more tests * update default file handling and state/category sort order * remove pre-return * add data download test * update feedstock config test and fix reverse geocode issue in config * add hourly conversion test and fix issues in function * update feedstock cost model checks * udpate docs * update changelog * fix str-None comparison * add the none default to get data function * fix ci test issues * remove extraneous note * add site_name and handling * add additional site handling * add user warning for modified price * add feedstock model tests * be specific about us-states when applicable * add handling for when no geocoder available * reinstate original file handler * update feedstock import pathway and use subpackage level import, not module * minor warning change --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Make sure `storage_baseclass` handles arbitrary `dt` (NatLabRockies#753) * Make sure storage_baseclass handles arbit dt * Format storage_baseclass * Add ability to handle different rate units * Add comments * Run pre-commit * Expand _time_step_bounds * Add tests for kgs and kwh * Fix comments --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Simplify import pathways for primary models and functionality (NatLabRockies#769) * Moving imports to different init files, updating supported_models * Adding missing init files * Renaming DOE to parameter sweep throughout (NatLabRockies#768) * Renaming DOE to parameter sweep throughout * Updated changelog * System level control framework (NatLabRockies#751) * added test to highlight error * fixed bug of charging with unavailable commodity * added subtests for charging less than available * added test for technology naming and updated logic for using_feedback_control in storage baseclass setup() * other fixes from merge * Add system_level_control to the CI * added control classifiers to key technology models * added curtailable component * added logic in h2imodel to add curtailment component to curtailment control_classifiers models * Working through start of system level controller * Added initial system level control example * Fixed SLC example * Reordering SLC settings * Updating the SLC example * Added curtailment * WIP: adding battery example for SLC * Improving plotting for battery SLC example * Moved combiner and changed battery model * Changed example so wind power is sometimes curtailed after charging battery * updated plotting script * added marker to curtailment component test * added super basic SLC tests to make sure examples dont break * fixed example tests * Minor name clarifications within SLC * Adding a notion of price-considering SLCs * WIP: working on marginal costs for SLC * Updating SLC example to be simpler * Removing prior SLC file * Consolidating methods for SLC * Moving SLC strategy definitions * moved solver options to a config class * minor fixes * Multi-commodity systems for SLC (NatLabRockies#717) * minor notes added to h2i model * added test for profit maximization example * updated slc base and h2i to handle multiple commodity streams * added test for multiple commodity streams * added test for multicommodity example, unsure if works * fixed example and updated non-demand commodity units handling in slc baseclass * some fixes in baseclass and tried to add logic for controlling multi-commodities * tried to update control logic to handle multiple commods * working on debugging whatever issue I caused * minor fixes for debugging * fixed bug fully * made electrolyzer capacity smaller * updated electrolyzer to not outputs nans * commented out finances for example 35 with hydrogen * aded output check for nans or infs * cleanups to slc baseclass * added docstrings and inline comments * added tiny comment * minor update to use self.technology_graph attribute * added use of _get_commodity_for_tech * Minor changes for clarity * Combining the three setup methods in SLC base * removed old keys in slc_config --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * added framework to interface with storage controllers and example * fix so curtailment only applied if using SLC * made slc_config an input option * actually make it so slc get option of slc conig and fixed bug in apply_curtailment * added check for slc in apply_curtailment() method * updated so slc_config is an input option and added control config class for profit max * Adding marginal_cost calcs * replaced lists of tech control classifiers with the dictionary * Directly connected demand profile * Shared commodity sell price as input from finance subgroups * Fixed bad set-point behavior * Uploading complex profit maximization * tried to add helper methods to slc baseclass * added marginal cost to cost models that dont inherit costmodelbaseconfig * updated cost model baseclass handling of marginal cost * updated so that demand is properly output from slc for storage controllers * fixed example 33 * Minor typographical changes * Added control classifiers to all technologies * updated baseclass methods and demand following controller * updated cost control strategies and cleaned up baseclass * updated find_converter_tech method * started adding logic for multi-commods * connected feedstocks * moved logic from compute to method in demand following * updated find_converter_techs and get_upstream_techs_for_commodity * bugfix * Ard control classifier * Updating for changes to slc_config * ard. * test * updated _find_feedstock_techs to use self.feedstock_comps * updated some doc strings in slc baseclass * docs wip * Minor refactor to demand following control * docs * docs * Additional refactoring of demand following control * Refactoring some SLC base methods * Added complex profit max example to test * updated location of cost_per_tech * update SLC add method * update SLC add method * Refactoring some SLC base methods * Expanding SLC docstrings * refactored test_slc_controllers * docs update * more docs * updated no_battery example * cleaned up other slc examples * removed SLC baseconfig since its unused * updated docstring in baseclass * docs added * minor update to demand following and updated profast to handle zero cf * added start of demand following docs * Adding to SLC docs * small doc mods * minor update to doc page * minor clarification in doc * Reconfiguring the connection for varopex * Shifting to five control classifiers * small doc change * connected storage duration * added more subtests to test_slc_examples for the two single-commodity demand following examples * Fixing example dir * fixed failing test * Checkpointing progress on all controllers for techs * Adding passthrough controller file * Updating tests * Added comments to passthrough controller * Fixing import statements * Fixing tests and examples * Partially addressing Jared's PR feedback * Renaming per PR feedback * Added notes in the docs that every tech group has a controller * Updating docs based on PR feedback * Apply suggestions from code review Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Updating adding a new tech doc page based on PR review * Merging * Updated control classifier docs * Removing now-defunct storage_techs_to_control parameter * Updating SLC plots and docstrings * Adding curtailment to more techs * Updated SLC battery example regression value * Adding notes that just one demand component is allowed now * Dramatically refactoring, renaming controller based things throughout * Added command value to electrolyzer * Docs build correctly locally * Updating tests and components for new command_value paradigm * Updating tests for set-point changes; simplifying sql timeseries test * Promoting controller parameters up a level * Reverting naming changes * Reverting final changes from naming * doc figures * Dramatic renaming for control nomenclature * doc figures update * PR feedback * Reverting errant storage control logic * inconsequential change to trigger CI --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: kbrunik <102193481+kbrunik@users.noreply.github.com> Co-authored-by: kbrunik <kbrunik@gmail.com> Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> * Dynamic Operating Constraints for Ammonia Synloop (NatLabRockies#770) * added draft of dynamic constraints on ammonia model and tests * cleanup on tests and added more drafts in * renamed startup methods to be more clear * updated tests and added missing method * fixed methods and got tests working * added recursive call to dynamic operation method * added more test drafts * added doc string and updated workflow to accurately calc feedstock consumption * added test for ramping constraints * Added test and made fix for multidt delay with a fraction * added lots of comments for question on applying both warm and cold start losses * added docstrings and inline comments * updated default in config so backward compatible * Much working and refactoring on the ammonia dynamics * Improving dynamics docs * Added dynamics tests and cleaned up comments * Addressing PR comments; changing warm+cold start behavior * adjust ramp rate * got ramping to work somewhat properly * updated docstrings, added inline comments, changed some code * minor cleanup * updated doc page to show operating bounds * doc fix * add full dynamics example to docs WIP * updated test values for ramping subtest * docs * dynamics update * fixed bug at i=0 for ramp down constraint violation * fixed ramp down constraint at start for multiple timesteps * added docstring * Added a whole bunch of subtests for stacking behavior and more * updated ramping limit to be flexible to varying timestep * updated tests for sub-dt timesteps * removed min production rate from ramping function * removed max_production_rate from ramping function * cleanup to consumption multiplier calc * updated failing tests * PR feedback * update test based on consumption multipler change * update ammonia integration test values after dynamics * Updated docs and slightly simplified ramping logic * updated small docs in dynamics.py * updated doc strings and documentation based on feedback * minor update to doc string in dynamics --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: kbrunik <102193481+kbrunik@users.noreply.github.com> Co-authored-by: kbrunik <kbrunik@gmail.com> * Expose common simulation/plant attributes on cost and performance baseclasses (NatLabRockies#783) * Moving plant_life etc * Updating changelog * Reducing examples 01 and 02's run times (NatLabRockies#782) * Reducing examples 01 and 02's run times * Reverting errant sql timeseries test regression * Updating examples 01 and 02 to recalculate power curve * more robust api calls, suggested by copilot after all my api calls we… (NatLabRockies#771) * more robust api calls, suggested by copilot after all my api calls were failing * run precommit * Update h2integrate/resource/utilities/download_tools.py Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> * import warnings was missing * Apply suggestions from code review Co-authored-by: John Jasa <john.jasa@nrel.gov> * Apply suggestion from @johnjasa --------- Co-authored-by: Pietro Bortolotti <pietro.bortolotti@nrel.gov> Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> Co-authored-by: John Jasa <john.jasa@nrel.gov> * Updating ammonia synloop example test values and tolerances to avoid errant CI issues (NatLabRockies#795) * Reworked how financial model units and commodity streams are defined (NatLabRockies#786) * removed logic in profast base for commodity units * updated profast LCO and baseclass - still have clean-up todo * fixed some bugs * got profast npv working * renamed function used for compute_units * udpated numpy npv * Added docstrings to compute_price_units and shifted it to tools * Removed is_electricity_producer * Adding back run_model call that was actually necessary * Updating other examples for new finance setup: * Trying to fix failing tests * Removed placeholder logic for NPV, just having sell price units explicitly defined * Removed last of logic that mapped names to units in H2IntegrateModel * added new tool and fixed NPV models * added in comments and docstring to tools * undid previous change to a test structure for an npv model * added test to profast_npv that uses nonstandard price units * added subtest for heat-based commodity * minor typo fix * Simplified the units handling * updated numpy npv to be able to handle nonstandard units --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Enhancement: Make `check_resource_dir` a general data folder routine (NatLabRockies#791) * add to and default feedstocks * move check_reource_dir tests to test_utilities and fix issues * update usage in resource tools * remove old usage * update changelog * add PR number * remove subtest for loading data into a context * update feedstock default directory and comment * fixed bug w tests * updated to use walruns operator * use a fixture to temporarily set environment variables and clean them up consistently * ensure the original value is preserved * fix user error * added comments warning folks about the sensitivities of the test that caused headaches --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> * Fix: Minor update to EIA NG pricing (NatLabRockies#777) * move url retrieval below filename check in case of file, not api * add missing and condition * add missing arguments to monthly conversion * add missing discrete i/o * add missing discrete i/o in super call * use all() on np.isclose() comparison * use expected numpy array, not pandas dataframe * remove unnecessary default price definition * add test for eia feedstock from file usage and model run * update changelog * fix test after change from pd to np * Update h2integrate/feedstocks/test/test_feedstocks.py Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> * Update h2integrate/feedstocks/test/test_feedstocks.py Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> * update all model calls --------- Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> Co-authored-by: John Jasa <johnjasa11@gmail.com> * Fix: Apply `temp_copy_of_example` to cleanup OpenMDAO local data creation (NatLabRockies#797) * convert temp_copy_of_example workaround to use temp_copy_of_example * update changelog --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Docs: refresh "add a new technology" guide, auto-generate model overview, unify the docs build (NatLabRockies#787) * Revamped the adding a new technology page * Updating model overview docs and other pages * Updated changelog * Update docs/developer_guide/adding_a_new_technology.md * Update docs/developer_guide/adding_a_new_technology.md * Update docs/developer_guide/adding_a_new_technology.md * Update docs/developer_guide/adding_a_new_technology.md * update docs * Cleaning up part of the docs --------- Co-authored-by: kbrunik <102193481+kbrunik@users.noreply.github.com> Co-authored-by: kbrunik <kbrunik@gmail.com> * Added more expansive feedstock tests for SLC (NatLabRockies#793) * Updates related to `demand_tech` in SLC framework (NatLabRockies#784) * updated SLC framework to require user to specify demand technology and only connect upstream techs to the controller * fixed bug * updated error message * added framework tests for SLC * renamed local variable of tech_graph to upstream_tech_graph * updated SLC doc page * added integration test example * added integration tests * added units with set and get_val and updated the one subtest thats failing to be more useful * added control classifier to demand comps * Renamed `slc_config` to `slc_topology` to reduce confusion * Minor SLC logic clarifications --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Feature: Add `feedstock_dir` option for the EIA Natural Gas functionality (NatLabRockies#801) * add feedstock_dir to EIA NG inputs * allow for parent directory creation in multi-folder creation * better control flow using data_subdir only when no input * fix issues and update fixture usage * update changelog * update changelog * only check feedstock_dir if filename is not None * update changelog * fix erroneous filename handling, duplicated directory resolution and missing assignment * Fix changelog --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Allow for connections including slice specifications (NatLabRockies#774) * move tests for split indices * revert obsolete changes to plant schema * fix bugs in length 3 connections for src_indices * remove use of eval * add ValueError for non-zero destination slice starts * allow for src_indices tiling into destination * update wind solar electrolyzer example to demonstrate different finance and connection approaches * add tests for each finance/connection approach added to wind solar electrolyzer example * add documentation for variable slicing in technology connections * update changelog * fix typo * Update docs/user_guide/connecting_technologies.md improve comment Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> --------- Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> * Reduce Sphinx documentation build warnings (NatLabRockies#800) * Removing a lot of warnings when building docs * Loosening ammonia tolerance in the example test * Cleaning up a few more docs build issues * Fixing minor doc indentation issues * Multi-year sell-price in ProFAST NPV (NatLabRockies#799) * made it so profast npv can take in multi-year sell price costs * updated error handling for misisng sell price value * added subtest for inflation test * updated profast NPV doc page * updated BasicProFASTParameterConfig.from_dict() so that user-provided escalation values are not overwritten with the inflation rate * updated so sell price is input as a dictionary to ProFAST * minor updates to profast parameter config and added test * updated so ProFAST NPV is input an array rather than a dict * minor clarification in doc page * updated changelog * Azimuth angle warnings and defaults in PySAM solar model (NatLabRockies#806) * added calc_azimuth_angle to pysam solar * removed azimuth specified in test_pvwatts_integration * Hooking up electricity buy price and more correctly (NatLabRockies#792) * Generalized functions for getting, setting, and finding environment variables (NatLabRockies#798) * added draft of generalizing api key tools * generalized api methods properly * moved base environment var functions to core * removed old work * minor clean-ups to generalized stuff * removed commented out code * removed usage of globals() * started adding in test file * added another test and bugfix in loading environment variable from file * removed commented out code and added todos * updated so that setter method is called after _get_env_with_fallback * added doc strings to setter methods in nlr_developer_api_keys.py * refactored environment tools with help from Rob * minor changes to nlr_developer_api_keys * updated tests for env_tools * updated conftest.py files * updated doc page * updated conftest.py files and get_environment_variables * added doc page for debugging environment variable problems * updated doc page so formatting is nice * typo fix in model_overview.md * updates to doc pages * added set_vars as input to get_nlr_developer_api_credential and updated conftest.py files * updated debugging environment variables doc page * cleaned up some of test_env_tools.py * finished cleanups to test_env_tools.py * updated conftest.py files * updated changelog * Added new method to check for duplicate defined environment variables and to raise a user warning * changed executable code cell to python code block in doc page * commiting johns doc changes --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Correct pipe rate units from galUS to galUS/h (NatLabRockies#813) * correct rate units from galUS to galUS/h * update pipe units for water from galUS to galUS/h * update changelog * Bugfix OpenMeteo Resource Local Timezone (NatLabRockies#814) * added draft of generalizing api key tools * generalized api methods properly * moved base environment var functions to core * removed old work * minor clean-ups to generalized stuff * removed commented out code * removed usage of globals() * started adding in test file * added another test and bugfix in loading environment variable from file * removed commented out code and added todos * updated so that setter method is called after _get_env_with_fallback * added doc strings to setter methods in nlr_developer_api_keys.py * refactored environment tools with help from Rob * minor changes to nlr_developer_api_keys * updated tests for env_tools * updated conftest.py files * updated doc page * updated conftest.py files and get_environment_variables * added doc page for debugging environment variable problems * updated doc page so formatting is nice * typo fix in model_overview.md * updates to doc pages * added set_vars as input to get_nlr_developer_api_credential and updated conftest.py files * updated debugging environment variables doc page * cleaned up some of test_env_tools.py * finished cleanups to test_env_tools.py * updated conftest.py files * updated changelog * Added new method to check for duplicate defined environment variables and to raise a user warning * changed executable code cell to python code block in doc page * trying to understand problem with openmeteo downloads * still trying to understand openmeteo downloads * got openmeteo solar data to work properly * minor change to solar_resource_base * cleaned up openmeteo solar data * removed print statement * minor cleanups to openmeteo solar model * added all resource files that could be used for testing - will likely remove some of these * Added big integration test of openmeteo solar models * updated openmeteo wind and moved make_time_index to utility file * removed 3 recently added resource files * updated solar resource test that was failing * added doc string and comments to new function in download_tools * removed commented out code from openmeteo solar * updated subtest values for example 13 and 26 * removed old download resource file and corresponding subtests * updated resource_index.md * updated changelog * Added fail-fast false to CI * Updated floris test --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Nuclear with thermal connection with HTSE (NatLabRockies#807) * Add nuclear thermal plant model with primary `heat` and `electricity` modes * Add HTSE/SOEC model with heat input --------- Co-authored-by: Daniel Mark Mikkelson <89001004104222@fedidcard.gov> Co-authored-by: mikkdm <74319124+mikkdm@users.noreply.github.com> Co-authored-by: John Jasa <johnjasa11@gmail.com> Co-authored-by: John Jasa <john.jasa@nrel.gov> * Adding inflation rate to numpy NPV calc (NatLabRockies#788) * Adding inflation rate to numpy NPV calc * Update docs/finance_models/numpy_financial_npv.md Co-authored-by: Jared Thomas <jaredthomas68@users.noreply.github.com> * renaming discount_rate to real_discount_rate for numpy npv * Inconsequential change to trigger CI * add wacc calculation to numpy npv and clarify that the numpy npv model is pre-tax --------- Co-authored-by: Jared Thomas <jaredthomas68@users.noreply.github.com> Co-authored-by: Jared Thomas <jaredthomas68@gmail.com> * Updating throughout to remove last of doc warnings (NatLabRockies#821) * Updating throughout to remove last of doc warnings * Fix PR reference for docs build warnings Updated PR reference for fixing docs build warnings in CHANGELOG. * Bug fix for output path handling (NatLabRockies#820) * Bug fix for output path handling * update changelog * update eia test to reflect expected failure mode * Changed EIA key logic to use monkeypatch --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Bugfix to `technology_graph` and other minor fixes (NatLabRockies#823) * updated how technology_graph edge data is added and added test and updated what info is given to the SLC model wrt upstream techs * updated slc baseclass method that would break with new use case * Making graph edges always a list * Updating commodity edges to always be lists * updated changelog * removed unused method and minor change --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Path type instance check for schema validators (NatLabRockies#822) * added Path validator to schema validators * added unit test * Very minor type checking changes to yaml schema --------- Co-authored-by: John Jasa <johnjasa11@gmail.com> * Open loop control base remove "storage" naming (NatLabRockies#828) * rename and move openloop_storage_control_base to remove 'storage' since it can be used for converters * update changelog and class hierarchy * Bump version ahead of 0.9 release (NatLabRockies#829) * update changelog * Bump version number --------- Co-authored-by: kbrunik <kbrunik@gmail.com> --------- Co-authored-by: Dakota Ramos <dakota.ramos@nrel.gov> Co-authored-by: jmartin4 <jonathan.martin@nrel.gov> Co-authored-by: kbrunik <kbrunik@gmail.com> Co-authored-by: Jonathan Martin <94018654+jmartin4nrel@users.noreply.github.com> Co-authored-by: kbrunik <102193481+kbrunik@users.noreply.github.com> Co-authored-by: elenya-grant <116225007+elenya-grant@users.noreply.github.com> Co-authored-by: Jared Thomas <jaredthomas68@gmail.com> Co-authored-by: svijaysh <svijaysh@nrel.gov> Co-authored-by: Jared Thomas <jaredthomas68@users.noreply.github.com> Co-authored-by: genevievestarke <103534902+genevievestarke@users.noreply.github.com> Co-authored-by: Hammond, Rob <13874373+RHammond2@users.noreply.github.com> Co-authored-by: Sanjana Vijayshankar <59975769+vijay092@users.noreply.github.com> Co-authored-by: Pietro Bortolotti <pietro.bortolotti@nlr.gov> Co-authored-by: Pietro Bortolotti <pietro.bortolotti@nrel.gov> Co-authored-by: Daniel Mark Mikkelson <89001004104222@fedidcard.gov> Co-authored-by: mikkdm <74319124+mikkdm@users.noreply.github.com> Co-authored-by: Zachary Tully <ztully@nlr.gov>
johnjasa
left a comment
There was a problem hiding this comment.
Thanks for these changes, Rob! I support this effort.
My only initial hang-up is how verbose some of the new lines become, especially the dict ones, e.g.:
peak_range: dict = field(
validator=validators.deep_mapping(
key_validator=validators.in_(["start", "end"]),
mapping_validator=(
validators.instance_of(dict),
validators.min_len(2),
validators.max_len(2),
),
)
)
That being said, being more strictly correct (I saw your note in the PR that we weren't fully checking everything for those cases) and also not wrapping our own validators seems worthwhile.
I like that you deleted all the now-unused functions in validators.py! Good to see.
This was the only validator I could see keeping, but only if we corrected it to apply the correct checks. This is also more of a use case for having a dataclass object represent the dictionary, so I would probably be in favor of folks taking the following approach for more involved checks as it's far cleaner. @define
class PeakRange(BaseConfig):
start: int = field(validator=validators.instance_of(int))
end: int = field(validator=validators.instance_of(int))
@define
class PeakLoadManagementHeuristicOpenLoopStorageControllerConfig:
demand_profile_upstream: int | float | list | None = field()
peak_range: PeakRange | dict = field(converter=PeakRange.from_dict) |
Thanks for this suggestion, Rob! I dig it. Sometimes it's tough to know when to use a certain object, but this validation is so much cleaner looking. |
Replace custom validators with
attrs.validatorsThis PR replaces the use of custom validators contained within
h2integrate/core/validators.pywith functionality provided byattrs.validatorsto reduce code maintenance by taking advantage of existing solutions.The only validator that was not replaced was
not_equal. There are two potential approaches we could take to ensure a complete replacement:in_validator:2. Wait for this PR to be merged and released to allow for:I would personally opt for option 2 as it doesn't allow for the addition of newly added values in the future, maintaining the spirit and implementation of themust_equal.attrsprefers the use ofin_to cover equality.Section 1: Type of Contribution
Section 2: Draft PR Checklist
N/A
Section 3: General PR Checklist
docs/files are up-to-date, or added when necessaryCHANGELOG.md"A complete thought. [PR XYZ]((https://github.com/NatLabRockies/H2Integrate/pull/XYZ)", where
XYZshould be replaced with the actual number.Section 4: Related Issues
Section 5: Impacted Areas of the Software
Section 5.1: New Files
N/A
Section 5.2: Modified Files
Too many files have been modified to reasonably list them all, so the replacement validators will be described below.
must_equal(val)gt_zeroge_zerorange_val(min_val, max_val)range_val_or_none(min_val, max_val)contains([val1, val2])has_required_keys([key1, key2])This validator's replacement is a little more involved. After discussion with @elenya-grant, it appears the checks in the original were incomplete, so we have to do a slightly roundabout validation to ensure only
key1andkey2are keys in the dictionary as was originally intended for the validator. First we use thekey_validatorto validate the key names are valid. Then we use themapping_validatorto enforce an equally sized min and max length of the of the dictionary to ensure we have a dictionary with exactlykey1andkey2keys.Section 6: Additional Supporting Information
Section 7: Test Results, if applicable
Tests pass.
Section 8 (Optional): New Model Checklist
N/A