Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
81b7d9f
PR: Adding work from previous PIs into feature branch (#24)
taddyb Aug 19, 2025
23ec477
Ngwpc 6956: Complete the test case and any changes to support USBR re…
taddyb Aug 22, 2025
c640211
Pin package versions in requirements.txt
taddyb Aug 22, 2025
fbaef3c
Owp merge GitHub master edfs troute pi6 (#26)
taddyb Aug 22, 2025
8cbe335
RFC Channel Loss algorithm implementation (#28)
taddyb Aug 25, 2025
fecf6a4
NGWPC 7006: Replace and route updates to the icefabric (#29)
taddyb Aug 30, 2025
326158b
patch: fixed broken lower colorado test (#31)
taddyb Aug 31, 2025
cebc049
NGWPC 7009: prepping t-route rnr for test (#30)
taddyb Sep 2, 2025
4a930cd
NGWPC-6058: added iac entrypoint to troute-rnr (#34)
taddyb Sep 13, 2025
e0ceee2
Patch 091325: typos and logic in postprocessing (#35)
taddyb Sep 15, 2025
69162dd
patch: removed collect (#36)
taddyb Sep 15, 2025
1ea19b4
patch: added bucket_name (#37)
taddyb Sep 15, 2025
19a38df
patch: added logging and IaC channel support (#38)
taddyb Sep 17, 2025
1ae5e76
patch: changed boto3 location for import in main (#39)
taddyb Sep 17, 2025
13323b1
patch: updated boto3 version (#40)
taddyb Sep 18, 2025
33a6ac4
Patch 091725 update boto3 version (#41)
taddyb Sep 18, 2025
19fbb92
patch: updated iac channel rabbit_mq_endpoint reference
kevinspurrier Sep 18, 2025
72437b4
patch: changed mkdir to create parents, added a store_true to --iac p…
taddyb Sep 18, 2025
05be027
patch: added numpy <2 dependency
taddyb Sep 18, 2025
a093887
patch: changed mkdir to create parents
kevinspurrier Sep 18, 2025
cc1d11e
feat: added debug logging
taddyb Sep 19, 2025
0ad742b
Patc: added debug flag to make log level optional
taddyb Sep 19, 2025
a57d10e
fix: updated incorrect file reference
taddyb Sep 19, 2025
2dbc17a
fix: removed data dir references
taddyb Sep 19, 2025
bfe553d
fix: removed xarray s3 writes (#42)
taddyb Sep 19, 2025
3fad399
merge: merging in rfc-channel loss
taddyb Nov 24, 2025
6d9d61b
merge: merged channel loss into developement
taddyb Nov 24, 2025
009ab2f
merge: merging usbr persistence
taddyb Nov 24, 2025
c35f79e
merge: merging in rnr
taddyb Nov 24, 2025
80c8065
merge: merging development
taddyb Nov 24, 2025
d0bdd34
tests: added test case for conecuh river test case (#57)
taddyb Nov 25, 2025
ff45933
Ngwpc 8461: added the NHF dataset integration and flow-scaling (#61)
taddyb Dec 4, 2025
8c8e183
NGWPC 8465: Added unit tests for flow-scaling (#63)
taddyb Dec 5, 2025
1055f6e
feat: updated the view_output.ipynb with flow-scaling verification
taddyb Dec 9, 2025
f6ec7db
patch: added text to notebook explaining flow and flow-scaling
taddyb Dec 9, 2025
4b3a980
nit: added a new flow animation
taddyb Dec 9, 2025
378d1f2
Merge branch 'pi-8-NGWPC-8442' into nhf-routing
dylanlee Feb 23, 2026
146ad4c
feat: NGWPC 9839 - Add first draft CONUS end-to-end test
sclaw Feb 23, 2026
0fbea36
add formal pytest
sclaw Feb 24, 2026
b93bd98
feat: draft flowpath discretization for nhf t-route
dylanlee Feb 25, 2026
e3a88aa
test: modify unit tests for new version of nhf-troute
dylanlee Feb 25, 2026
93e7332
fix: correctly use virtual nexuses as fixed nodes
dylanlee Feb 26, 2026
2e124ce
test: update nhf-routing ipython notebook
dylanlee Feb 26, 2026
e5f1933
test: update Conecuh test data
dylanlee Feb 26, 2026
ecad1ef
test: add script to create Conecuh test data from nhf gpkg
dylanlee Feb 26, 2026
318b46a
test: update test flow animation
dylanlee Feb 26, 2026
d674756
feat: add first draft of routing diagnostics suite
sclaw Feb 27, 2026
80a357a
feat: working draft routing diagnostics
sclaw Mar 2, 2026
47e9902
feat: add uplink routing using offnetwork_upstreams
dylanlee Mar 2, 2026
508cb88
test: update nhf routing tests for new upstream Q inputs
dylanlee Mar 2, 2026
57337cb
refactor: split up NHF.py for readability
dylanlee Mar 3, 2026
c9b1251
fix: offnetwork_upstreams sub indexing in compute.py
dylanlee Mar 3, 2026
a643052
fix: additional offnetwork_upstreams fix in compute.py
dylanlee Mar 3, 2026
0aea537
nit: add comment clarifying 0 vel and depth
dylanlee Mar 4, 2026
9913130
Add draft of diagnostic plots for review
sclaw Mar 4, 2026
daff840
attempt to fix markdown equation blocks
sclaw Mar 4, 2026
d42dc7b
add additional diagnostics to nhf tests_physics tests
sclaw Mar 4, 2026
120995f
refactor physics test. Utilize pre-compiled mc kernel. remove pytest…
sclaw Mar 5, 2026
84f1123
perf: vectorize part of build_flowveldepth_interorder
sclaw Mar 6, 2026
a02d91e
fix: make _connections dict construction static
dylanlee Mar 6, 2026
919f334
chore: update test imports
dylanlee Mar 6, 2026
58c9f5e
fix: remove uneccessary virtual segment check in NHF.py
dylanlee Mar 6, 2026
966155f
create CLI for forcing generation. Add reference datasets to forcing…
sclaw Mar 6, 2026
83f42d4
add "runout" period to retrospective datasets
sclaw Mar 6, 2026
b93d7c0
rename diagnostics file
sclaw Mar 6, 2026
99bcf43
fix: catchment flow distribution
dylanlee Mar 6, 2026
424dfe5
feat: map computational link IDs back to fp_id in output
dylanlee Mar 6, 2026
285653c
fix: update notebook for fp_id output
dylanlee Mar 6, 2026
b4816cf
debug forcing reference dataset generation
sclaw Mar 6, 2026
f687c2a
feat: add ability to drive NHF via bmi
dylanlee Mar 6, 2026
7af53b4
update dependencies for tests
sclaw Mar 6, 2026
0aba495
add plots to diagnostics. convert to CLI-style access. update directo…
sclaw Mar 6, 2026
3b57957
debug forcing generation
sclaw Mar 6, 2026
fd12620
remove test_conus (placeholder)
sclaw Mar 6, 2026
ddff7aa
Merge branch 'feat/nhf-upstream-flow' into pi-9-NGWPC-9834
sclaw Mar 9, 2026
772c353
Merge remote-tracking branch 'origin/feature/nhf-bmi' into pi-9-NGWPC…
sclaw Mar 9, 2026
dd32a04
partial refactor for new network discretization
sclaw Mar 9, 2026
cd4aca5
refactor discretization for clarity, performance, external access, an…
sclaw Mar 9, 2026
582a4da
Merge branch 'refactor/nhf-links-and-nodes' into pi-9-NGWPC-9834
sclaw Mar 9, 2026
7d73a83
update diagnostics for discretization and qlat changes
sclaw Mar 10, 2026
6341b3a
debug diagnostics
sclaw Mar 10, 2026
e59f667
debug diagnostics and add external n-samples control
sclaw Mar 10, 2026
fc9846c
update NHF test documentation
sclaw Mar 10, 2026
8406266
speed up gpkg reads with better geometry ignore
sclaw Mar 10, 2026
ad11edc
speed up link connection construction
sclaw Mar 10, 2026
620a21a
speed up file reads
sclaw Mar 11, 2026
6203a33
speed up distribute_catchment_discharge
sclaw Mar 11, 2026
d5ae772
refactor nhf code to increase speed and decrease memory footprint.
sclaw Mar 13, 2026
1f44596
revert handling of qlats for lower memory footprint
sclaw Mar 13, 2026
520cfcf
revert diagnostics for fp_id
sclaw Mar 13, 2026
81cfb25
correct qlat addition code to align with OWP NHF guidance
sclaw Mar 14, 2026
aa7291f
wip discretize refactor
sclaw Mar 16, 2026
7b541f3
stable NHF implementation that aligns with OWP qlat guidance
sclaw Mar 18, 2026
5113aa6
cleanup and formatting
sclaw Mar 18, 2026
edc1f21
Remove temporary handling of null upstream virtual nexus
sclaw Mar 18, 2026
519f54b
clean up NHF and limit data loading for performance
sclaw Mar 18, 2026
6e6693b
docstring + comment formatting
sclaw Mar 18, 2026
063d050
Add subset nhf functionality to tests
sclaw Mar 18, 2026
541525d
debug and apply formatting to diagnostics
sclaw Mar 19, 2026
4ff31d5
update merging process from batch to one at a time merging
sclaw Mar 19, 2026
9a5bbc8
update link to fp_id mapping approach to handle merged reaches
sclaw Mar 19, 2026
fcf0604
remove test data
sclaw Mar 19, 2026
9684efa
fix bug in forcing generation
sclaw Mar 19, 2026
403b4b7
update diagnostics to conform to new NHF fp_link mapping structure.
sclaw Mar 19, 2026
2c55527
fix bug in flow scaling scheme
sclaw Mar 19, 2026
37a3cfe
add temporary patches for NHF build
sclaw Mar 19, 2026
4c7ed88
update code for performance when scaling to conus
sclaw Mar 19, 2026
7ff8b9c
add devcontainer setup
sclaw Mar 19, 2026
d7df09b
update conus test readme
sclaw Mar 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
28 changes: 28 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"name": "troute dev container",
"build": {
"dockerfile": "../docker/Dockerfile.dev",
"context": "..",
"target": "dev"
},
"runArgs": ["--ulimit", "nofile=10000:10000"],
"workspaceFolder": "/t-route",
"workspaceMount": "source=${localWorkspaceFolder},target=/t-route,type=bind",
"mounts": [
"source=/path/to/hydrofabric,target=/hydrofabric,type=bind"
],
"customizations": {
"vscode": {
"settings": {
"terminal.integrated.defaultProfile.linux": "bash"
},
"extensions": [
"ms-python.python",
"charliermarsh.ruff",
"yy0931.vscode-sqlite3-editor",
"ms-python.debugpy",
"ms-toolsai.jupyter"
]
}
}
}
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,20 @@ bower_components/
src/vendor/
dist/

# Build dirs #
##############
build/

# .vscode files #
#################
.vscode/

# UV lock #
###########
uv.lock

# data #
########
data/
output/
*.gif
155 changes: 93 additions & 62 deletions compiler.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
#!/bin/bash

#TODO add options for clean/noclean, make/nomake, cython/nocython
#TODO include instuctions on blowing away entire package for fresh install e.g. rm -r ~/venvs/mesh/lib/python3.6/site-packages/troute/*

#set root folder of github repo (should be named t-route)
REPOROOT=`pwd`

#For each build step, you can set these to true to make it build
#or set it to anything else (or unset) to skip that step
build_mc_kernel=true
Expand All @@ -18,23 +22,51 @@ then
export F90="gfortran"
echo "using F90=${F90}"
fi

if [ -z "$CC" ]
then
export CC="gcc"
echo "using CC=${CC}"
fi

#preserve old/default behavior of installing packages with -e
# Parse command line arguments
WITH_EDITABLE=true
if [ "$1" == 'no-e' ]
then
WITH_EDITABLE=false
USE_UV=false

while [[ $# -gt 0 ]]; do
case $1 in
no-e)
WITH_EDITABLE=false
shift
;;
--uv)
USE_UV=true
shift
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 [no-e] [--uv]"
echo " no-e: Install packages without editable mode"
echo " --uv: Use uv pip instead of pip"
exit 1
;;
esac
done

# set pip command based on UV flag
if [[ "$USE_UV" == true ]]; then
PIP_CMD="uv pip"
echo "Using uv pip for package installation"
else
PIP_CMD="pip"
echo "Using standard pip for package installation"
fi

#if you have custom static library paths, uncomment below and export them
#export LIBRARY_PATH=<paths>:$LIBRARY_PATH
#if you have custom dynamic library paths, uncomment below and export them
#export LD_LIBRARY_PATHS=<paths>:$LD_LIBRARY_PATHS

if [ -z "$NETCDF" ]
then
export NETCDFINC=/usr/include/openmpi-x86_64/
Expand All @@ -45,87 +77,86 @@ then
# (before ./compiler.sh)
if [ -n "$NETCDFALTERNATIVE" ]
then
echo "using alternative NETCDF inc ${NETCDFALTERNATIVE}"
export NETCDFINC=$NETCDFALTERNATIVE
echo "using alternative NETCDF inc ${NETCDFALTERNATIVE}"
export NETCDFINC=$NETCDFALTERNATIVE
fi
else
export NETCDFINC="${NETCDF}"
fi
echo "using NETCDFINC=${NETCDFINC}"

if [[ "$build_mc_kernel" == true ]]; then
#building reach and resevoir kernel files .o
cd $REPOROOT/src/kernel/muskingum/
make clean
make || exit
make install || exit
if [[ "$build_mc_kernel" == true ]]; then
#building reach and resevoir kernel files .o
cd $REPOROOT/src/kernel/muskingum/
make clean
make || exit
make install || exit
fi

if [[ "$build_diffusive_tulane_kernel" == true ]]; then
#building reach and resevoir kernel files .o
cd $REPOROOT/src/kernel/diffusive/
make clean
make diffusive.o
make pydiffusive.o
make chxsec_lookuptable.o
make pychxsec_lookuptable.o
make install || exit
if [[ "$build_diffusive_tulane_kernel" == true ]]; then
#building reach and resevoir kernel files .o
cd $REPOROOT/src/kernel/diffusive/
make clean
make diffusive.o
make pydiffusive.o
make chxsec_lookuptable.o
make pychxsec_lookuptable.o
make install || exit
fi

if [[ "$build_reservoir_kernel" == true ]]; then
cd $REPOROOT/src/kernel/reservoir/
make clean
#make NETCDFINC=`nc-config --includedir` || exit
#make binding_lp.a
#make install_lp || exit
make
make install_lp || exit
make install_rfc || exit

cd $REPOROOT/src/kernel/reservoir/
make clean
#make NETCDFINC=`nc-config --includedir` || exit
#make binding_lp.a
#make install_lp || exit
make
make install_lp || exit
make install_rfc || exit
fi

if [[ "$build_framework" == true ]]; then
#creates troute package
cd $REPOROOT/src/troute-network
rm -rf build

if [[ ${WITH_EDITABLE} == true ]]; then
pip install --no-build-isolation --config-setting='--build-option=--use-cython' --editable . --config-setting='editable_mode=compat' || exit
else
pip install --no-build-isolation --config-setting='--build-option=--use-cython' . || exit
fi
#creates troute package
cd $REPOROOT/src/troute-network
rm -rf build
if [[ ${WITH_EDITABLE} == true ]]; then
$PIP_CMD install --no-build-isolation --config-setting='--build-option=--use-cython' --editable . --config-setting='editable_mode=compat' || exit
else
$PIP_CMD install --no-build-isolation --config-setting='--build-option=--use-cython' . || exit
fi
fi

if [[ "$build_routing" == true ]]; then
#updates troute package with the execution script
cd $REPOROOT/src/troute-routing
rm -rf build

if [[ ${WITH_EDITABLE} == true ]]; then
pip install --no-build-isolation --config-setting='--build-option=--use-cython' --editable . --config-setting='editable_mode=compat' || exit
else
pip install --no-build-isolation --config-setting='--build-option=--use-cython' . || exit
fi
#updates troute package with the execution script
cd $REPOROOT/src/troute-routing
rm -rf build
if [[ ${WITH_EDITABLE} == true ]]; then
$PIP_CMD install --no-build-isolation --config-setting='--build-option=--use-cython' --editable . --config-setting='editable_mode=compat' || exit
else
$PIP_CMD install --no-build-isolation --config-setting='--build-option=--use-cython' . || exit
fi
fi

if [[ "$build_config" == true ]]; then
#updates troute package with the execution script
cd $REPOROOT/src/troute-config
if [[ ${WITH_EDITABLE} == true ]]; then
pip install --editable . || exit
else
pip install . || exit
fi
#updates troute package with the execution script
cd $REPOROOT/src/troute-config
if [[ ${WITH_EDITABLE} == true ]]; then
$PIP_CMD install --editable . || exit
else
$PIP_CMD install . || exit
fi
fi

if [[ "$build_nwm" == true ]]; then
#updates troute package with the execution script
cd $REPOROOT/src/troute-nwm
if [[ ${WITH_EDITABLE} == true ]]; then
pip install --editable . || exit
else
pip install . || exit
fi
#updates troute package with the execution script
cd $REPOROOT/src/troute-nwm
if [[ ${WITH_EDITABLE} == true ]]; then
$PIP_CMD install --editable . || exit
else
$PIP_CMD install . || exit
fi
fi

if [[ "$build_bmi" == true ]]; then
Expand Down
128 changes: 0 additions & 128 deletions compiler_uv.sh

This file was deleted.

Loading