Skip to content

Clean up challenge #902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ldecicco-USGS merged 3 commits into from
Jun 2, 2026
+69 −21
Merged

Clean up challenge #902

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0727eb3
Merge branch 'main' of github.com:DOI-USGS/dataRetrieval
ldecicco-USGS Jun 2, 2026
c6b75f8
is it me?
ldecicco-USGS Jun 2, 2026
4ea4390
formatting
ldecicco-USGS Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 69 additions & 21 deletions tutorials/basic_slides_deck.qmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ title-slide-attributes:
data-background-position: 2% 2%
editor: source
engine: knitr
knitr:
opts_chunk:
python.reticulate: TRUE
editor_options:
chunk_output_type: console
execute:
Expand All @@ -29,11 +32,13 @@ params:
```{r}
#| echo: false
#| include: false
# library(dataRetrieval)
#| label: rsetup
library(ggplot2)
library(dplyr)
library(reticulate)

reticulate::py_config()

py_require("dataretrieval")
py_require("panda")
py_require("matplotlib")
Expand Down Expand Up @@ -80,6 +85,19 @@ dt_me <- function(
}
```

```{python}
#| echo: false
#| label: loadlibs

import matplotlib.pyplot as plt
import pandas as pd
import seaborn
import matplotlib.pyplot as plt
import geopandas as gpd
import os
import dataretrieval
# from dataretrieval import waterdata
```

## Introduction {background-image="combo_flip.png" background-size="25%" background-position="95% 90%" }

Expand All @@ -103,13 +121,15 @@ The goal of these slides are to:
```{r}
#| echo: true
#| eval: false
#| label: rinstall
install.packages("dataRetrieval")
```

Then each time you open R, you'll need to load the library:

```{r}
#| message: true
#| label: rload
library(dataRetrieval)
```

Expand All @@ -118,6 +138,7 @@ library(dataRetrieval)
```{bash}
#| echo: true
#| eval: false
#| label: pythoninstall
pip install dataretrieval
```

Expand All @@ -126,13 +147,15 @@ or
```{bash}
#| echo: true
#| eval: false
#| label: pythoninstallconda
conda install conda-forge::dataretrieval
```

Then each time you open Python, you'll need to load the library:

```{python}
#| eval: !expr evaluate_python
#| eval: true
#| label: loadpython
from dataretrieval import waterdata
```

Expand Down Expand Up @@ -162,6 +185,7 @@ Within R, you can call help files for any `dataRetrieval` function:
```{r}
#| echo: true
#| eval: false
#| label: rhelp
?read_waterdata_daily
```

Expand All @@ -184,6 +208,7 @@ Examples

```{r}
#| eval: false
#| label: rexample
site <- "USGS-02238500"
dv_data_sf <- read_waterdata_daily(
monitoring_location_id = site,
Expand All @@ -202,6 +227,7 @@ Within Python, you can call help for any `dataretrieval` function:

```{python}
#| eval: !expr evaluate_python
#| label: pythonhelp
help(waterdata.get_daily)
```

Expand Down Expand Up @@ -278,7 +304,10 @@ The USGS uses various codes for basic retrievals. These codes can have leading z
Here are some examples of a few common codes:


```{r echo=FALSE, eval=TRUE}
```{r}
#| echo: false
#| eval: true
#| label: commonpcodes
library(knitr)

df <- data.frame(
Expand Down Expand Up @@ -306,6 +335,7 @@ knitr::kable(list(df, df2))

```{r}
#| eval: false
#| label: rmetadataexamples
parameter_codes <- read_waterdata_metadata("parameter-codes")
statistic_codes <- read_waterdata_metadata("statistic-codes")
# Others:
Expand All @@ -327,6 +357,7 @@ states <- read_waterdata_metadata("states")

```{python}
#| eval: false
#| label: pythonmetadataexamples
parameter_codes = waterdata.get_reference_table("parameter-codes")
statistic_codes = waterdata.get_reference_table("statistic-codes")
# Others:
Expand Down Expand Up @@ -364,19 +395,22 @@ Each function returns a Tuple, containing a dataframe and a Metadata class.

1. Open your preferred IDE (RStudio, VSCode, PyCharm, etc) or Jupyter notebook

2. R: Install `dataRetrieval`, `ggplot2`, `dplyr`, `leaflet` (if you haven't already)
2. Install packages if needed:

- R: `dataRetrieval`, `ggplot2`, `dplyr`, `leaflet`

2. Python: Install `dataretrieval`, `matplotlib`, `geopandas`, `folium`, `seaborn` (if you haven't already)
- Python: `dataretrieval`, `matplotlib`, `geopandas`, `folium`, `seaborn`

3. Load `dataRetrieval` (R) / waterdata module in `dataretrieval` (Python)

4. Open the help file for the function `read_waterdata_daily`
4. Open the help file for the function `read_waterdata_daily` (R) or `waterdata.get_daily` (Python)


### R

```{r}
#| eval: false
#| label: rchallengeintro
install.packages(c("dataRetrieval", "ggplot2", "leaflet", "dplyr"))
library(dataRetrieval)
?read_waterdata_daily
Expand All @@ -387,12 +421,14 @@ library(dataRetrieval)
```{bash}
#| echo: true
#| eval: false
#| label: pythonchallengeintro
pip install dataretrieval matplotlib geopandas folium seaborn
```


```{python}
#| eval: false
#| label: pythonchallengeintro2
from dataretrieval import waterdata

help(waterdata.get_daily)
Expand Down Expand Up @@ -445,6 +481,7 @@ Let's pretend the token sent you was "abc123"
```{r}
#| echo: true
#| eval: false
#| label: usethis
usethis::edit_r_environ()
```

Expand All @@ -462,6 +499,7 @@ API_USGS_PAT = "abc123"

```{r}
#| eval: false
#| label: apitest1
Sys.getenv("API_USGS_PAT")
```

Expand All @@ -485,9 +523,11 @@ API_USGS_PAT = "abc123"
```{python}
#| echo: true
#| eval: false
#| label: pythonapitest
import os

os.getenv("API_USGS_PAT")
'abc123'
"abc123"
```

::: {.callout-note collapse="true"}
Expand All @@ -502,27 +542,31 @@ Your .env file should never be pushed to a public repository.

```{bash}
#| eval: false
#| label: activateenv
conda activate flow_bootcamp
```

3. Add variable:

```{bash}
#| eval: false
#| label: setapi2
conda env config vars set API_USGS_PAT="abc123"
```

4. Reactivate enviornment

```{bash}
#| eval: false
#| label: reactivateenv
conda activate flow_bootcamp
```

5. Open your enviornment (for example Jupyter), and test that it's there:

```{python}
#| eval: false
#| label: checkapi3
import os

print(os.getenv("API_USGS_PAT"))
Expand Down Expand Up @@ -635,6 +679,7 @@ Here are a bunch of valid inputs:

```{r}
#| code-line-numbers: "1-9|10-11|12-15|16-19"
#| label: rtime
# Ask for exact times:
time = "2025-01-01"
time = as.Date("2025-01-01")
Expand All @@ -659,6 +704,8 @@ time = "PT12H" # past hours
### Python

```{python}
#| label: pythontime

# Ask for exact times:
time = "2025-01-01"
# Ask for specific range
Expand Down Expand Up @@ -705,6 +752,7 @@ Let's get all the monitoring locations for Dane County, Wisconsin:
### R

```{r}
#| label: rdanecountysites
#| message: false
site_info <- read_waterdata_monitoring_location(
state_name = "Wisconsin",
Expand All @@ -716,7 +764,7 @@ site_info <- read_waterdata_monitoring_location(

```{python}
#| eval: !expr evaluate_python

#| label: pythondanecountysites
site_info, md = waterdata.get_monitoring_locations(
state_name="Wisconsin", county_name="Dane County"
)
Expand All @@ -730,6 +778,7 @@ site_info, md = waterdata.get_monitoring_locations(
`read_waterdata_monitoring_location` requires "County" in the county_name argument. You can check county names using:
```{r}
#| eval: false
#| label: rcheckcounties
counties <- check_waterdata_sample_params(service = "counties")
```
:::
Expand All @@ -742,6 +791,7 @@ counties <- check_waterdata_sample_params(service = "counties")

```{r}
#| echo: false
#| label: showsiteinfo
dt_me(site_info, 5, "0.6em")
```

Expand All @@ -759,6 +809,7 @@ Now that we've seen the whole data set, maybe we realize in the future we can as

```{r}
#| message: true
#| label: rsiteinforefined
site_info_refined <- read_waterdata_monitoring_location(
state_name = "Wisconsin",
county_name = "Dane County",
Expand All @@ -776,6 +827,7 @@ site_info_refined <- read_waterdata_monitoring_location(

```{python}
#| eval: !expr evaluate_python
#| label: pythonsiteinforefined
site_info_refined, md = waterdata.get_monitoring_locations(
state_name="Wisconsin",
county_name="Dane County",
Expand Down Expand Up @@ -921,11 +973,11 @@ dt_me(

1. How many USGS stream sites are within Tuscaloosa County, Alabama?

2. What are the unique parameter_names that come back from USGS sites in Tuscaloosa County, Alabama?
2. What are the unique parameter_names that come back from those stream sites?

3. What site has the longest period of record for daily mean discharge?
3. Of those sites, what site has the longest period of record for daily mean discharge?

4. Bonus: Create an interactive map of all sites that measure gage height.
4. Bonus: Create an interactive map of all sites that measure daily mean discharge.

The amount you get done during this break will highly depend on the extent of your coding background. Use this time to explore dataretrieval functions and outputs.

Expand All @@ -944,7 +996,6 @@ tus <- read_waterdata_combined_meta(
site_type = "Stream"
)
tus |>
filter(parameter_name == "Gage height") |>
select(monitoring_location_id) |>
distinct() |>
pull() |>
Expand Down Expand Up @@ -983,7 +1034,7 @@ leaflet(

## Workflow 3: Get Latest Data {.smaller}

Let's get the daily discharge measurements in Dane County, WI (parameter code = "00060") that has measured data within the last 14 days.
Let's get the continuous discharge measurements in Dane County, WI (parameter code = "00060") that have measured data within the last 14 days.

::: {.panel-tabset}

Expand Down Expand Up @@ -1125,7 +1176,7 @@ df, md = waterdata.get_daily(
```{r}
ggplot(data = daily) +
geom_line(aes(x = time, y = value, color = approval_status)) +
facet_grid(monitoring_location_id ~ ., )
facet_grid(monitoring_location_id ~ ., scale = "free_y")
```

### Python
Expand All @@ -1138,9 +1189,10 @@ import seaborn

levels, categories = pd.factorize(df["approval_status"])
graph = seaborn.FacetGrid(
df, row="monitoring_location_id", hue="approval_status", height=3, aspect=5
df, row="monitoring_location_id", hue="approval_status",
height=3, aspect=3, sharey=False
)
graph.map(plt.scatter, "time", "value").add_legend()
graph.map(plt.plot, "time", "value").add_legend()
```

:::
Expand Down Expand Up @@ -1282,11 +1334,7 @@ dt_me(df, escape = FALSE, paging = FALSE)

## Data Sources

The examples in these slides got all the data from the Water Data API:

<https://api.waterdata.usgs.gov/ogcapi/v0/>

The `dataretrieval` packages also include functions to access:
* [Water Data API](https://api.waterdata.usgs.gov/ogcapi/v0/)

* [Water Quality Portal](<https://www.waterqualitydata.us/)

Expand Down
Loading