diff --git a/.gitignore b/.gitignore index 497e825..42d27dd 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,3 @@ .httr-oauth .DS_Store ramptools.Rproj -*.html -*.Rd -docs diff --git a/R/make_maps.R b/R/make_maps.R new file mode 100644 index 0000000..e69de29 diff --git a/docs/404.html b/docs/404.html new file mode 100644 index 0000000..1e9ceac --- /dev/null +++ b/docs/404.html @@ -0,0 +1,115 @@ + + + + + + + +Page not found (404) • ramptools + + + + + + + + + + + +
+
+ + + + +
+
+
+

Page not found (404)

+
+ +Content not found. Please use links in the navbar. + +
+ + + +
+ + + + +
+ + + + + + + + diff --git a/docs/LICENSE-text.html b/docs/LICENSE-text.html new file mode 100644 index 0000000..add9725 --- /dev/null +++ b/docs/LICENSE-text.html @@ -0,0 +1,89 @@ + +License • ramptools + + +
+
+ + + +
+
+
+

License

+
+ +
YEAR: 2023
+COPYRIGHT HOLDER: ramptools authors
+
+ +
+ + + +
+ + + +
+ + + + + + + + diff --git a/docs/LICENSE.html b/docs/LICENSE.html new file mode 100644 index 0000000..291347e --- /dev/null +++ b/docs/LICENSE.html @@ -0,0 +1,93 @@ + +MIT License • ramptools + + +
+
+ + + +
+
+
+

MIT License

+
+ +
+ +

Copyright (c) 2023 ramptools authors

+

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

+

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

+

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

+
+ +
+ + + +
+ + + +
+ + + + + + + + diff --git a/docs/articles/cloud.html b/docs/articles/cloud.html new file mode 100644 index 0000000..3965957 --- /dev/null +++ b/docs/articles/cloud.html @@ -0,0 +1,158 @@ + + + + + + + +Big Query • ramptools + + + + + + + + + + + +
+
+ + + + +
+
+
+

Big Query

+ + + + + +
+ + + +
+

Setup +

+

The pipeline stores data in Google BigQuery under project +uganda-malaria, dataset uga_facility_data.

+

Tables:

+ ++++ + + + + + + + + + + + + + + + + + + + + + + +
TableDescription
raw_{frequency}_dataAppend-only versioned raw DHIS pulls
raw_{frequency}_version_metadataProvenance metadata per version
clean_{frequency}_dataLatest clean aggregated data (overwritten each run)
imputed_{frequency}_facility_dataFacility-level imputed data (overwritten each run)
+

To initialize:

+
+library(ramptools)
+bq_init_tables(frequency = "both")
+
+
+ + + +
+ + + + +
+ + + + + + + + diff --git a/docs/articles/index.html b/docs/articles/index.html new file mode 100644 index 0000000..c9f9e2a --- /dev/null +++ b/docs/articles/index.html @@ -0,0 +1,88 @@ + +Articles • ramptools + + +
+
+ + + +
+
+
+

Articles

+
+ +
+

All vignettes

+

+ +
Big Query
+
+
Installation
+
+
+
+
+ + +
+ + + + + + + + diff --git a/docs/articles/installation.html b/docs/articles/installation.html new file mode 100644 index 0000000..f454247 --- /dev/null +++ b/docs/articles/installation.html @@ -0,0 +1,124 @@ + + + + + + + +Installation • ramptools + + + + + + + + + + + +
+
+ + + + +
+
+
+

Installation

+ + + + + +
+ + + +

To install ramptools, you will need the +devtools package.

+
+install.packages("devtools")
+devtools::install_github("dd-harp/ramptools")
+
+ + + +
+ + + + +
+ + + + + + + + diff --git a/docs/authors.html b/docs/authors.html new file mode 100644 index 0000000..096bddc --- /dev/null +++ b/docs/authors.html @@ -0,0 +1,108 @@ + +Authors and Citation • ramptools + + +
+
+ + + +
+
+
+
+

Authors and Citation

+
+ + +
  • +

    Austin Carter. Author, maintainer. +

    +
    • +
+
+
+

Citation

+ +
+
+ + +

Carter A (2026). +ramptools: RAMP Uganda Malaria Data Tools. +R package version 0.2.0, https://dd-harp.github.io/ramptools/. +

+ 
@Manual{,
+  title = {ramptools: RAMP Uganda Malaria Data Tools},
+  author = {Austin Carter},
+  year = {2026},
+  note = {R package version 0.2.0},
+  url = {https://dd-harp.github.io/ramptools/},
+}
+ +
+ +
+ + + +
+ + + + + + + + diff --git a/docs/index.html b/docs/index.html new file mode 100644 index 0000000..115d2da --- /dev/null +++ b/docs/index.html @@ -0,0 +1,424 @@ + + + + + + + +RAMP Uganda Malaria Data Tools • ramptools + + + + + + + + + + + + +
+
+ + + + +
+
+ +
+

ramptools +

+

Shared R package for the Uganda RAMP (Routine Assessment of Malaria Programs) project. Provides metadata tables, shapefiles, and database utilities used by the ETL pipeline (uga-etl-facility-data) and analytics repos (outbreak). An R package providing utility functions and reference data for working with health information systems databases, particularly DHIS2 (District Health Information System 2) data from Uganda.

+
+

Overview +

+

ramptools simplifies working with versioned health data by providing:

+
    +
  • +Database utilities for managing and querying versioned SQLite databases
    • +
  • +Period handling for DHIS2 temporal data formats (weekly/monthly)
    • +
  • +Reference datasets for Uganda’s administrative geography and health indicators
    • +
  • +Data transformation utilities for working with health information data
    • +
+
+
+

Key Features +

+
+

Database Management +

+ +
+
+

Period Utilities +

+ +
+
+

Output Management +

+ +
+
+

Data Transformation +

+ +
+
+
+

Included Datasets +

+

The package includes reference data for Uganda:

+
+

Geographic Data +

+
    +
  • +uga_district_shp - District-level shapefiles
    • +
  • +uga_subcounty_shp - Subcounty-level shapefiles
    +
    • +
  • +uga_region_shp - Region-level shapefiles
    • +
  • +uga_water_shp - Water body shapefiles
    • +
+
+
+

Reference Tables +

+
    +
  • +loc_table - Location hierarchy with administrative units
    • +
  • +district_pop - Population data by district
    • +
  • +indicator_table - DHIS2 indicator definitions and metadata
    • +
  • +age_sex_table - Age-sex stratification reference data
    • +
+
+
+
+

Installation +

+
+# install.packages("devtools")
+devtools::install_github("dd-harp/ramptools")
+
+
+

Contents +

+
+

Data +

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ObjectDescription
loc_tableLocation hierarchy — 11,229 facilities and admin units with parent-child relationships (Uganda → Region → District → DLG → Subcounty → Facility)
health_facility_table8,676 health facilities with ownership, status, coordinates, facility type
indicator_table~100 DHIS2 data element mappings (DHIS ID → code_name, display_name, frequency, dhis_version)
district_popDistrict-level population estimates
age_sex_tableAge-sex disaggregated indicator definitions
uga_district_shpDistrict-level shapefile
uga_subcounty_shpSubcounty-level shapefile
uga_region_shpRegion-level shapefile
uga_water_shpWater body geometries
+
+
+

Functions +

+
+

SQLite utilities (local/legacy) +

+ +
+
+

BigQuery utilities (cloud) +

+ +
+
+

Period utilities +

+ +
+
+

Output management +

+ +
+
+
+
+

BigQuery Setup +

+

The pipeline stores data in Google BigQuery under project uganda-malaria, dataset uga_facility_data. Tables:

+ ++++ + + + + + + + + + + + + + + + + + + + + + + +
TableDescription
raw_{frequency}_dataAppend-only versioned raw DHIS pulls
raw_{frequency}_version_metadataProvenance metadata per version
clean_{frequency}_dataLatest clean aggregated data (overwritten each run)
imputed_{frequency}_facility_dataFacility-level imputed data (overwritten each run)
+

To initialize:

+
+library(ramptools)
+bq_init_tables(frequency = "both")
+
+
+
+

Install from source +

+

devtools::install_github(“yourusername/ramptools”) ```

+
+

Dependencies +

+
    +
  • data.table
    • +
  • DBI
    • +
  • RSQLite
    • +
  • dplyr
    • +
+
+
+

Use Case +

+

This package is designed for teams working with: - DHIS2 health information systems data - Versioned data workflows requiring audit trails - Uganda health and geographic data analysis - Time series analysis of health indicators

+
+
+

License +

+

MIT License

+
+
+ +
+ + +
+ + + +
+ + + + + + + + diff --git a/docs/pkgdown.yml b/docs/pkgdown.yml index c67840b..aa36823 100644 --- a/docs/pkgdown.yml +++ b/docs/pkgdown.yml @@ -4,7 +4,7 @@ pkgdown_sha: ~ articles: cloud: cloud.html installation: installation.html -last_built: 2026-03-04T22:20Z +last_built: 2026-03-04T22:37Z urls: reference: https://dd-harp.github.io/ramptools/reference article: https://dd-harp.github.io/ramptools/articles diff --git a/docs/reference/age_sex_table.html b/docs/reference/age_sex_table.html new file mode 100644 index 0000000..88efca7 --- /dev/null +++ b/docs/reference/age_sex_table.html @@ -0,0 +1,97 @@ + +Age-sex indicator table — age_sex_table • ramptools + + +
+
+ + + +
+
+
+

Age-sex indicator table

+ + +
+ +
+

Age-sex indicator table

+
+ +
+ 
age_sex_table
+
+ +
+

Format

+

An object of class data.table (inherits from data.frame) with 14 rows and 4 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_append_raw_data.html b/docs/reference/bq_append_raw_data.html new file mode 100644 index 0000000..6192a28 --- /dev/null +++ b/docs/reference/bq_append_raw_data.html @@ -0,0 +1,114 @@ + +Append raw data to BigQuery — bq_append_raw_data • ramptools + + +
+
+ + + +
+
+
+

Append raw data to BigQuery

+ + +
+ +
+

Append raw data to BigQuery

+
+ +
+ 
bq_append_raw_data(dt, con = NULL, frequency = "monthly", chunk_size = 500000L)
+
+ +
+

Arguments

+ + +
dt
+

data.table to append (must have: dataElement, period, orgUnit, +value, timestamp, version)

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
chunk_size
+

Upper limit on download chunk

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_append_version_metadata.html b/docs/reference/bq_append_version_metadata.html new file mode 100644 index 0000000..651b902 --- /dev/null +++ b/docs/reference/bq_append_version_metadata.html @@ -0,0 +1,109 @@ + +Write version metadata to BigQuery — bq_append_version_metadata • ramptools + + +
+
+ + + +
+
+
+

Write version metadata to BigQuery

+ + +
+ +
+

Write version metadata to BigQuery

+
+ +
+ 
bq_append_version_metadata(version_df, con = NULL, frequency = "monthly")
+
+ +
+

Arguments

+ + +
version_df
+

data.frame with version metadata

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_connect.html b/docs/reference/bq_connect.html new file mode 100644 index 0000000..035fd18 --- /dev/null +++ b/docs/reference/bq_connect.html @@ -0,0 +1,121 @@ + +BigQuery Utilities for RAMP Data Pipeline — bq_connect • ramptools + + +
+
+ + + +
+
+
+

BigQuery Utilities for RAMP Data Pipeline

+ + +
+ +
+

Read/write functions for interacting with Google BigQuery as the + primary data backend for DHIS2 facility data. +Create a BigQuery connection

+
+ +
+ 
bq_connect(
+  project = "uganda-malaria",
+  dataset = "uga_facility_data",
+  billing = project
+)
+
+ +
+

Arguments

+ + +
project
+

GCP project ID (default: "uganda-malaria")

+ + +
dataset
+

BigQuery dataset name (default: "uga_facility_data")

+ + +
billing
+

GCP project to bill for queries (defaults to project)

+ +
+
+

Value

+

A DBI connection object

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_clean_data.html b/docs/reference/bq_get_clean_data.html new file mode 100644 index 0000000..92064ed --- /dev/null +++ b/docs/reference/bq_get_clean_data.html @@ -0,0 +1,122 @@ + +Read clean aggregated data from BigQuery — bq_get_clean_data • ramptools + + +
+
+ + + +
+
+
+

Read clean aggregated data from BigQuery

+ + +
+ +
+

Read clean aggregated data from BigQuery

+
+ +
+ 
bq_get_clean_data(
+  con = NULL,
+  frequency = "monthly",
+  code_names = NULL,
+  levels = NULL
+)
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
code_names
+

Character vector of indicator code_names to filter (optional)

+ + +
levels
+

Integer vector of admin levels to filter (optional)

+ +
+
+

Value

+

data.table with clean aggregated data

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_data.html b/docs/reference/bq_get_data.html new file mode 100644 index 0000000..d7cf0dc --- /dev/null +++ b/docs/reference/bq_get_data.html @@ -0,0 +1,124 @@ + +Read raw DHIS data from BigQuery — bq_get_data • ramptools + + +
+
+ + + +
+
+
+

Read raw DHIS data from BigQuery

+ + +
+ +
+

Retrieves the latest version of each observation, analogous to +get_data for SQLite.

+
+ +
+ 
bq_get_data(
+  con = NULL,
+  frequency = "monthly",
+  id_list = NULL,
+  version_id = NULL
+)
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
id_list
+

Named list of ID columns and values to filter on (optional)

+ + +
version_id
+

Maximum version to include (NULL = latest)

+ +
+
+

Value

+

data.table with the requested data

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_db_diff.html b/docs/reference/bq_get_db_diff.html new file mode 100644 index 0000000..1c13bd4 --- /dev/null +++ b/docs/reference/bq_get_db_diff.html @@ -0,0 +1,115 @@ + +Get new/changed rows not yet in BigQuery — bq_get_db_diff • ramptools + + +
+
+ + + +
+
+
+

Get new/changed rows not yet in BigQuery

+ + +
+ +
+

Compares new_data against what is currently stored and returns +only the diff (new or changed values).

+
+ +
+ 
bq_get_db_diff(new_data, con = NULL, frequency = "monthly")
+
+ +
+

Arguments

+ + +
new_data
+

data.frame with freshly pulled DHIS data

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ +
+
+

Value

+

data.table of rows that are new or have changed values

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_imputed_sample.html b/docs/reference/bq_get_imputed_sample.html new file mode 100644 index 0000000..fb19d5f --- /dev/null +++ b/docs/reference/bq_get_imputed_sample.html @@ -0,0 +1,115 @@ + +Sample imputed facility data from BigQuery — bq_get_imputed_sample • ramptools + + +
+
+ + + +
+
+
+

Sample imputed facility data from BigQuery

+ + +
+ +
+

Returns a random sample of rows from the imputed facility data table. +Useful for computing outlier summaries without reading the full table.

+
+ +
+ 
bq_get_imputed_sample(con = NULL, frequency = "monthly", n = 500000L)
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
n
+

Maximum number of rows to sample

+ +
+
+

Value

+

data.table with sampled imputed data

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_latest_version.html b/docs/reference/bq_get_latest_version.html new file mode 100644 index 0000000..096621b --- /dev/null +++ b/docs/reference/bq_get_latest_version.html @@ -0,0 +1,109 @@ + +Get the latest version number from BigQuery — bq_get_latest_version • ramptools + + +
+
+ + + +
+
+
+

Get the latest version number from BigQuery

+ + +
+ +
+

Get the latest version number from BigQuery

+
+ +
+ 
bq_get_latest_version(con = NULL, frequency = "monthly")
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ +
+
+

Value

+

Integer value of the latest version

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_outbreak_data.html b/docs/reference/bq_get_outbreak_data.html new file mode 100644 index 0000000..b914855 --- /dev/null +++ b/docs/reference/bq_get_outbreak_data.html @@ -0,0 +1,105 @@ + +Read pre-computed outbreak index data from BigQuery — bq_get_outbreak_data • ramptools + + +
+
+ + + +
+
+
+

Read pre-computed outbreak index data from BigQuery

+ + +
+ +
+

Read pre-computed outbreak index data from BigQuery

+
+ +
+ 
bq_get_outbreak_data(con = NULL)
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ +
+
+

Value

+

data.table with outbreak index data

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_get_version_metadata.html b/docs/reference/bq_get_version_metadata.html new file mode 100644 index 0000000..57c6ece --- /dev/null +++ b/docs/reference/bq_get_version_metadata.html @@ -0,0 +1,113 @@ + +Get version metadata from BigQuery — bq_get_version_metadata • ramptools + + +
+
+ + + +
+
+
+

Get version metadata from BigQuery

+ + +
+ +
+

Get version metadata from BigQuery

+
+ +
+ 
bq_get_version_metadata(con = NULL, frequency = "monthly", version_id = NULL)
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
version_id
+

Version to retrieve (NULL = latest)

+ +
+
+

Value

+

data.table with version metadata

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_init_tables.html b/docs/reference/bq_init_tables.html new file mode 100644 index 0000000..cd0dce1 --- /dev/null +++ b/docs/reference/bq_init_tables.html @@ -0,0 +1,107 @@ + +Initialize BigQuery tables for the DHIS data pipeline — bq_init_tables • ramptools + + +
+
+ + + +
+
+
+

Initialize BigQuery tables for the DHIS data pipeline

+ + +
+ +
+

Creates the required tables in BigQuery if they don't exist. Run once during +initial setup.

+
+ +
+ 
bq_init_tables(con = NULL, frequency = "both")
+
+ +
+

Arguments

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly", "monthly", or "both"

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_write_clean_data.html b/docs/reference/bq_write_clean_data.html new file mode 100644 index 0000000..cf2b461 --- /dev/null +++ b/docs/reference/bq_write_clean_data.html @@ -0,0 +1,123 @@ + +Write clean aggregated data to BigQuery (full replace) — bq_write_clean_data • ramptools + + +
+
+ + + +
+
+
+

Write clean aggregated data to BigQuery (full replace)

+ + +
+ +
+

Overwrites the clean data table with the latest processed output.

+
+ +
+ 
bq_write_clean_data(
+  dt,
+  con = NULL,
+  frequency = "monthly",
+  chunk_size = 500000L,
+  append_mode = FALSE
+)
+
+ +
+

Arguments

+ + +
dt
+

data.table with clean aggregated data

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
chunk_size
+

Upper limit on upload

+ + +
append_mode
+

FALSE=overwrite, TRUE=append

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_write_imputed_data.html b/docs/reference/bq_write_imputed_data.html new file mode 100644 index 0000000..169073c --- /dev/null +++ b/docs/reference/bq_write_imputed_data.html @@ -0,0 +1,123 @@ + +Write imputed facility data to BigQuery (full replace) — bq_write_imputed_data • ramptools + + +
+
+ + + +
+
+
+

Write imputed facility data to BigQuery (full replace)

+ + +
+ +
+

Write imputed facility data to BigQuery (full replace)

+
+ +
+ 
bq_write_imputed_data(
+  dt,
+  con = NULL,
+  frequency = "monthly",
+  chunk_size = 500000L,
+  append_mode = FALSE
+)
+
+ +
+

Arguments

+ + +
dt
+

data.table with imputed facility-level data

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ + +
frequency
+

Either "weekly" or "monthly"

+ + +
chunk_size
+

Upper limit on upload

+ + +
append_mode
+

FALSE=overwrite, TRUE=append

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/bq_write_outbreak_data.html b/docs/reference/bq_write_outbreak_data.html new file mode 100644 index 0000000..fe8428c --- /dev/null +++ b/docs/reference/bq_write_outbreak_data.html @@ -0,0 +1,109 @@ + +Write pre-computed outbreak index data to BigQuery — bq_write_outbreak_data • ramptools + + +
+
+ + + +
+
+
+

Write pre-computed outbreak index data to BigQuery

+ + +
+ +
+

Overwrites the outbreak_data table with the latest computed outbreak +indices. Designed to be called after clean/aggregate to keep the Shiny app +read-only.

+
+ +
+ 
bq_write_outbreak_data(dt, con = NULL)
+
+ +
+

Arguments

+ + +
dt
+

data.table with columns: date, code_name, name, value, Year, Month

+ + +
con
+

A DBI BigQuery connection (or NULL to create one)

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/district_pop.html b/docs/reference/district_pop.html new file mode 100644 index 0000000..05f2bf7 --- /dev/null +++ b/docs/reference/district_pop.html @@ -0,0 +1,97 @@ + +District population data — district_pop • ramptools + + +
+
+ + + +
+
+
+

District population data

+ + +
+ +
+

District population data

+
+ +
+ 
district_pop
+
+ +
+

Format

+

An object of class data.table (inherits from data.frame) with 146 rows and 2 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_data.html b/docs/reference/get_data.html new file mode 100644 index 0000000..28ee0bb --- /dev/null +++ b/docs/reference/get_data.html @@ -0,0 +1,109 @@ + +Get the data from a database with the option to subset to a specified version and id values — get_data • ramptools + + +
+
+ + + +
+
+
+

Get the data from a database with the option to subset to a specified version and id values

+ + +
+ +
+

Get the data from a database with the option to subset to a specified version and id values

+
+ +
+ 
get_data(db_path, id_list = NULL, version_id = NULL)
+
+ +
+

Arguments

+ + +
db_path
+

Location of the database

+ + +
id_list
+

A list of id names and values, if NULL, pulls full table

+ + +
version_id
+

Integer version number, if NULL, pulls latest version

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_db_diff.html b/docs/reference/get_db_diff.html new file mode 100644 index 0000000..e1e46b5 --- /dev/null +++ b/docs/reference/get_db_diff.html @@ -0,0 +1,109 @@ + +Get the data from new data that differs from what is present in the database — get_db_diff • ramptools + + +
+
+ + + +
+
+
+

Get the data from new data that differs from what is present in the database

+ + +
+ +
+

Get the data from new data that differs from what is present in the database

+
+ +
+ 
get_db_diff(new_data, db_path)
+
+ +
+

Arguments

+ + +
new_data
+

data.frame with new data for comparison to the data in the database

+ + +
db_path
+

Location of the database

+ +
+
+

Value

+

data.frame with the data from new data that differs from what is present in the database

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_id_vars.html b/docs/reference/get_id_vars.html new file mode 100644 index 0000000..6df3fd0 --- /dev/null +++ b/docs/reference/get_id_vars.html @@ -0,0 +1,105 @@ + +Get the names of the variables that uniquely identify an observation — get_id_vars • ramptools + + +
+
+ + + +
+
+
+

Get the names of the variables that uniquely identify an observation

+ + +
+ +
+

Get the names of the variables that uniquely identify an observation

+
+ +
+ 
get_id_vars(db_path)
+
+ +
+

Arguments

+ + +
db_path
+

Location of the database

+ +
+
+

Value

+

Vector of column names that uniquely identify an observation

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_latest_output_date_index.html b/docs/reference/get_latest_output_date_index.html new file mode 100644 index 0000000..c1af880 --- /dev/null +++ b/docs/reference/get_latest_output_date_index.html @@ -0,0 +1,112 @@ + +get the latest index for given an output dir and a date — get_latest_output_date_index • ramptools + + +
+
+ + + +
+
+
+

get the latest index for given an output dir and a date

+ + +
+ +
+

directories are assumed to be named in YYYY_MM_DD.VV format with sane +year/month/date/version values.

+
+ +
+ 
get_latest_output_date_index(dir, date)
+
+ +
+

Arguments

+ + +
dir
+

path to directory with versioned dirs

+ + +
date
+

character in be YYYY_MM_DD format

+ +
+
+

Value

+

largest version in directory tree or 0 if there are no version OR +the directory tree does not exist

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_latest_version.html b/docs/reference/get_latest_version.html new file mode 100644 index 0000000..96c2528 --- /dev/null +++ b/docs/reference/get_latest_version.html @@ -0,0 +1,105 @@ + +Get the latest version number from the database — get_latest_version • ramptools + + +
+
+ + + +
+
+
+

Get the latest version number from the database

+ + +
+ +
+

Get the latest version number from the database

+
+ +
+ 
get_latest_version(db_path)
+
+ +
+

Arguments

+ + +
db_path
+

Location of the database

+ +
+
+

Value

+

Integer value of latest version

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_output_dir.html b/docs/reference/get_output_dir.html new file mode 100644 index 0000000..fd7015d --- /dev/null +++ b/docs/reference/get_output_dir.html @@ -0,0 +1,105 @@ + +Get output directory for results to save in — get_output_dir • ramptools + + +
+
+ + + +
+
+
+

Get output directory for results to save in

+ + +
+ +
+

Returns an appropriate path to save results in, creating it if necessary.

+
+ +
+ 
get_output_dir(root, date)
+
+ +
+

Arguments

+ + +
root
+

path to root of output results

+ + +
date
+

character date in form of "YYYY_MM_DD" or "today". "today" will be interpreted as today's date.

+ +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_period_range.html b/docs/reference/get_period_range.html new file mode 100644 index 0000000..1bd7b9c --- /dev/null +++ b/docs/reference/get_period_range.html @@ -0,0 +1,121 @@ + +Get the a vector of periods follow DHIS formatting that spans that start and end period provided — get_period_range • ramptools + + +
+
+ + + +
+
+
+

Get the a vector of periods follow DHIS formatting that spans that start and end period provided

+ + +
+ +
+

Get the a vector of periods follow DHIS formatting that spans that start and end period provided

+
+ +
+ 
get_period_range(frequency, year_start, sub_year_start, year_end, sub_year_end)
+
+ +
+

Arguments

+ + +
frequency
+

Text of either weekly or monthly

+ + +
year_start
+

Integer for start year

+ + +
sub_year_start
+

Integer for either week or month of the year to start

+ + +
year_end
+

Integer for end year

+ + +
sub_year_end
+

Integer for either week or month of the year to end

+ +
+
+

Value

+

A vector of periods following the DHIS formatting standard of either YYYYW{week number with no leading zero} or YYYY{month number with a leading zero for single digit integers}

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_value_var.html b/docs/reference/get_value_var.html new file mode 100644 index 0000000..a7083b2 --- /dev/null +++ b/docs/reference/get_value_var.html @@ -0,0 +1,105 @@ + +Get the name of the value column — get_value_var • ramptools + + +
+
+ + + +
+
+
+

Get the name of the value column

+ + +
+ +
+

Get the name of the value column

+
+ +
+ 
get_value_var(db_path)
+
+ +
+

Arguments

+ + +
db_path
+

Location of the database

+ +
+
+

Value

+

Column name for the value

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/get_version_metadata.html b/docs/reference/get_version_metadata.html new file mode 100644 index 0000000..43976a0 --- /dev/null +++ b/docs/reference/get_version_metadata.html @@ -0,0 +1,109 @@ + +Get metadata for a specific version — get_version_metadata • ramptools + + +
+
+ + + +
+
+
+

Get metadata for a specific version

+ + +
+ +
+

Get metadata for a specific version

+
+ +
+ 
get_version_metadata(db_path, version_id = NULL)
+
+ +
+

Arguments

+ + +
db_path
+

Location of the database

+ + +
version_id
+

Integer for version number to pull metadata for

+ +
+
+

Value

+

Integer value of latest version

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/health_facility_table.html b/docs/reference/health_facility_table.html new file mode 100644 index 0000000..e156342 --- /dev/null +++ b/docs/reference/health_facility_table.html @@ -0,0 +1,165 @@ + +Health Facility and Administrative Units Table — health_facility_table • ramptools + + +
+
+ + + +
+
+
+

Health Facility and Administrative Units Table

+ + +
+ +
+

A comprehensive list of health facilities and administrative hierarchies, +including ownership, operational status, and geographic coordinates.

+
+ +
+ 
health_facility_table
+
+ +
+

Format

+

A data frame with 8676 rows and 22 columns:

location_name
+

Name of Health facility

+ +
location_id
+

DHIS2 assigned ID for health facility

+ +
parent_id
+

DHIS2 assigned ID of parent health facility (subcounty)

+ +
level
+

DHIS2 assigned hierarchical level

+ +
authority
+

The governing body or authority responsible for the facility

+ +
facility_level
+

The original classification level of the health facility

+ +
medical_bureaus
+

The medical bureau affiliation (e.g., UPMB, UCMB)

+ +
operational_status
+

Current status of the facility (e.g., Functional, Non-Functional)

+ +
ownership
+

Type of ownership (e.g., Public, Private, PNFP)

+ +
private_facilities
+

Indicator for private sector classification (PFP, PNFP)

+ +
public_facilities
+

Indicator for public sector classification (e.g. MOH,BOU, Local Government)

+ +
reporting_status
+

Current status of data reporting compliance

+ +
path_to_top_parent
+

String of DHIS2 IDs from current health facility to the top-level parent

+ +
region_name
+

Name of region associated with health facility

+ +
district_name
+

Name of district associated with health facility

+ +
DLG_name
+

Name of District Local Government (DLG) associated with health facility

+ +
subcounty_name
+

Name of subcounty associated with health facility

+ +
latitude
+

Geographic coordinate: North-South position

+ +
longitude
+

Geographic coordinate: East-West position

+ +
geom_enhanced
+

Indicator if geometry was added through external web-scraping and not present in DHIS2

+ +
facility_level_2
+

Secondary facility level classification (clinic, hospital, drugshop, other)

+ +
facility_type
+

Secondary facility label created to match other surveys on health seeking (private, gov_HC, private_hosp, drugshop, gov_hosp, other)

+ + +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/index.html b/docs/reference/index.html new file mode 100644 index 0000000..111a1e3 --- /dev/null +++ b/docs/reference/index.html @@ -0,0 +1,247 @@ + +Package index • ramptools + + +
+
+ + + +
+
+
+

Reference

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

Metadata

+

Data Objects

+
+

loc_table

+

Location hierarchy table

+

health_facility_table

+

Health Facility and Administrative Units Table

+

indicator_table

+

DHIS indicator table

+

district_pop

+

District population data

+

age_sex_table

+

Age-sex indicator table

+

Data Utilities

+

Utilities to Work with Metadata

+
+

get_latest_output_date_index()

+

get the latest index for given an output dir and a date

+

get_output_dir()

+

Get output directory for results to save in

+

get_period_range()

+

Get the a vector of periods follow DHIS formatting that spans that start and end period provided

+

make_month_map()

+

Make a table that provides the date information associated with a DHIS period

+

make_week_map()

+

Make a table that provides the date information associated with a DHIS period

+

Shapefiles

+

GIS Shapefiles

+
+

uga_subcounty_shp

+

Subcounty shapefile version2.0

+

uga_district_shp

+

District shapefile version1.0

+

uga_region_shp

+

region shapefile version1.0

+

uga_water_shp

+

Water shapefile version1.0

+

SQLite Utilities

+

Data Objects

+
+

get_data()

+

Get the data from a database with the option to subset to a specified version and id values

+

get_db_diff()

+

Get the data from new data that differs from what is present in the database

+

get_latest_version()

+

Get the latest version number from the database

+

get_version_metadata()

+

Get metadata for a specific version

+

get_id_vars()

+

Get the names of the variables that uniquely identify an observation

+

get_value_var()

+

Get the name of the value column

+

make_human_readable()

+

Merge on human readable columns to raw data

+

Cloud Utilities

+

Utilities to Access the Data Warehouse

+
+

bq_connect()

+

BigQuery Utilities for RAMP Data Pipeline

+

bq_get_latest_version()

+

Get the latest version number from BigQuery

+

bq_get_version_metadata()

+

Get version metadata from BigQuery

+

bq_get_data()

+

Read raw DHIS data from BigQuery

+

bq_get_db_diff()

+

Get new/changed rows not yet in BigQuery

+

bq_get_clean_data()

+

Read clean aggregated data from BigQuery

+

bq_get_imputed_sample()

+

Sample imputed facility data from BigQuery

+

bq_append_raw_data()

+

Append raw data to BigQuery

+

bq_append_version_metadata()

+

Write version metadata to BigQuery

+

bq_write_clean_data()

+

Write clean aggregated data to BigQuery (full replace)

+

bq_write_imputed_data()

+

Write imputed facility data to BigQuery (full replace)

+

bq_init_tables()

+

Initialize BigQuery tables for the DHIS data pipeline

+

bq_write_outbreak_data()

+

Write pre-computed outbreak index data to BigQuery

+

bq_get_outbreak_data()

+

Read pre-computed outbreak index data from BigQuery

+

ramptools ramptools-package

+

ramptools: RAMP Uganda Malaria Data Tools

+ + +
+ + +
+ + + + + + + + diff --git a/docs/reference/indicator_table.html b/docs/reference/indicator_table.html new file mode 100644 index 0000000..441d557 --- /dev/null +++ b/docs/reference/indicator_table.html @@ -0,0 +1,97 @@ + +DHIS indicator table — indicator_table • ramptools + + +
+
+ + + +
+
+
+

DHIS indicator table

+ + +
+ +
+

DHIS indicator table

+
+ +
+ 
indicator_table
+
+ +
+

Format

+

An object of class data.table (inherits from data.frame) with 118 rows and 6 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/loc_table.html b/docs/reference/loc_table.html new file mode 100644 index 0000000..1a2bf9e --- /dev/null +++ b/docs/reference/loc_table.html @@ -0,0 +1,124 @@ + +Location hierarchy table — loc_table • ramptools + + +
+
+ + + +
+
+
+

Location hierarchy table

+ + +
+ +
+

A list of all health facilities and administrative units, and their parent-child relationships

+
+ +
+ 
loc_table
+
+ +
+

Format

+

A data frame with 11219 rows and 9 columns:

location_name
+

Name of Geographic Entity

+ +
location_id
+

DHIS2 assigned ID for geographic entity

+ +
parent_id
+

DHIS2 assigned ID of parent geographic entity

+ +
level
+

DHIS2 assigned hierarchical level

+ +
path_to_top_parent
+

string of DHIS2 IDs from current geographic entity to Uganda as a whole

+ +
region_name
+

name of region associated with geographic entity. Blank is entity is region

+ +
district_name
+

name of district associated with geographic entity. Blank is entity is district

+ +
DLG_name
+

name of DLG (District Level Government) associated with geographic entity. Blank is entity is DLG

+ +
subcounty_name
+

name of subcounty associated with geographic entity. Blank is entity is subcounty

+ + +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/make_human_readable.html b/docs/reference/make_human_readable.html new file mode 100644 index 0000000..70d19bf --- /dev/null +++ b/docs/reference/make_human_readable.html @@ -0,0 +1,105 @@ + +Merge on human readable columns to raw data — make_human_readable • ramptools + + +
+
+ + + +
+
+
+

Merge on human readable columns to raw data

+ + +
+ +
+

Merge on human readable columns to raw data

+
+ +
+ 
make_human_readable(dt)
+
+ +
+

Arguments

+ + +
dt
+

data.table with raw data in it

+ +
+
+

Value

+

data.table with human readable columns merged on

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/make_month_map.html b/docs/reference/make_month_map.html new file mode 100644 index 0000000..502454f --- /dev/null +++ b/docs/reference/make_month_map.html @@ -0,0 +1,105 @@ + +Make a table that provides the date information associated with a DHIS period — make_month_map • ramptools + + +
+
+ + + +
+
+
+

Make a table that provides the date information associated with a DHIS period

+ + +
+ +
+

Make a table that provides the date information associated with a DHIS period

+
+ +
+ 
make_month_map(min_date = "2013-01-01")
+
+ +
+

Arguments

+ + +
min_date
+

Date string in YYYY-MM-DD format for start date of the map

+ +
+
+

Value

+

A table with the period in DHIS format, year, month, date_start, date_mid, and date_end

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/make_week_map.html b/docs/reference/make_week_map.html new file mode 100644 index 0000000..ded8ebf --- /dev/null +++ b/docs/reference/make_week_map.html @@ -0,0 +1,105 @@ + +Make a table that provides the date information associated with a DHIS period — make_week_map • ramptools + + +
+
+ + + +
+
+
+

Make a table that provides the date information associated with a DHIS period

+ + +
+ +
+

Make a table that provides the date information associated with a DHIS period

+
+ +
+ 
make_week_map(min_date = "2012-12-31")
+
+ +
+

Arguments

+ + +
min_date
+

Date string in YYYY-MM-DD format for start date of the map

+ +
+
+

Value

+

A table with the period in DHIS format, year, week, date_start, date_mid, and date_end

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/ramptools-package.html b/docs/reference/ramptools-package.html new file mode 100644 index 0000000..0bdc551 --- /dev/null +++ b/docs/reference/ramptools-package.html @@ -0,0 +1,99 @@ + +ramptools: RAMP Uganda Malaria Data Tools — ramptools-package • ramptools + + +
+
+ + + +
+
+
+

ramptools: RAMP Uganda Malaria Data Tools

+ + +
+ +
+

Shared metadata tables and utility functions for the Uganda RAMP (Routine Assessment of Malaria Programs) project. Provides location hierarchies, indicator mappings, shapefiles, and database read/write utilities for both local SQLite and Google BigQuery backends.

+
+ + +
+

See also

+

Useful links:

+
+
+

Author

+

Maintainer: Austin Carter aucarter@uw.edu (ORCID)

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/ramptools.html b/docs/reference/ramptools.html new file mode 100644 index 0000000..cf283fa --- /dev/null +++ b/docs/reference/ramptools.html @@ -0,0 +1,8 @@ + + + + + + + + diff --git a/docs/reference/uga_district_shp.html b/docs/reference/uga_district_shp.html new file mode 100644 index 0000000..4d82ef4 --- /dev/null +++ b/docs/reference/uga_district_shp.html @@ -0,0 +1,97 @@ + +District shapefile version1.0 — uga_district_shp • ramptools + + +
+
+ + + +
+
+
+

District shapefile version1.0

+ + +
+ +
+

District shapefile version1.0

+
+ +
+ 
uga_district_shp
+
+ +
+

Format

+

An object of class sf (inherits from tbl_df, tbl, data.frame) with 146 rows and 3 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/uga_region_shp.html b/docs/reference/uga_region_shp.html new file mode 100644 index 0000000..b007786 --- /dev/null +++ b/docs/reference/uga_region_shp.html @@ -0,0 +1,97 @@ + +region shapefile version1.0 — uga_region_shp • ramptools + + +
+
+ + + +
+
+
+

region shapefile version1.0

+ + +
+ +
+

region shapefile version1.0

+
+ +
+ 
uga_region_shp
+
+ +
+

Format

+

An object of class sf (inherits from tbl_df, tbl, data.frame) with 15 rows and 3 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/uga_subcounty_shp.html b/docs/reference/uga_subcounty_shp.html new file mode 100644 index 0000000..f7a439e --- /dev/null +++ b/docs/reference/uga_subcounty_shp.html @@ -0,0 +1,120 @@ + +Subcounty shapefile version2.0 — uga_subcounty_shp • ramptools + + +
+
+ + + +
+
+
+

Subcounty shapefile version2.0

+ + +
+ +
+

An R readable, valid, shapefile with subcounty shapes combatible with administrative +hierarchies in loc_table

+
+ +
+ 
uga_subcounty_shp
+
+ +
+

Format

+

A shapefile with 2204 rows and 8 variables

name
+

Name of Subcounty

+ +
id
+

DHIS2 assigned ID of Subcounty

+ +
geometry
+

geometry of subcounty (polygon or multipolygon)

+ +
dup_geom
+

indicator flag if the geometry is shared with another subcounty and cannot be resolved

+ +
split_sc
+

indicator flag that geometry needs to be split, but has not been done

+ +
shp_as_district
+

indicator flag if the subcounty shape is unknown and has therefore been assigned the district shape

+ +
district_name
+

Name of parent district of subcounty

+ + +
+ +
+ +
+ + +
+ + + + + + + + diff --git a/docs/reference/uga_water_shp.html b/docs/reference/uga_water_shp.html new file mode 100644 index 0000000..f65661d --- /dev/null +++ b/docs/reference/uga_water_shp.html @@ -0,0 +1,97 @@ + +Water shapefile version1.0 — uga_water_shp • ramptools + + +
+
+ + + +
+
+
+

Water shapefile version1.0

+ + +
+ +
+

Water shapefile version1.0

+
+ +
+ 
uga_water_shp
+
+ +
+

Format

+

An object of class sf (inherits from tbl_df, tbl, data.frame) with 217 rows and 2 columns.

+
+ +
+ +
+ + +
+ + + + + + + + diff --git a/man/bq_append_raw_data.Rd b/man/bq_append_raw_data.Rd new file mode 100644 index 0000000..60ca826 --- /dev/null +++ b/man/bq_append_raw_data.Rd @@ -0,0 +1,21 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_append_raw_data} +\alias{bq_append_raw_data} +\title{Append raw data to BigQuery} +\usage{ +bq_append_raw_data(dt, con = NULL, frequency = "monthly", chunk_size = 500000L) +} +\arguments{ +\item{dt}{data.table to append (must have: dataElement, period, orgUnit, +value, timestamp, version)} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{chunk_size}{Upper limit on download chunk} +} +\description{ +Append raw data to BigQuery +} diff --git a/man/bq_append_version_metadata.Rd b/man/bq_append_version_metadata.Rd new file mode 100644 index 0000000..6baa6d5 --- /dev/null +++ b/man/bq_append_version_metadata.Rd @@ -0,0 +1,18 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_append_version_metadata} +\alias{bq_append_version_metadata} +\title{Write version metadata to BigQuery} +\usage{ +bq_append_version_metadata(version_df, con = NULL, frequency = "monthly") +} +\arguments{ +\item{version_df}{data.frame with version metadata} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} +} +\description{ +Write version metadata to BigQuery +} diff --git a/man/bq_connect.Rd b/man/bq_connect.Rd new file mode 100644 index 0000000..6652dab --- /dev/null +++ b/man/bq_connect.Rd @@ -0,0 +1,27 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_connect} +\alias{bq_connect} +\title{BigQuery Utilities for RAMP Data Pipeline} +\usage{ +bq_connect( + project = "uganda-malaria", + dataset = "uga_facility_data", + billing = project +) +} +\arguments{ +\item{project}{GCP project ID (default: \code{"uganda-malaria"})} + +\item{dataset}{BigQuery dataset name (default: \code{"uga_facility_data"})} + +\item{billing}{GCP project to bill for queries (defaults to \code{project})} +} +\value{ +A DBI connection object +} +\description{ +Read/write functions for interacting with Google BigQuery as the + primary data backend for DHIS2 facility data. +Create a BigQuery connection +} diff --git a/man/bq_get_clean_data.Rd b/man/bq_get_clean_data.Rd new file mode 100644 index 0000000..0cb4183 --- /dev/null +++ b/man/bq_get_clean_data.Rd @@ -0,0 +1,28 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_clean_data} +\alias{bq_get_clean_data} +\title{Read clean aggregated data from BigQuery} +\usage{ +bq_get_clean_data( + con = NULL, + frequency = "monthly", + code_names = NULL, + levels = NULL +) +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{code_names}{Character vector of indicator code_names to filter (optional)} + +\item{levels}{Integer vector of admin levels to filter (optional)} +} +\value{ +data.table with clean aggregated data +} +\description{ +Read clean aggregated data from BigQuery +} diff --git a/man/bq_get_data.Rd b/man/bq_get_data.Rd new file mode 100644 index 0000000..8a2bda7 --- /dev/null +++ b/man/bq_get_data.Rd @@ -0,0 +1,29 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_data} +\alias{bq_get_data} +\title{Read raw DHIS data from BigQuery} +\usage{ +bq_get_data( + con = NULL, + frequency = "monthly", + id_list = NULL, + version_id = NULL +) +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{id_list}{Named list of ID columns and values to filter on (optional)} + +\item{version_id}{Maximum version to include (NULL = latest)} +} +\value{ +data.table with the requested data +} +\description{ +Retrieves the latest version of each observation, analogous to +\code{\link{get_data}} for SQLite. +} diff --git a/man/bq_get_db_diff.Rd b/man/bq_get_db_diff.Rd new file mode 100644 index 0000000..ecff170 --- /dev/null +++ b/man/bq_get_db_diff.Rd @@ -0,0 +1,22 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_db_diff} +\alias{bq_get_db_diff} +\title{Get new/changed rows not yet in BigQuery} +\usage{ +bq_get_db_diff(new_data, con = NULL, frequency = "monthly") +} +\arguments{ +\item{new_data}{data.frame with freshly pulled DHIS data} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} +} +\value{ +data.table of rows that are new or have changed values +} +\description{ +Compares \code{new_data} against what is currently stored and returns +only the diff (new or changed values). +} diff --git a/man/bq_get_imputed_sample.Rd b/man/bq_get_imputed_sample.Rd new file mode 100644 index 0000000..9ed1320 --- /dev/null +++ b/man/bq_get_imputed_sample.Rd @@ -0,0 +1,22 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_imputed_sample} +\alias{bq_get_imputed_sample} +\title{Sample imputed facility data from BigQuery} +\usage{ +bq_get_imputed_sample(con = NULL, frequency = "monthly", n = 500000L) +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{n}{Maximum number of rows to sample} +} +\value{ +data.table with sampled imputed data +} +\description{ +Returns a random sample of rows from the imputed facility data table. +Useful for computing outlier summaries without reading the full table. +} diff --git a/man/bq_get_latest_version.Rd b/man/bq_get_latest_version.Rd new file mode 100644 index 0000000..476871e --- /dev/null +++ b/man/bq_get_latest_version.Rd @@ -0,0 +1,19 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_latest_version} +\alias{bq_get_latest_version} +\title{Get the latest version number from BigQuery} +\usage{ +bq_get_latest_version(con = NULL, frequency = "monthly") +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} +} +\value{ +Integer value of the latest version +} +\description{ +Get the latest version number from BigQuery +} diff --git a/man/bq_get_outbreak_data.Rd b/man/bq_get_outbreak_data.Rd new file mode 100644 index 0000000..d61dfc7 --- /dev/null +++ b/man/bq_get_outbreak_data.Rd @@ -0,0 +1,17 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_outbreak_data} +\alias{bq_get_outbreak_data} +\title{Read pre-computed outbreak index data from BigQuery} +\usage{ +bq_get_outbreak_data(con = NULL) +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} +} +\value{ +data.table with outbreak index data +} +\description{ +Read pre-computed outbreak index data from BigQuery +} diff --git a/man/bq_get_version_metadata.Rd b/man/bq_get_version_metadata.Rd new file mode 100644 index 0000000..8924294 --- /dev/null +++ b/man/bq_get_version_metadata.Rd @@ -0,0 +1,21 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_get_version_metadata} +\alias{bq_get_version_metadata} +\title{Get version metadata from BigQuery} +\usage{ +bq_get_version_metadata(con = NULL, frequency = "monthly", version_id = NULL) +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{version_id}{Version to retrieve (NULL = latest)} +} +\value{ +data.table with version metadata +} +\description{ +Get version metadata from BigQuery +} diff --git a/man/bq_init_tables.Rd b/man/bq_init_tables.Rd new file mode 100644 index 0000000..530a326 --- /dev/null +++ b/man/bq_init_tables.Rd @@ -0,0 +1,17 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_init_tables} +\alias{bq_init_tables} +\title{Initialize BigQuery tables for the DHIS data pipeline} +\usage{ +bq_init_tables(con = NULL, frequency = "both") +} +\arguments{ +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"}, \code{"monthly"}, or \code{"both"}} +} +\description{ +Creates the required tables in BigQuery if they don't exist. Run once during +initial setup. +} diff --git a/man/bq_write_clean_data.Rd b/man/bq_write_clean_data.Rd new file mode 100644 index 0000000..7ba2d0a --- /dev/null +++ b/man/bq_write_clean_data.Rd @@ -0,0 +1,28 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_write_clean_data} +\alias{bq_write_clean_data} +\title{Write clean aggregated data to BigQuery (full replace)} +\usage{ +bq_write_clean_data( + dt, + con = NULL, + frequency = "monthly", + chunk_size = 500000L, + append_mode = FALSE +) +} +\arguments{ +\item{dt}{data.table with clean aggregated data} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{chunk_size}{Upper limit on upload} + +\item{append_mode}{FALSE=overwrite, TRUE=append} +} +\description{ +Overwrites the clean data table with the latest processed output. +} diff --git a/man/bq_write_imputed_data.Rd b/man/bq_write_imputed_data.Rd new file mode 100644 index 0000000..4d8203f --- /dev/null +++ b/man/bq_write_imputed_data.Rd @@ -0,0 +1,28 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_write_imputed_data} +\alias{bq_write_imputed_data} +\title{Write imputed facility data to BigQuery (full replace)} +\usage{ +bq_write_imputed_data( + dt, + con = NULL, + frequency = "monthly", + chunk_size = 500000L, + append_mode = FALSE +) +} +\arguments{ +\item{dt}{data.table with imputed facility-level data} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} + +\item{frequency}{Either \code{"weekly"} or \code{"monthly"}} + +\item{chunk_size}{Upper limit on upload} + +\item{append_mode}{FALSE=overwrite, TRUE=append} +} +\description{ +Write imputed facility data to BigQuery (full replace) +} diff --git a/man/bq_write_outbreak_data.Rd b/man/bq_write_outbreak_data.Rd new file mode 100644 index 0000000..0f3029e --- /dev/null +++ b/man/bq_write_outbreak_data.Rd @@ -0,0 +1,18 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/bq_utils.R +\name{bq_write_outbreak_data} +\alias{bq_write_outbreak_data} +\title{Write pre-computed outbreak index data to BigQuery} +\usage{ +bq_write_outbreak_data(dt, con = NULL) +} +\arguments{ +\item{dt}{data.table with columns: date, code_name, name, value, Year, Month} + +\item{con}{A DBI BigQuery connection (or NULL to create one)} +} +\description{ +Overwrites the \code{outbreak_data} table with the latest computed outbreak +indices. Designed to be called after clean/aggregate to keep the Shiny app +read-only. +} diff --git a/vignettes/cloud.html b/vignettes/cloud.html new file mode 100644 index 0000000..b028cb1 --- /dev/null +++ b/vignettes/cloud.html @@ -0,0 +1,400 @@ + + + + + + + + + + + + + + +Big Query + + + + + + + + + + + + + + + + + + + + + + + + + + +

Big Query

+ + + +
+

Setup

+

The pipeline stores data in Google BigQuery under project +uganda-malaria, dataset uga_facility_data.

+

Tables:

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + +
TableDescription
raw_{frequency}_dataAppend-only versioned raw DHIS pulls
raw_{frequency}_version_metadataProvenance metadata per version
clean_{frequency}_dataLatest clean aggregated data (overwritten each run)
imputed_{frequency}_facility_dataFacility-level imputed data (overwritten each run)
+

To initialize:

+
library(ramptools)
+bq_init_tables(frequency = "both")
+
+ + + + + + + + + + + diff --git a/vignettes/installation.html b/vignettes/installation.html new file mode 100644 index 0000000..e7658d3 --- /dev/null +++ b/vignettes/installation.html @@ -0,0 +1,365 @@ + + + + + + + + + + + + + + +Installation + + + + + + + + + + + + + + + + + + + + + + + + + + +

Installation

+ + + +

To install ramptools, you will need the +devtools package.

+
install.packages("devtools")
+devtools::install_github("dd-harp/ramptools")
+ + + + + + + + + + +