Testing Guidelines and Examples

Testing in toaster

We use testthat package and follow general guidelines for testing R packages found here.

All tests files live in tests/testthat directory under names that start with test- followed by function name being tested.

There is also helper file helper-tests.R that hosts utility functions shared across many tests. If your tests share some functionality then it probably belongs to this file as a stand-alone function.

toaster functions are divided into 2 types: compute functions that use RODBC connection to run SQL statement(s) in Aster and create functions that produce visualizations. compute functions should be fully testable for errors, warnings, and success. create functions are usually tested for errors only (when possible).

How to write testable function

Tests never use live connections as CRAN runs them with vanilla R. Instead, test should check generated SQL for correctness. For this reason, each compute function (function that accepts RODBC connection to run SQL in Aster) defines test parameter. When it's set to TRUE instead of using live connection and running SQL, function should return character string containing SQL statement(s).

Thus, tests simply call function with test=TRUE to check generated SQL for correctness. For example, function computeAggregates test:

    expect_equal_normalized(computeAggregates(channel=NULL, "teams_enh",
                                              by = c("teamid", "decadeid"),
                                              aggregates = c("min(rank) minrank", "max(rank) maxrank"),
                                              where = "lgid = 'AL'", 
                                              test = TRUE),
                      "SELECT teamid, decadeid, min(rank) minrank, max(rank) maxrank
                         FROM teams_enh
                        WHERE lgid = 'AL'
                        GROUP BY teamid, decadeid"
                      )

If we look inside computeAggregates function we can see its body consists of 3 logical parts. First part checks and validates parameters and their integrity:

computeAggregates <- function(channel, tableName, 
                          aggregates = c("COUNT(*) cnt"), 
                          by = vector(), where = NULL, 
                          stringsAsFactors = FALSE, test = FALSE) {

  if (missing(tableName)) {
     stop("Must have table name.")
  }

  if (missing(by) || length(by) == 0) {
     stop("Must have one or more columns/expressions in 'by' parameter.")
  }

  if (is.null(aggregates) || length(aggregates) < 1) {
      stop("Must have at least one aggregate defined.")
  }

In fact, it's imperative to have tests for each error message and integrity check.

Middle part contains main logic that constructs SQL to execute:

  where_clause = makeWhereClause(where)

  columnExpr = sub(by, pattern = " [a-zA-Z0-9_]*$", replacement = "")

  # construct column list
  columnList = paste(paste(by, collapse=", "), paste(aggregates, collapse=", "), sep=", ")
  # construct group by list by removing aliases (if any)
  groupByList = paste(columnExpr, collapse=", ")
  # construct sql
  sql = paste0("SELECT ", columnList, " FROM ", tableName,  
           where_clause,
           " GROUP BY ", groupByList)

Tests that check function for correctness are concerned with this part.

Lastly, function either executes SQL in live (non-test) mode or returns SQL as a string to test:

  if (test) {
    return(sql)
  }else {
    return(toaSqlQuery(channel, sql, stringsAsFactors=stringsAsFactors))
  }
}

There is no feasible way to construct tests for the last part within CRAN environment.

How to write test

Each create function tests belong to dedicated test file. Its context is usually just a function name and it contains at least 2 test_that groups of tests: for errors and for correct SQL.

When testing errors all error messages that function produce must be tested with expect_error tests. If function also produce warnings then they should be tested with expect_warning.

Then SQL tests should use helper function expect_equal_normalized to compare SQL produce with SQL expected. Option info parameter is recommended to clarify which test fails.