README.Rmd

---
title: "R Utility for Pairwise Comparisons of Margins Computed with Functions in Package 'effects'"
author: "Simo Goshev"
date: "`r format(Sys.time(), '%B %d, %Y')`"
output:
  html_document:
    toc: true
    toc_float: true
    collapsed: false
    smooth_scroll: false
    number_sections: true
    theme: default
    highlight: textmate
---

Introduction
======

Package 'effects' in R offers excellent functionality for
computing the type of marginal effects known as average partial
effects. An unfortunate shotfall of the package is that it does not
offer capabilities for computing and testing of pairwise differences
in the margins. The goal of this project is to fill this void.


Installation
======

Download and source *effcomp.R* into R. 


Functionality
======

File *effcomp.R* contains one primary function and several utility
functions. Users would want to use only the primary function.


Primary function
------

The primary function is `effcomp`.

`effcomp` computes pairwise differences in margins and also reports
significance tests of differences in the margins, if requested. Its
syntax is:  

    effcomp(effects_obj, lincon, all = FALSE, tests = FALSE, ...)

and its arguments are:

+ `effects_obj`: the object created by running either `effect` or
  `Effect` from Package 'effects'
  
+ `lincon`: stands for "linear contrasts" and is a matrix of
  pairwise linear contrasts which is provided by the user. This
  argument can be omitted if `all = TRUE`
  
+ `all`: a logical flag which indicates whether all possible
  pairwise contrasts are to be computed. At its default value
  `FALSE`, the user has to provide a matrix of pairwise linear
  contrasts.
  
+ `tests`: a logical flag which indicates whether tests of
  significance of the differences in margins are to be computed. By
  default `tests = FALSE`
  
+ `...`: if `tests = TRUE`, the user may wish pass specify a method
  for adjusting the p-values of the tests for multiple comparisons.
  This is done using the argument `adjmethod` which takes all methods
  described in `p.adjust` of base R as well as `"sidak"` and
  `"scheffe"`. For no adjustment, use `"none"`. The default method is
  `"bonferroni"`.
  
  
Utility functions
-------

Function `testpwcomp` computes the significance tests, `p_adjust`
computes the p-values adjusted for multiple comparisons. 


Examples of usage
=======

We start by removing all objects in memory and loading the effect package.

```{r}
## Clear memory and load package effects
rm(list=ls())
library(effects)
```

Next, we source *effcomp.R*

```{r}
# Source in effcomp.R
source("effcomp.R")
```

Finally, we load an example dataset provided in the _effects_ package and make
some minor changes to the variables

```{r}
## Making minor changes to the data
Prestige$educ <- round(Prestige$education)
Prestige$educ <- ifelse(Prestige$educ <= 12, 12, Prestige$educ)
Prestige$educ <- ifelse(Prestige$educ >= 14, 14, Prestige$educ)
Prestige$educ <- as.factor(Prestige$educ)

Prestige$income <- Prestige$income/1000
```

We are now ready to illustrate the usage of `effcomp`.

Contrasts of margins, one variable 
------

We run our model and compute the margins of variable *type*.

```{r}
fit_lm <- lm(prestige ~ income + type + educ, data = Prestige)
type_eff <- effect("type", fit_lm)       # compute effects
as.data.frame(type_eff)                  # print effects
```

To obtain all possible contrasts of the margins of variable *`type`*, we
execute

```{r}
comps <- effcomp(type_eff, all = TRUE)   # compute all contrasts
summary(comps)
```

If we also want to test whether the margins are different from each
other, we run

```{r}
comps <- effcomp(type_eff, all = TRUE, tests = TRUE)   # Bonferroni adjustment, default
summary(comps)
```

In case we want to use a different adjustment, for example Scheffe's, we
specify the command in the following way:

```{r}
## Scheffe adjustment
comps <- effcomp(type_eff, all = TRUE, tests = TRUE, adjmethod ="scheffe") 
summary(comps)
```

All standard methods for adjustment are available. Sidak's method has
also be added.


`effcomp` also accepts a user provided matrix for constructing desired contrasts.
```{r}
contr_mat <- matrix(c(-1, 1, 0,
                      -1, 0, 1),
                    nrow = 2, byrow = TRUE)

comps <- effcomp(type_eff,contr_mat)   # compute contrasts
summary(comps)
```

To test the differences in the margins (and apply adjustments for
multiple comparisons), we use the following syntax:

+ No adjustment for multiple comparisons

```{r}
## No adjustment for multiple comparisons
comps <- effcomp(type_eff, contr_mat, tests = TRUE, adjmethod = "none") 
summary(comps)
```

+ Bonferroni adjustment for multiple comparisons

```{r}
## Bonferroni adjustment
comps <- effcomp(type_eff, contr_mat, tests = TRUE)                     
summary(comps)
```

+ Scheffe adjustment for multiple comparisons

```{r}
## Scheffe's adjustment
comps <- effcomp(type_eff, contr_mat, tests = TRUE, adjmethod = "scheffe")
summary(comps)
```


Contrasts of margins, two or more variables
-----

`effcomp` can be used to construct contrasts of margins computed
over multiple variables.

Here, we compute margins over *`type`* and *`educ`*:

```{r}
type_educ <- Effect(c("type","educ"), fit_lm)       # compute effects
as.data.frame(type_educ)                            # print effects
```

To obtain the contrasts of these margins, we use
```{r}
comps <- effcomp(type_educ, all = TRUE)   # compute all contrasts
summary(comps)
```

If we need to test for significance of the differences, we run:

+ No adjustment for multiple comparisons

```{r}
## Compute statistical tests, no adjustment for mulitple comparisons
comps <- effcomp(type_educ, all = TRUE, tests = TRUE, adjmethod = "none")
summary(comps)
```
+ Bonferroni adjustment for multiple comparisons

```{r}
## Bonferroni adjustment for multimple comparisons
comps <- effcomp(type_educ, all = TRUE, tests = TRUE)
summary(comps)
```

+ Scheffe adjustment for multiple comparisons

```{r}
## Scheffe's adjustment for multimple comparisons
comps <- effcomp(type_educ, all = TRUE, tests = TRUE, adjmethod = "scheffe")
summary(comps)
```

As before, `effcomp` accepts a matrix of user-specified contrasts.
We first create the matrix of contrasts
```{r}
contr_mat <- matrix(c(-1, 1, 0, 0, 0, 0, 0, 0, 0,
                       0,-1, 0, 0, 0, 0, 0, 1, 0,
                       0, 0,-1, 0, 0, 1, 0, 0, 0),
                    nrow = 3, byrow = TRUE)

```
and then feed it to `effcomp`:

```{r}
comps <- effcomp(type_educ, contr_mat)   # compute all contrasts
summary(comps)
```

Again, to test the differences (and adjust for multiple comparisons
using Scheffe's method) we run

```{r}
comps <- effcomp(type_educ, contr_mat, tests = TRUE, adjmethod = "scheffe")
summary(comps)
```