Seeing if adding Index entries is not breaking other stuff

001-introduction.Rmd

@@ -7,7 +7,7 @@ library( tidyverse )
 
 # Introduction {#introduction}
 
-Monte Carlo simulations are a tool for studying the behavior of random processes, such as the behavior of a statistical estimation procedure when applied to a sample of data. 
+Monte Carlo\index{Monte Carlo simulation} simulations are a tool for studying the behavior of random processes, such as the behavior of a statistical estimation procedure when applied to a sample of data. 
 Within quantitatively oriented fields, researchers developing new statistical methods or evaluating the use of existing methods nearly always use Monte Carlo simulations as part of their research process. 
 In the context of methodological development, researchers use simulations in a way analogous to how a chef would use their test kitchen to develop new recipes before putting them on the menu, how a manufacturer would use a materials testing laboratory to evaluate the safety and durability of a new consumer good before bringing it to market, or how an astronaut would prepare for a spacewalk by practicing the process in an underwater mock-up.
 Simulation studies provide a clean and controlled environment for testing out data analysis approaches before putting them to use with real empirical data.
@@ -27,7 +27,7 @@ The basic approach for doing so is as follows:
 3. Repeat Steps 1 and 2 many times. 
 4. Summarize the results across these repetitions in order to understand the general trends or patterns in how the method works. 
 
-Simulation is useful because one can control the data-generating process and therefore fully know the truth---something that is almost always uncertain when analyzing real, empirical data. 
+Simulation is useful because one can control the data-generating process\index{data-generating process} and therefore fully know the truth---something that is almost always uncertain when analyzing real, empirical data. 
 Having full control of the data-generating process makes it possible to assess how well a procedure works by comparing the estimates produced by the data analysis procedure against this known truth.
 For instance, we can see if estimates from a statistical procedure are consistently too high or too low (i.e., whether an estimator is systematically biased).
 We can also compare multiple data analysis procedures by assessing the degree of error in each set of results to determine which procedure is generally more accurate when applied to the same collection of artificial datasets.
@@ -110,7 +110,7 @@ During the process of proposing, seeking funding for, and planning an empirical
 Part of such justifications may involve a _power analysis_, or an approximation of the probability that the study will show a statistically significant effect, given assumptions about the magnitude of true effects and other aspects of the data-generating process.
 Researchers may also wish to compare the power of different possible designs in order to inform decisions about how to carry out the proposed study given a set of monetary and temporal constraints.
 
-Many guidelines and tools are available for conducting power analysis for various research designs, including software such as [PowerUp!](https://www.causalevaluation.org/power-analysis.html) [@dong2013PowerUpToolCalculating], [the Generalizer](https://www.thegeneralizer.org/) [@tipton2014stratified],  [G*Power](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower) [@faul2009StatisticalPowerAnalyses], and [PUMP](https://cran.r-project.org/web//packages/PUMP/index.html) [@hunter2023PowerMultiplicityProject].
+Many guidelines and tools are available for conducting power analysis\index{power analysis} for various research designs, including software such as [PowerUp!](https://www.causalevaluation.org/power-analysis.html) [@dong2013PowerUpToolCalculating], [the Generalizer](https://www.thegeneralizer.org/) [@tipton2014stratified],  [G*Power](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower) [@faul2009StatisticalPowerAnalyses], and [PUMP](https://cran.r-project.org/web//packages/PUMP/index.html) [@hunter2023PowerMultiplicityProject].
 These tools use analytic formulas for power, which are often derived using approximations and simplifying assumptions about a planned design. Simulation provides a very general-purpose alternative for power calculations, which can avoid such approximations and simplifications.
 By repeatedly simulating data based on a hypothetical process and then analyzing data following a specific protocol, one can _computationally_ approximate the power to detect an effect of a specified size.
 
@@ -160,7 +160,7 @@ One can partially address questions of generalization by examining a wide range
 Even this strategy has limitations, though.
 Except for very simple processes, we can seldom consider every possible set of conditions. 
 
-As we will see in later chapters, the design of a simulation study typically entails making choices over very large spaces of possibility. 
+As we will see in later chapters, the design of a simulation study\index{simulation study} typically entails making choices over very large spaces of possibility. 
 This flexibility leaves lots of room for discretion and judgement, and even for personal or professional biases [@boulesteix2020Replication].
 Due to this flexibility, simulation findings are held in great skepticism by many.
 The following motto summarizes the skeptic's concern:
@@ -173,7 +173,7 @@ If our goal is to show something that we already believe is correct (e.g., that
 
 [^guest-speakers]: A comment from James: I recall attending seminars in the statistics department during graduate school, where guest speakers usually presented both some theory and some simulation results. A few years into my graduate studies, I realized that the simulation part of the presentation could nearly always be replaced with a single slide that said "we did some simulations and showed that our new method works better than old methods under conditions that we have cleverly selected to be favorable for our approach." I hope that my own work is not as boring or predictable as my memory of these seminars.
 
-Critiques of simulation studies often revolve around the _realism_, _relevance_, or _generality_ of the data generating process.
+Critiques of simulation studies often revolve around the _realism_, _relevance_, or _generality_ of the data generating process\index{data-generating process}.
 Are the simulated data realistic, in the sense that they follow similar patterns to what one would see in real empirical data?
 Are the explored aspects of the simulation relevant to what we would expect to find in practice?
 Was the simulation systematic in exploring a wide variety of scenarios, so that general conclusions are warranted?
@@ -268,12 +268,12 @@ We have also included many case studies throughout the book; these are designed
 
 We divided the book into several parts.
 In Part I (which you are reading now), we lay out our case for learning simulation, introduces some guidance on programming, and presents an initial, very simple simulation to set the stage for later discussion of design principles.
-Part II lays out the core components (generating artificial data, applying analytic procedures, executing the simulations, and analyzing the simulation results) for simulating a single scenario. It then presents some more involved case studies that illustrate the principles of modular, tidy simulation design.
+Part II lays out the core components (generating artificial data, applying analytic procedures, executing the simulations, and analyzing the simulation results) for simulating a single scenario. It then presents some more involved case studies that illustrate the principles of modular, tidy simulation design.\index{tidy design}
 <!-- Need to define modular, tidy simulation design. -->
 Part III moves to multifactor simulations, meaning simulations that look at more than one scenario or context.
 Multifactor simulation is central to the design of more formal simulation studies because it is only by evaluating or comparing estimation procedures across multiple scenarios that we can begin to understand their general properties.
 
 The book closes with two final parts.
 Part IV covers some technical challenges that are commonly encountered when programming simulations, including reproducibility, parallel computing, and error handling.
-Part V covers three extensions to specialized forms of simulation: simulating to calculate power, simulating within a potential outcomes framework for causal inference, and the parametric bootstrap. The specialized applications underscore how simulation can be used to answer a wide variety of questions across a range of contexts, thus connecting back to the broader purposes discussed above.
+Part V covers three extensions to specialized forms of simulation: simulating to calculate power, simulating within a potential outcomes\index{potential outcomes framework} framework for causal inference, and the parametric bootstrap\index{parametric bootstrap}. The specialized applications underscore how simulation can be used to answer a wide variety of questions across a range of contexts, thus connecting back to the broader purposes discussed above.
 The book also includes appendices with some further guidance on writing R code and pointers to further resources.

003-programming-preliminaries.Rmd

@@ -56,10 +56,11 @@ In writing code for simulations, we will make extensive use of this particular f
 We will have more to say about random number generation later.
 
 ### Rolling your own {#rolling-your-own}
+\index{functions!custom written}
 
 In R, you can create your own function by specifying the pieces of input information, the steps to follow in transforming the inputs, and the result to return as output. 
 Learning to write your own functions to carry out calculations is an immensely useful skill that will greatly enhance your ability to accomplish a range of tasks.
-Writing custom functions is also central to our approach to coding Monte Carlo simulations, and so we highlight some of the key considerations here.
+Writing custom functions is also central to our approach to coding Monte Carlo\index{Monte Carlo simulation} simulations, and so we highlight some of the key considerations here.
 [Chapter 19 of R for Data Science (1st edition)](https://r4ds.had.co.nz/functions.html) provides an in-depth discussion of how to write your own functions. 
 
 Here is an example of a custom function called `one_pval()`:
@@ -260,7 +261,7 @@ Later chapters will have much more to say about the process of writing custom fu
 
 ### Function skeletons {#function-skeletons}
 
-In discussing how to write functions for simulations, we will often present _function skeletons_. By a skeleton, we mean code that creates a function with a specific set of input arguments, but where the body is left partially or fully unspecified. 
+In discussing how to write functions for simulations, we will often present _function skeletons_. By a skeleton, we mean code that creates a function with a specific set of input arguments, but where the body is left partially or fully unspecified.\index{skeleton} 
 Here is a cursory example of a function skeleton:
 ```{r}
 run_simulation <- function( N, J, mu, sigma, tau ) {
@@ -322,7 +323,7 @@ Pipes are a very nice technique for writing clear code that is easy for others t
 
 As we will elaborate in subsequent chapters, we follow a modular approach to writing simulations, in which each component of the simulation is represented by its own custom function or its own object in R. 
 This modular approach leads to code that always has the same broad structure and where the process of implementing the simulation follows a set sequence of steps.
-We start by coding a data-generating process, then write one or more data-analysis methods, then determine how to evaluate the performance of the methods, and finally implement an experimental design to examine the performance of the methods across multiple scenarios.
+We start by coding a data-generating process\index{data-generating process}, then write one or more data-analysis methods, then determine how to evaluate the performance of the methods, and finally implement an experimental design to examine the performance of the methods across multiple scenarios.
 Over the next several chapters, we will walk through this process several times.
 
 Although we always follow the same broad process, the case studies that we will present are _not_ intended as a cookbook that must be rigidly followed.
@@ -360,4 +361,4 @@ Adapting a good example is usually much easier than starting from a blank screen
     }
     ```
 
-5. Modify `one_pval()` to allow the user to specify a hypothesized value for the population mean, to use when calculating the one-sample t-test results.
+5. Modify `one_pval()` to allow the user to specify a hypothesized value for the population mean, to use when calculating the one-sample t-test results.

005-initial-t-test-simulation.Rmd

@@ -1,9 +1,10 @@
 
 # An initial simulation {#t-test-simulation}
+\index{example!t-test simulation}
 
 To begin learning about simulation, a good starting place is to examine a small, concrete example.
 This example illustrates how simulation involves replicating the data-generating and data-analysis processes, followed by aggregating the results across replications. 
-Our little example encapsulates the bulk of our approach to Monte Carlo simulation, touching on all the main components involved.
+Our little example encapsulates the bulk of our approach to Monte Carlo\index{Monte Carlo simulation} simulation, touching on all the main components involved.
 In subsequent chapters we will look at each of these components in greater detail.
 But first, let us look at a simulation of a very simple statistical problem. 
 
@@ -100,7 +101,7 @@ We got about 95% coverage, which is good news. In `r table(rps)[[1]]` out of the
 It is important to recognize that this set of simulations results, and our coverage rate of `r 100 * mean( rps == 1 )`%, itself has some uncertainty in it. 
 Because we only repeated the simulation 1000 times, what we really have is a _sample_ of 1000 independent replications, out of an infinite number of possible simulation runs.
 Our coverage of `r round(100*mean(rps==1),digits=1)`% is an _estimate_ of what the true coverage would be, if we ran more and more replications.
-The source of uncertainty of our estimate is called _Monte Carlo simulation error (MCSE)_.
+The source of uncertainty of our estimate is called _Monte Carlo simulation error (MCSE\index{Monte Carlo standard error})_.
 We can actually assess the Monte Carlo simulation error in our simulation results using standard statistical procedures for independent and identically distributed data. 
 Here we use a proportion test to check whether the estimated coverage rate is consistent with a true coverage rate of 95%:
 ```{r}
@@ -238,8 +239,8 @@ Although the general trend is pretty clear, the graph is still a bit messy becau
 So far, we have executed a simple simulation to assess how well a statistical method works in a given circumstance, then expanded the simulation by running a single-factor experiment in which we varied the sample size to see how the method's performance changes.
 In our example, we found that coverage is below what it should be for small sample sizes, but improves for sample sizes in the 100's.
 
-This example captures all the major steps of a simulation study, which we outlined at the start of Chapter \@ref(introduction). 
-We generated some hypothetical data according to a fully-specified data-generating process: we did both a normal distribution and a geometric distribution, each with a mean of 4.
+This example captures all the major steps of a simulation study\index{simulation study}, which we outlined at the start of Chapter \@ref(introduction). 
+We generated some hypothetical data according to a fully-specified data-generating process\index{data-generating process}: we did both a normal distribution and a geometric distribution, each with a mean of 4.
 We applied a defined data-analysis procedure to the simulated data: we used a confidence interval based on the $t$ distribution. 
 We assessed how well the procedure worked across replications of the data-generating and data-analysis processes: in this case we focused on the coverage rate of the confidence interval.
 After creating a function to implement this whole process for a single scenario, we investigated how the performance of the confidence interval changed depending on sample size.