LINGUIST 245B / SYMSYS 195L
| Name: | Robert Hawkins |
| Contact: | rdhawkins@stanford.edu |
| Office: | 460-113 |
Please book a time here if you would like to meet!
| Name: | Bran Papineau |
| Contact: | branpap@stanford.edu |
| Office hours: | 1-2pm Monday/Wednesday |
| Name: | Jinyoung Jo |
| Contact: | jinyoungjo@stanford.edu |
| Office hours: | book a meeting here |
Linguistic theories make empirical claims. What data can we draw on to rigorously evaluate these claims? This course takes up one powerful source of empirical evidence, providing a hands-on introduction to the design, implementation, and analysis of psycholinguistic experiments. A particular focus is placed on careful reasoning about our linking hypotheses, the often-implicit assumptions connecting abstract theories to concrete, measurable behavior.
You will conduct a complete replication of a published psycholinguistics experiment. Through this project, students will gain experience with the full experimental workflow: designing and programming interactive web experiments, recruiting participants online via Prolific, analyzing and visualizing data in R, managing projects with git/GitHub, and pre-registering studies on the Open Science Framework.
After taking this course, students should know how to:
- Articulate explicit linking hypotheses between theoretical predictions and empirical response patterns, recognizing how different measures provide distinct windows into mental processes.
- Critically evaluate methodological choices in psycholinguistics research, including dependent measures (response times, self-paced reading, acceptability judgments), sample size justification, measurement validity, and potential confounds or design limitations.
- Conduct a complete replication of a psycholinguistics study, incorporating open science practices including pre-registration, version control, and reproducible analysis pipelines.
- Analyze psycholinguistic data using appropriate statistical models (including mixed-effects regression) and create effective data visualizations.
- Interpret statistical results in light of power, generalizability, and measurement concerns, drawing appropriate theoretical conclusions.
- Communicate experimental findings through multiple modalities—oral presentations, written replication reports, and reproducible code—following standards for transparency and rigor.
- 50% weekly project milestones
- 25% presentations (proposal and project)
- 25% in-class participation and reading responses
- LINGUIST 245A (Introduction to Psycholinguistics).
- Basic familiarity with R. If you have never used R, install R and RStudio and complete the "Introduction to R" on datacamp. We'll use it for data analysis and visualization.
- Basic familiarity with navigating the command line. Complete at least the first two tutorials of this UNIX tutorial if you have never used the command line.
- Basic familiarity with javascript/HTML/CSS. If you have never used these, complete this javascript tutorial and this HTML tutorial. You may also find this CSS tutorial useful.
- We'll use git & GitHub to store collected data, analysis scripts, and basic documentation. Make sure they're installed on your system.
- We'll use the Open Science Framework to pre-register your experiments. Sign up for an account if you don't already have one.
If you feel overwhelmed by any of these except (1), don't worry. It will take work, but by the end of the quarter you'll have acquired all of these tools to at least a basic working level. This website contains lots of useful information about tool setup.
We'll be making use of original papers, blog posts / online articles, as well as the following textbooks:
- Frank, M.C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T., Hawkins, R., Mathur, M., and Williams, R. (2025). Experimentology: An Open Science Approach to Experimental Psychology Methods. MIT Press.
- Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
- Winter, B. (2020). Statistics for Linguists: An Introduction Using R. Routledge.
- Gerstenberg, T. (2025). Statistical Methods for Behavioral and Social Sciences.
Readings & discussion: Brief 1-2 paragraph responses for readings will be collected in advance to facilitate class discussion (due on Canvas by midnight the night before).
Project: Each student will complete a replication of a previously published experiment. Assessment will be based on lightning project proposal presentations (5 minutes), final project presentations (last week of class), and a final replication report. Undergraduates are welcome to work in groups, with instructor permission.
The course will meet Tuesdays and Thursdays from 4:30-5:50pm in Building 20, Rm 21B.
| Week | 🗣️ Tuesday | 💻 Thursday |
|---|---|---|
| 1 | Mar 31 Course planning. What does it mean to test a theory? ✍️ None. |
Apr 2 (on Zoom) Implement a response time study. ✍️ Bott & Noveck (2004) ✍️ Chapters 1, 8, & 9 of Experimentology. |
| 2 | Apr 7 Linking hypotheses. Types of psycholinguistic data. ✍️ Pick one: Tanenhaus (2004), Breheny et al. (2006), or Sun & Breheny (2020) |
Apr 9 Implement a self-paced reading time study. ✍️ Submit working version. |
| 3 | Apr 14 What is open science? What is pre-registration? ✍️ Wilson et al. (2017) ✍️ Chapters 3 and 11 of Experimentology. |
Apr 16 Version control and data management. ✍️ Chapter 13, Appendix B of Experimentology |
| 4 | Apr 21 Lightning presentations. |
Apr 23 Deployment using GitHub Pages + Datapipe. ✍️ Rodd (2024) |
| 5 | Apr 28 Power and sample size justification. ✍️ Button et al. (2013) ✍️ Lakens (2022) ✍️ Chapters 10 and 12 of Experimentology |
Apr 30 Run a power analysis in R. Use Quarto for reproducible reports. ✍️ Submit template with power analysis. |
| 6 | May 5 Exploratory data analysis. How to make good figures. ✍️ Franconeri et al. (2021) ✍️ Chapter 15 of Experimentology |
May 7 Experiment workshop. ✍️ Chapters 2–5 of Gerstenberg (2025) |
| 7 | May 12 Basics of statistical estimation and inference. Linear regression. ✍️ Chapters 5, 6, and 7.1 of Experimentology [optional] Chapters 1, 2, and 5 of Statistical Rethinking [optional] Calin-Jageman & Cumming (2019) |
May 14 Run a linear regression with lm.✍️ Chapters 10–13 of Gerstenberg (2025) |
| 8 | May 19 Handling random effects at the item-level and subject-level. ✍️ Yarkoni (2022) ✍️ Chapters 14 of Winter (2020) [optional] Barr et al. (2013) |
May 21 Run a mixed-effects regression with lmer.✍️ Chapters 17–20 of Gerstenberg (2025) |
| 9 | May 26 Data analysis workshop. ✍️ None. |
May 28 Data analysis workshop. ✍️ None. |
| 10 | Jun 2 Final presentations. |
Jun 4 Final presentations. |
| Week | Action |
|---|---|
| 1 | Pick a paper to replicate, submit your choice via Canvas. |
| 2 | Book a 15 minute meeting to discuss your project with your point person. |
| 3 | Create a GitHub repo for your experiment, initialize jsPsych, add your stimuli. |
| 4 | Submit a link to initial prototype of experiment to your point person for feedback. |
| 5 | Complete methods and analysis pre-registration, incorporate feedback on experiment. |
| 6 | Run Pilot A (in class). Submit a working link to your experiment for Pilot B. |
| 7 | Finalize experiment for full sample. Get instructor permission to run. |
| 8 | Submit updated report with analysis pipeline. Run final sample on Prolific. |
| 9 | Push anonymized data to repo. Update replication report with statistical results. |
| 10 | Push final report to repository. |