Randomization tests as alternative analysis methods for behavior‐analytic data
Randomization tests give you valid statistical inference for small-n or single-case data without assuming normal distributions.
01Research in Context
What this study did
Craig et al. (2019) wrote a how-to paper. They explain randomization tests for single-case data. No new experiment was run. The goal is to give analysts a tool that needs no normal distribution or large sample.
What they found
The paper shows that randomization tests give valid p-values for small-n and single-case graphs. You still get clear cut yes-or-no answers about intervention effects. The method works even when data bounce around in ways that break regular t-tests.
How this fits with other research
Andersen et al. (2022) already showed trial-based functional analyses save 71% of assessment time. Craig’s tool lets you analyze those brief data right away without waiting for more points.
Hodges et al. (2020) reviewed 14 functional analysis studies. Most used classic visual inspection. Craig offers a numbers-based check that can back up—or challenge—what your eyes see.
Matson et al. (1999) noted that after a functional analysis teams pick reinforcement treatments more often. Adding randomization tests gives you quicker, cleaner evidence that the chosen treatment really works.
Why it matters
You can plug randomization tests into any small-n study you already run. One free online calculator is enough. Next time you graph a reversal or multiple-baseline, let the test tell you if the jump is real. You keep your single-case power and still give funders, parents, or teachers a simple p-value they trust.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Run your last AB graph through a free randomization test calculator and compare the p-value to your visual inspection.
02At a glance
03Original abstract
Randomization statistics offer alternatives to many of the statistical methods commonly used in behavior analysis and the psychological sciences, more generally. These methods are more flexible than conventional parametric and nonparametric statistical techniques in that they make no assumptions about the underlying distribution of outcome variables, are relatively robust when applied to small-n data sets, and are generally applicable to between-groups, within-subjects, mixed, and single-case research designs. In the present article, we first will provide a historical overview of randomization methods. Next, we will discuss the properties of randomization statistics that may make them particularly well suited for analysis of behavior-analytic data. We will introduce readers to the major assumptions that undergird randomization methods, as well as some practical and computational considerations for their application. Finally, we will demonstrate how randomization statistics may be calculated for mixed and single-case research designs. Throughout, we will direct readers toward resources that they may find useful in developing randomization tests for their own data.
Journal of the Experimental Analysis of Behavior, 2019 · doi:10.1002/jeab.500