Replicability and randomization test logic in behavior analysis

Jacobs wrote a how-to paper, not an experiment. He looked at how we usually analyze single-case data. He asked: why are we still using t-tests that need big, normal-shaped groups?

He explained randomization tests. These tests count up what actually happened versus what could have happened if the treatment order were shuffled. No need for normal curves or large N.

The paper says randomization tests fit single-case logic better. You get a clean p-value for one learner without pretending the data are normal.

Jacobs showed the math is simple: list every possible order of A and B phases, compute the stat for each order, and see where your real result falls.

Manolov et al. (2022) built a free visual tool that uses the same shuffle logic. You upload your data and get a modified Brinley plot that tells you if the effect replicates across kids. It turns Jacobs’ idea into a one-click website.

Iversen (2025) goes smaller, not bigger. He says each single trial inside a session is already a mini-replication. You graph moment-to-moment responses to see stimulus control appear or vanish. Same goal as Jacobs: stop waiting for big groups and use the data you have.

Solanas et al. (2010) looks like a rival at first. They offer new slope and level estimators instead of randomization tests. But the two methods answer different questions: their tool describes size of change, Jacobs’ tool tests if change is unlikely under chance. You can run both on the same data set.

Next time you run an AB design, swap your t-test for a randomization test. Free online calculators now exist, so you can still get a p-value while staying true to single-case philosophy. Your graph stays small-N, but your inference gets stronger.