A Simulation Study on Two Analytical Techniques for Alternating Treatments Designs.
Pair ALIV’s mean-difference number with a randomization p value whenever you analyze a randomized ATD.
01Research in Context
What this study did
Manolov (2019) ran a computer simulation. The goal was to see which numbers best tell you an ATD really worked.
Two tools were compared: ALIV (a mean-difference stat) plus a randomization p value, versus the older VSC checklist plus a binomial test.
Thousands of fake ATD graphs were scored with each method so error rates could be counted.
What they found
The ALIV/randomization pair kept false positives low while still spotting true effects.
VSC with the binomial test missed more real gains and still cried “significant” too often when nothing happened.
Bottom line: add ALIV’s mean difference and a randomization p value whenever you randomize ATD conditions.
How this fits with other research
Smit et al. (2019) published the same year with the opposite advice. Their simulation said the VSC checklist is safe if you have at least five data points per condition. The clash is only skin-deep: Rumen looked at power and Type I error together, while J et al. focused only on keeping false alarms low.
Weaver et al. (2019) backs the randomization half of the package. They showed real ATD papers yield believable p values when a randomization test is used, giving the Rumen combo a real-world footing.
Ferron et al. (2023) extends the same randomization logic to changing-criterion designs, proving the idea travels beyond ATDs.
Why it matters
If you run ATDs, stop relying on eyeballs alone. Plug your raw numbers into free randomization software, tick the ALIV box, and read the p value. You get a single sentence for reports: “Visual overlap is minimal, ALIV = X, randomization p = Y,” which reviewers and funders like. One extra minute gives you a shield against “it’s just visual opinion” critiques.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Download a free randomization-test Excel sheet, enter last week’s ATD data, and report the ALIV value plus p value in your next note.
02At a glance
03Original abstract
Alternating treatments designs (ATDs) are single-case experimental designs entailing the rapid alternation of conditions, and the specific sequence of conditions is usually determined at random. The visual analysis of ATD data entails comparing the data paths formed by connecting the measurements from the same condition. Apart from visual analyses, there are at least two quantitative analytical options also comparing data paths. On option is a visual structured criterion (VSC) regarding the number of comparisons for which one conditions has to be superior to the other to consider that the difference is not only due to random fluctuations. Another option, denoted as ALIV (a comparison involving Actual and Linearly Interpolated Values), computes the mean difference between the data paths and uses a randomization test to obtain a p value. In the current study, these two options are compared, along with a binomial test, in the context of simulated data, representing ATDs with a maximum of two consecutive administrations of the same condition and a randomized block design. Both VSC and ALIV control Type I error rates, although these are closer to the nominal 5% for ALIV. In contrast, the binomial test is excessively liberal. In terms of statistical power, ALIV plus a randomization test is superior to VSC. We recommend that applied researchers complement visual analysis with the quantification of the mean difference, as per ALIV, and with a p value whenever the alternation sequence was determined at random. We have extended an already existing website providing the graphical representation and the numerical results.
Behavior modification, 2019 · doi:10.1177/0145445518777875