Autocorrelation and estimates of treatment effect size for single‐case experimental design data
Drop NAP, IRD, and PND for autocorrelated data—use Tau-U, SMD, or LRR instead.
01Research in Context
What this study did
The team ran computer simulations on single-case data sets. They added different levels of autocorrelation, the tendency for today’s score to resemble yesterday’s.
They checked how six common effect-size numbers reacted. The goal was to see which ones stayed honest when autocorrelation was high.
What they found
NAP, IRD, and PND grew too big as autocorrelation rose. They made weak looks look strong.
Tau-U, SMD, and LRR kept their size steady. They told the truth even when scores clumped together.
How this fits with other research
Mantzoros et al. (2022) used Tau-U in their big review of vocal-stereotypy studies. Their choice lines up with this paper’s advice to drop NAP, IRD, and PND.
Kratochwill et al. (2022) and Slocum et al. (2022) both push for tighter single-case designs. This paper adds another layer: pick the right math after you collect the data.
Petursdottir et al. (2018) gives a checklist for validity threats. Autocorrelation bias is now one more box to tick before you publish.
Why it matters
If you keep using NAP or PND on sticky data, you risk calling a tiny change huge. That can mislead teams, parents, and funders. Switch to Tau-U or SMD today and your effect sizes will mean what they say.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Open your last single-case graph, recompute the effect with Tau-U, and compare it to the old NAP value.
02At a glance
03Original abstract
AbstractWe examined the degree of autocorrelation among single‐case design data with six measures used to estimate treatment effect size. The most commonly used measures of effect size for single‐case data over the last 5 years published in peer‐reviewed journals were selected for comparison. The overall mean degree of autocorrelation was 0.46 (SD = 0.33) across the 304 data paths, which represents a moderate degree of autocorrelation. Overall, it appears that non‐parametric measures of effect size (i.e., percent of non‐overlapping data [PND], non‐overlap of all pairs [NAP], and improvement rate difference [IRD] values) were substantially and significantly more influenced by the degree of autocorrelation. Tau‐U effect size estimate was the non‐parametric exception as it was not significantly influenced by the degree of autocorrelation. Parametric measures of effect sizes such as standardized mean difference (SMD) and log response ratio (LRR) values did not appear to be significantly influenced by the degree of autocorrelation. For SMD, LRR, and Tau‐U values, the correlation between the effect size value and the degree of autocorrelation was minimal. For NAP, IRD, and PND values, the correlation between the effect size value and the degree of autocorrelation was moderate, indicating that these estimates of effect size should be avoided as the degree of autocorrelation between data points increases.
Behavioral Interventions, 2021 · doi:10.1002/bin.1783