Methodological and assessment considerations in evaluating reinforcement programs in applied settings.
Add a second dependent variable or control probe to every single-case design so sequence effects cannot fake your treatment victory.
01Research in Context
What this study did
McMillan (1973) looked hard at reversal and multiple-baseline designs. The paper asks: do we really know the treatment caused the change?
It lists hidden traps like practice, boredom, or weather that can fake a treatment effect. The author wants extra checks built into every study.
What they found
The review finds many studies stop too soon. They show a graph goes up or down, but skip added measures that prove the change is real.
Sequence effects are a big worry. If A phase always comes first, the improvement might come from learning the routine, not the intervention.
How this fits with other research
One year earlier Christophersen et al. (1972) said you can plug reversal data into ANOVA. McMillan (1973) answers, "Yes, but only if you first guard against sequence effects." The two papers talk to each other: stats are fine, design comes first.
Cariveau et al. (2022) checked 27 journals and saw half of adapted-alternating-treatments studies still skip control pairs. That gap is the exact warning McMillan (1973) gave fifty years ago.
Fahmie et al. (2023) turned the abstract worry into a checklist. Where E said "measure more," Fahmie shows exactly what extra items to track for ecological validity.
Why it matters
Next time you plan an ABAB or multiple-baseline, add a second measure. Track problem behavior plus correct responses, or add a no-treatment probe. One extra row on your data sheet can save your study from a hidden sequence effect and make your graph bullet-proof in supervision or court.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Pick one client, add a short generalization probe after each session, and plot it under the main graph.
02At a glance
03Original abstract
The extensive use of reinforcement programs in applied settings has led to experimentation that often fails to consider potential problems in design. The logic of the within-subject design is reviewed and specific designs employed in reinforcement programs are discussed. For each design (ABAB, or multiple-baseline design across behaviors, individuals, or situations), effects are discussed that make that design less powerful with respect to demonstrating the effect of the experimental variable. Problems in interpreting results of experiments in this area of inquiry are evaluated from the standpoint of internal and external validity. The issue of control groups is presented with considerations as to situations that require their use. Finally, the assessment strategy for evaluating operant programs is discussed and recommendations are made for measurement of behaviors in addition to the target response.
Journal of applied behavior analysis, 1973 · doi:10.1901/jaba.1973.6-517