The “Reproducibility Crisis:” Might the Methods Used Frequently in Behavior-Analysis Research Help?
Our within-subject replications can rescue mainstream science from flaky p-value results.
01Research in Context
What this study did
Branch (2019) wrote a think-piece. He asked if behavior-analytic methods can fix the 'reproducibility crisis' in psychology.
The crisis means many classic findings fail when new labs repeat them. Branch says the cause is over-reliance on p-values.
What they found
The paper argues our single-subject designs already solve the problem. We repeat effects within each participant, not across large groups.
We graph data in real time. We replicate before we publish. That makes our work easier to repeat.
How this fits with other research
Critchfield et al. (2023) later counted altmetrics for Behavior Analysis in Practice. Their data show our journals do reach outsiders, backing Branch's claim that we are open and repeatable.
Lerman (2024) extends the idea. She gives a step-by-step plan to teach our methods to teachers and nurses, proving the field is ready to export its tools.
Demello et al. (1992) looked lonely: JEAB mostly cited itself. That seems to clash with Branch's upbeat view, but the papers ask different questions. R et al. tracked who cites us; Branch says our methods are sound when anyone tries them.
Why it matters
Next time you write a report, show your reversal graph and note how many times the effect returned. That small line tells doctors, teachers, or funders the finding is real and can be checked again. You just gave them a built-in replication, no p-value needed.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Add a 'replication panel' to your next graph: mark each time the behavior returns after you withdraw the intervention.
02At a glance
03Original abstract
Mainstream biomedical and behavioral sciences are facing what has been dubbed “the reproducibility crisis.” The crisis is borne out of failures to replicate the results of published research at an average rate of somewhere near 50%. In this paper I make a case that the prime culprit leading to this unsatisfactory state of affairs has been the widespread use of p-values from tests of statistical significance as a criterion for publication. Even though it has been known, and made public, for decades that p-values provide no quantitative information about the likelihood that experimental results are likely to be repeatable, they remain a fundamental criterion for publication. A growing realization among researchers that p-values do not provide information that bears on repeatability may offer an opportunity for wider application of research methods frequently used in the research specialty known as Behavior Analysis, as well as a few other research traditions. These alternative approaches are founded on within- and between-participant replication as integral parts of research designs. The erosion of public confidence in science, which is bolstered by the reproducibility crisis, is a serious threat. Anything that the field of Behavior Analysis can offer as assistance in ameliorating the problem should be welcomed.
Perspectives on Behavior Science, 2019 · doi:10.1007/s40614-018-0158-5