Search and Selection Procedures of Literature Reviews in Behavior Analysis
Half of behavior-analytic reviews hide their search steps, so always check the methods before you trust their takeaways.
01Research in Context
What this study did
King and colleagues read 1,228 behavior-analytic reviews published between 1997 and 2017. They asked one simple question: did the authors tell readers how they found and picked the studies?
They sorted each review as narrative, systematic, or meta-analysis. Then they scored how clear the search and selection steps were.
What they found
More than half of the reviews were plain narrative stories with no listed search plan. Only one in four followed the full, step-by-step rules we now call a systematic review.
When the team looked for basic facts—databases used, key words, inclusion rules—most papers came up empty. Readers could not rerun the search if they tried.
How this fits with other research
Kaiser et al. (2022) shows the upside: when reviewers do list every step, the payoff is big. Their tidy meta-analysis of token economies in K-5 found large, believable gains because the search was transparent and repeatable.
Taylor et al. (2017) and Matson et al. (2009) sit inside King’s 20-year window. Both papers are systematic, so they are the rare exceptions King applauds, not the sloppy majority.
Kemp et al. (2024) extends King’s warning to a new topic. Their 2024 audit of edible-reinforcer studies again finds shaky reporting, proving the problem is alive today.
Why it matters
Before you bank an intervention on a review, flip to the methods page. If you see no database list, no PRISMA flow chart, and no exclusion reasons, treat the conclusions as a friendly blog post, not evidence. Ask the author for the search strings or run your own quick database check. Your clients deserve plans built on see-through science, on hidden hunches.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Open the last review you cited and confirm it lists databases, key words, and inclusion rules; if not, find a better one.
02At a glance
03Original abstract
Literature reviews allow professionals to identify effective interventions and assess developments in research and practice. As in other forms of scientific inquiry, the transparency of literature searches enhances the credibility of findings, particularly in regards to intervention research. The current review evaluated the characteristics of search methods employed in literature reviews appearing in publications concerning behavior analysis (n = 28) from 1997 to 2017. Specific aims included determining the frequency of narrative, systematic, and meta-analytic reviews over time; examining the publication of reviews in specific journals; and evaluating author reports of literature search and selection procedures. Narrative reviews (51.30%; n = 630) represented the majority of the total sample (n = 1,228), followed by systematic (31.51%; n = 387) and meta-analytic (17.18%; n = 211) reviews. In contrast to trends in related fields (e.g., special education), narrative reviews continued to represent a large portion of published reviews each year. The evaluated reviews exhibited multiple strengths; nonetheless, issues involving the reporting and execution of searches may limit the validity and replicability of literature reviews. A discussion of implications for research follows an overview of findings.
Perspectives on Behavior Science, 2020 · doi:10.1007/s40614-020-00265-9