Challenges in evaluating psychosocial interventions for Autistic Spectrum Disorders.
The field still lacks the big, uniform RCTs this paper said we need—so vet every new autism program with the 2016 external-validity checklist.
01Research in Context
What this study did
Lord et al. (2005) looked at every psychosocial autism study they could find. They asked one simple question: can we trust the results?
The team read hundreds of papers and wrote a consensus report. They checked how many used random assignment, blind raters, or the same outcome measure.
What they found
Most studies were small, short, and used different yardsticks. Few used random assignment or blind raters.
The authors said the field needs more RCTs and one shared yardstick before we can say what really works.
How this fits with other research
Fernell et al. (2014) took the same plea and aimed it at early-screening studies. They agreed we need rigor, but said shorter trials can still count.
Jonsson et al. (2016) later showed that even when RCTs exist, they rarely tell us if the program will work in real schools. They built a checklist so you can spot missing details.
Stewart et al. (2018) pooled 19 parent-training RCTs and found small but real gains. Their meta-analysis is exactly the kind of evidence Lord et al. (2005) said was missing.
Why it matters
If you pick an autism program today, you still face a patchwork of weak studies. Use the 2016 external-validity checklist before you adopt any new package. Push vendors for RCT data and demand they use the same outcome measure your clinic already tracks.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Run the Ulf et al. (2016) external-validity checklist on your current social-skills program and note any missing pieces.
02At a glance
03Original abstract
In 2002, the National Institutes of Health sponsored a meeting concerning methodological challenges of research in psychosocial interventions in Autism Spectrum Disorders. This paper provides a summary of the presentations and the discussions that occurred during this meeting. Recommendations to federal and private agencies included the need for randomized clinical trials of comprehensive interventions for autism as the highest, but not the sole priority. Ongoing working groups were proposed to address psychosocial interventions with a focus on relevant statistics, standardized documentation and methods of diagnosis, development of outcome measures, establishment of standards in research; and the need for innovative treatment designs, including application of designs from other research areas to the study of interventions in ASD.
Journal of autism and developmental disorders, 2005 · doi:10.1007/s10803-005-0017-6