On mosaics and melting pots: conceptual considerations of comparison and matching strategies.
Comparison groups in autism work must be custom-picked and the reason must be spelled out.
01Research in Context
What this study did
Burack et al. (2004) wrote a think-piece, not an experiment.
They looked at how autism studies pick control or comparison groups.
The authors say every matching choice must fit the exact question you are asking.
What they found
There is no single “right” comparison group for autism research.
Matching kids with autism to typical peers, to other diagnoses, or to siblings can all be wrong or right.
You must tell readers why you chose your match and what limits that choice brings.
How this fits with other research
Elsmore et al. (1994) set the stage. They told clinicians to treat each JEAB study as a sample and each client problem as a comparison—an early call for study-specific matching.
Katz et al. (2003) gives a real example. They compared kids from families with one autistic child to kids from families with many. No behavior difference showed up, proving that a matching choice can lead to a null finding.
Brown et al. (2011) supply the numbers side. They list nine ways to size up single-case data. Burack et al. (2004) warn that even the best effect size is weak if the comparison group makes no sense.
Why it matters
Next time you read an autism study, do not just glance at p values. Ask: “Why these groups?” If you run a study, write a short paragraph that starts with: “We matched X to Y because…” Your supervisor and future readers will thank you.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Open the last autism study on your desktop. Highlight the sentence that explains the comparison group choice. If you cannot find one, add a comment: “Rationale needed.”
02At a glance
03Original abstract
Conceptual and pragmatic issues relevant to the study of persons with autism are addressed within the context of comparison groups and matching strategies. We argue that no choice of comparison group or matching strategy is perfect, but rather needs to be determined by specific research objectives and theoretical questions. Thus, strategies can differ between studies in which the goal is to delineate developmental profiles and those in which the focus is the study of a specific aspect of functioning. We promote the notion of a "mosaic," rather than a "melting pot," approach to science in which researchers communicate conservative and precise interpretations of empirical findings.
Journal of autism and developmental disorders, 2004 · doi:10.1023/b:jadd.0000018076.90715.00