Assessment & Research

On mosaics and melting pots: conceptual considerations of comparison and matching strategies.

Burack et al. (2004) · Journal of autism and developmental disorders 2004
★ The Verdict

Comparison groups in autism work must be custom-picked and the reason must be spelled out.

✓ Read this if BCBAs who write or review autism research.
✗ Skip if RBTs who only run already-written protocols.

01Research in Context

01

What this study did

Burack et al. (2004) wrote a think-piece, not an experiment.

They looked at how autism studies pick control or comparison groups.

The authors say every matching choice must fit the exact question you are asking.

02

What they found

There is no single “right” comparison group for autism research.

Matching kids with autism to typical peers, to other diagnoses, or to siblings can all be wrong or right.

You must tell readers why you chose your match and what limits that choice brings.

03

How this fits with other research

Elsmore et al. (1994) set the stage. They told clinicians to treat each JEAB study as a sample and each client problem as a comparison—an early call for study-specific matching.

Katz et al. (2003) gives a real example. They compared kids from families with one autistic child to kids from families with many. No behavior difference showed up, proving that a matching choice can lead to a null finding.

Brown et al. (2011) supply the numbers side. They list nine ways to size up single-case data. Burack et al. (2004) warn that even the best effect size is weak if the comparison group makes no sense.

04

Why it matters

Next time you read an autism study, do not just glance at p values. Ask: “Why these groups?” If you run a study, write a short paragraph that starts with: “We matched X to Y because…” Your supervisor and future readers will thank you.

Free CEUs

Want CEUs on This Topic?

The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.

Join Free →
→ Action — try this Monday

Open the last autism study on your desktop. Highlight the sentence that explains the comparison group choice. If you cannot find one, add a comment: “Rationale needed.”

02At a glance

Intervention
not applicable
Design
theoretical
Population
autism spectrum disorder
Finding
not reported

03Original abstract

Conceptual and pragmatic issues relevant to the study of persons with autism are addressed within the context of comparison groups and matching strategies. We argue that no choice of comparison group or matching strategy is perfect, but rather needs to be determined by specific research objectives and theoretical questions. Thus, strategies can differ between studies in which the goal is to delineate developmental profiles and those in which the focus is the study of a specific aspect of functioning. We promote the notion of a "mosaic," rather than a "melting pot," approach to science in which researchers communicate conservative and precise interpretations of empirical findings.

Journal of autism and developmental disorders, 2004 · doi:10.1023/b:jadd.0000018076.90715.00