Establishing equivalence: methodological progress in group-matching design and analysis.
Report effect sizes and variance ratios—not just p-values—to prove your IDD groups are truly matched.
01Research in Context
What this study did
The authors looked at how researchers prove two groups are alike.
They focused on studies with kids who have IDD, autism, or Down syndrome.
Instead of new data, they reviewed old papers and gave new rules for writing up group matches.
What they found
Most papers only say "p > .05" to claim groups match.
That is not enough.
The team says you must also report effect sizes and variance ratios.
These numbers show how close the groups really are.
How this fits with other research
Ganz et al. (2004) first warned against using age-equivalent scores and loose p-values.
Flapper et al. (2013) now adds the next step: give exact effect sizes and variance ratios.
Jarrold et al. (2004) told autism researchers to match on control tasks,Flapper et al. (2013) agrees and says show the numbers for those matches too.
Morrison et al. (2017) compared toddlers with and without delays; the new rules would make that study’s group match clearer to readers.
Why it matters
When you write or read a group study, look past the p-value. Ask for Cohen’s d and the variance ratio. If the authors give them, you can trust the groups are truly alike. If they don’t, the match may be weak.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Add one line to your next report: give the Cohen’s d and variance ratio for every matched variable.
02At a glance
03Original abstract
This methodological review draws attention to the challenges faced by intellectual and developmental disabilities researchers in the appropriate design and analysis of group comparison studies. We provide a brief overview of matching methodologies in the field, emphasizing group-matching designs used in behavioral research on cognition and language in neurodevelopmental disorders, including autism spectrum disorder, Fragile X syndrome, Down syndrome, and Williams syndrome. The limitations of relying on p values to establish group equivalence are discussed in the context of other existing methods: equivalence tests, propensity scores, and regression-based analyses. Our primary recommendation for advancing research on intellectual and developmental disabilities is the use of descriptive indices of adequate group matching: effect sizes (i.e., standardized mean differences) and variance ratios.
American journal on intellectual and developmental disabilities, 2013 · doi:10.1352/1944-7558-118.1.3