Assessment & Research

Statistical inference in behavior analysis: Discussant's remarks.

Shull (1999) · The Behavior analyst 1999
★ The Verdict

Statistical shortcuts can hide individual change—plot each client first, then decide if group stats add anything.

✓ Read this if BCBAs who write grant reports or publish group analyses.
✗ Skip if RBTs whose role ends after session data are collected.

01Research in Context

01

What this study did

Moerk (1999) is not an experiment. It is a discussant's talk that warns behavior analysts about five traps in statistical inference.

The paper says we often over-trust p-values, ignore effect size, and forget that group averages can hide individual curves.

02

What they found

There is no new data. The piece is a caution flag: if you use t-tests, ANOVAs, or confidence intervals, check that the numbers match what you can see in each single subject's chart.

03

How this fits with other research

Frankot et al. (2024) extends L's worry. They show that big-data tools like mixed models and machine learning can keep each person's curve while still giving group-level power.

Fraley (1998) came one year earlier and gave concrete options—matching-to-sample plus feedback—to study judgment. Moerk (1999) then asked how far we should push the stats on those options.

McGeown et al. (2013) and Furrebøe et al. (2017) promote demand-curve metrics from behavioral economics. Moerk (1999) would count these as statistical inference and remind us to test them one subject at a time.

04

Why it matters

Before you run a t-test on your next treatment study, graph each client's data. If the curves do not all move the same way, skip the group mean and stick with single-case logic.

FREE CEUs

Get CEUs on This Topic — Free

The ABA Clubhouse has 60+ on-demand CEUs including ethics, supervision, and clinical topics like this one. Plus a new live CEU every Wednesday.

60+ on-demand CEUs (ethics, supervision, general)
New live CEU every Wednesday
Community of 500+ BCBAs
100% free to join
Join The ABA Clubhouse — Free →
→ Action — try this Monday

Open yesterday's Excel sheet, split the group column into separate client lines, and eyeball whether everyone moved in the same direction before you pool.

02At a glance

Intervention
not applicable
Design
theoretical
Finding
not reported

03Original abstract

A collection of essays on the roles of inferential statistics in behavior-analytic research prompted consideration of five issues: (a) the acceptance of research that focuses on the behavior of individual organisms; (b) the need to apply methods thoughtfully; (c) the heuristic value of statistical description; (d) the treatment of aberrant data in the search for general principles; and (e) the role of derived measures in the search for invariances.

The Behavior analyst, 1999 · doi:10.1007/BF03391989