Assessment & Research

Statistical inference in behavior analysis: Discussant's remarks.

Shull (1999) · The Behavior analyst 1999
★ The Verdict

Statistical shortcuts can hide individual change—plot each client first, then decide if group stats add anything.

✓ Read this if BCBAs who write grant reports or publish group analyses.
✗ Skip if RBTs whose role ends after session data are collected.

01Research in Context

01

What this study did

Moerk (1999) is not an experiment. It is a discussant's talk that warns behavior analysts about five traps in statistical inference.

The paper says we often over-trust p-values, ignore effect size, and forget that group averages can hide individual curves.

02

What they found

There is no new data. The piece is a caution flag: if you use t-tests, ANOVAs, or confidence intervals, check that the numbers match what you can see in each single subject's chart.

03

How this fits with other research

Frankot et al. (2024) extends L's worry. They show that big-data tools like mixed models and machine learning can keep each person's curve while still giving group-level power.

Fraley (1998) came one year earlier and gave concrete options—matching-to-sample plus feedback—to study judgment. Moerk (1999) then asked how far we should push the stats on those options.

McGeown et al. (2013) and Furrebøe et al. (2017) promote demand-curve metrics from behavioral economics. Moerk (1999) would count these as statistical inference and remind us to test them one subject at a time.

04

Why it matters

Before you run a t-test on your next treatment study, graph each client's data. If the curves do not all move the same way, skip the group mean and stick with single-case logic.

Free CEUs

Want CEUs on This Topic?

The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.

Join Free →
→ Action — try this Monday

Open yesterday's Excel sheet, split the group column into separate client lines, and eyeball whether everyone moved in the same direction before you pool.

02At a glance

Intervention
not applicable
Design
theoretical
Finding
not reported

03Original abstract

A collection of essays on the roles of inferential statistics in behavior-analytic research prompted consideration of five issues: (a) the acceptance of research that focuses on the behavior of individual organisms; (b) the need to apply methods thoughtfully; (c) the heuristic value of statistical description; (d) the treatment of aberrant data in the search for general principles; and (e) the role of derived measures in the search for invariances.

The Behavior analyst, 1999 · doi:10.1007/BF03391989