Assessing the effectiveness of programmed generalization.

Kelly (1973) re-examined a famous generalization study by Walker and Buckley. The author showed that the original paper claimed big maintenance gains. But those gains vanished when you adjusted for where kids started.

The paper is a warning, not new data. It tells analysts to use change scores or ANCOVA before they brag about generalization.

Without baseline adjustment, the program looked like a winner. After adjustment, the same numbers said the program barely worked.

One simple math switch flipped the whole conclusion.

Pigott (1987) extends the same worry. The author adds a scatterplot trick. Plot baseline covariation between treated and untreated behaviors. If the lines already move together, your "generalization" might be pre-existing, not new.

Scheithauer et al. (2020) tests the idea in functional analysis. They show that different baseline sources give the same treatment decision. This sounds like it clashes with Kelly (1973), but it doesn’t. Scheithauer compared baseline sources, not baseline adjustment. Both papers agree: pick your baseline carefully, then analyze it right.

Grauerholz-Fisher et al. (2023) shows another baseline trap. Multiple-opportunity probes inflate baseline skill scores in task analyses. Again, the method you pick changes the story you tell.

Before you claim a skill "generalized," subtract where the client started. Run an ANCOVA or simple change score in Excel. Do it even when the graph looks great. This five-minute step can save you from picking the wrong intervention or over-selling results to parents and funders.