Inconsistent visual analyses of intrasubject data.

The authors mailed 22 graphs to 65 behavior analysts. Each graph showed one client’s data across two phases. The reviewers simply answered: “Did behavior change?”

No stats were used. The study wanted to see how often two reviewers agreed by eye alone.

Agreement was only 61 %. Clear jumps in the phase means helped people agree. High or low data points, trend, or big shift size did not help much.

In short, messy graphs made people disagree.

Nelson et al. (1978) warned that high autocorrelation makes visual and statistical answers clash. Annable et al. (1979) now show that even without stats, visual eyes can clash on their own.

Branch (2019) later claimed that behavior analysis can save psychology from the “reproducibility crisis” because we replicate and visually inspect. The 1979 data say: first fix the visual inspection step, or the crisis walks right into our own journals.

Gilroy et al. (2018) offered a new number-crunching tool for discounting curves. Their spirit matches A et al.: when old eyeball methods wobble, tighten the metric or the graph.

Your graphs are your main evidence. Make phase means jump off the page: bold mean lines, clean phase change labels, and stable baselines. Share graphs with a co-worker before you call an intervention a success. One extra set of eyes can catch what yours missed.