Visual inspection of data revisited: Do the eyes still have it?
Eyes catch level jumps but miss trends—always back up visual inspection with a simple trend rule.
01Research in Context
What this study did
The team ran four computer games with graphs. Each game showed pretend ABA data. Trained BCBAs looked at the graphs. They had to say if a change happened. The study checked what the eyes caught and what they missed.
What they found
People spotted sudden level jumps. They almost always missed slow trends. Extra cues, like colored lines, did not help. The eyes are good at big jumps, bad at gentle slopes.
How this fits with other research
Hartmann et al. (1982) already warned that p-values for agreement lie when data points link together. Fisch (1998) adds a second warning: eyes lie too, just in a different way.
Parsons et al. (1981) showed that observers cheat their own numbers when they do the math. The new paper shows the cheat can happen earlier—at the glance stage—before any math.
Schaaf et al. (2015) taught staff to build graphs fast with video models. Their work pairs with Fisch (1998): build the graph right, then inspect it with care, not with eyeball faith alone.
Why it matters
You graph data every day. Trust your eyes for big jumps, but never for slow trends. Add a trend line or use a split-middle test before you call a victory. Teach the same rule to new staff: see the graph, then check it with a tool.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Draw a split-middle line on today’s graph before you decide the intervention worked.
02At a glance
03Original abstract
In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition. However, when serial dependence is present in the data, the use of visual inspection by itself may prove to be problematic. Previously published reports demonstrate that autocorrelated data influence trained observers' ability to identify level treatment effects and trends that occur in the intervention phase of experiments. In this report, four recent studies are presented in which autoregressive equations were used to produce point-to-point functions to simulate experimental data. In each study, various parameters were manipulated to assess trained observers' responses to changes in point-to-point functions from the baseline condition to intervention. Level shifts over baseline behavior (treatment effect), as well as no change from baseline (no treatment effect or trend), were most readily identified by observers, but trends were rarely recognized. Furthermore, other factors previously thought to augment and improve observers' responses had no impact. Results are discussed in terms of the use of visual inspection and the training of behavior analysts.
The Behavior analyst, 1998 · doi:10.1007/BF03392786