A study of the effects of frequency of probe data collection and graph characteristics on teachers' visual analysis.

Malagodi et al. (1989) asked teachers to look at graphs. The graphs showed student progress. Some graphs had many data points. Others had few. The team wanted to know if the number of points changed how teachers read the graphs.

They also tested different graph styles. Some lines were steep. Some were flat. Some bounced around. Teachers judged each graph by eye.

When the line clearly went up, teachers agreed. More data points did not help. When the line was flat, falling, or jumpy, teachers disagreed. More data points helped them agree better.

So, extra points matter most when the trend is messy.

Dykens et al. (1991) ran a near-copy study two years later. They added one twist: some teachers saw numbers only, no graph. Result: graph or no graph, the same three things swayed teachers—trend, variability, and how often you record data. The 1991 paper builds on the 1989 finding instead of fighting it.

Wolfe et al. (2023) zoomed in on agreement. They showed that steep trend and big effect size hurt inter-rater agreement the most. This extends the 1989 warning: messy trends need extra care.

Kahng et al. (2010) looks like a contradiction at first. Their experts agreed strongly when judging graphs. But their judges were trained BCBAs, not regular teachers. The 1989 study used teachers with little graph training. The gap shows training, not the graph, drives agreement.

When you eyeball a graph, add more probe days if the data are flat or jumpy. Extra points cut teacher guesswork. If you train staff, show them how trend and variability fool the eye. Pair visual checks with simple rules or brief computer training like O’Grady et al. (2018) to keep teams consistent.