Effects of data sampling on graphical depictions of learning.

Carey et al. (2014) built fake learning graphs. They removed every second trial or every third session. Then they asked experts to judge when a skill was mastered.

The team timed how long the shortened records saved. They wanted to see if less data still gave the same visual answer.

Short samples lied. First-three-trial slices often made mastery look slower. Every-third-session slices often made mastery look faster.

The time saved was tiny—only minutes. The wrong decisions were big.

Cook et al. (2020) later showed the same risk with momentary time sampling. They told researchers to spot-check full duration now and then so drift does not hide.

Suhrheinrich et al. (2020) seems to disagree. They found a 3-point fidelity checklist matched trial-by-trial coding and saved hours. The difference is purpose: Mary-Katherine tested learning graphs for mastery timing; Suhrheinrich tested fidelity scores for staff feedback. Different jobs, different rules.

Jessel et al. (2020) echoed the warning. Truncating a 10-min IISCA to 3-5 min weakened experimental control. Less data, weaker truth.

If you graph progress for insurance or parents, sample every trial. A skipped point can hide a dip or spike and push you to move on too soon—or stay too long. The few minutes you save are not worth the wrong call.