Toward an Understanding of Data Collection Integrity

Morris and his team sent a 25-question survey to 232 board-certified behavior analysts.

They asked where data collection usually breaks down and what fixes people try.

Answers came from clinics, schools, and home teams across the United States.

Almost every BCBA can name a time the data sheet did not match what really happened.

Top trouble spots: rushed staff, vague definitions, and no one double-checking the numbers.

Quick wins that helped were daily check-ins, short video reviews, and clear examples of what counts as a behavior.

The warning signs line up with the 1984 paper by J that first listed "therapist drift" as a research killer.

Colombo et al. (2021) also used a BCBA survey and found nearly half of new grads start severe-behavior cases with zero supervision — another place where data can slide.

Vladescu et al. (2022) show one fix: their computer module taught three BCBAs to run a safety task with near-perfect fidelity, proving training can be fast and cheap.

Together the papers say the same thing in different words: without checks, human error wins.

You can audit your own team this week. Pick one client, watch five minutes of session, and compare the therapist’s clicks to what you saw on video. If they don’t match, use the Morris checklist — tighten definitions, add a daily 2-minute review, and have staff trade data sheets for a quick cross-check. Small moves like these saved teams in the survey hours of re-training later.