Practitioner Development

The Efficacy of Variations of Observation and Data Recording on Trainee Performance

Blackman et al. (2022) · Journal of Organizational Behavior Management 2022
★ The Verdict

Watching plus data sheets rarely trains staff to criterion—add brief, specific feedback every time.

✓ Read this if BCBAs who train new RBTs or direct-care staff in clinics, schools, or homes.
✗ Skip if Practitioners who already run full BST with live feedback for every trainee.

01Research in Context

01

What this study did

Blackman et al. (2022) tested three ways to teach new staff to watch clients and write data. One group watched videos and filled out sheets. One group watched a live co-worker and got peer feedback. One group watched videos and the trainer later told them what they missed.

All trainees had to score 90 % on data sheets for two runs in a row to pass.

02

What they found

Only a few rookies reached 90 % with just watching and scoring. Most stayed stuck near 60 %. Once the trainer added clear verbal feedback, scores jumped to 90 % within two sessions.

The "observer effect"—hoping people improve just because they know they are watched—was too weak alone.

03

How this fits with other research

Yaw et al. (2014) saw the same jump when they paired a short class with quick feedback. Their residential staff doubled data accuracy after the extra comments.

Moss et al. (2009) meta-analysis of 55 studies backs this up. The top training mix is class time plus on-the-job coaching with praise and correction. Blackman’s results are a fresh single-case example of that rule.

Cochrane et al. (2022) used peers to give video feedback in a gym. Lifters fixed form right away. Blackman shows the same peer-plus-video idea works for data sheets, not just deadlifts.

04

Why it matters

Plan for feedback from day one. Have a veteran watch the first three live sessions and give two positives and one fix after each. Stop hoping the newbie will "self-correct"—the data say most won’t.

Free CEUs

Want CEUs on This Topic?

The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.

Join Free →
→ Action — try this Monday

Schedule a five-minute feedback check after each trainee’s first three data sheets—praise two correct entries, correct one error.

02At a glance

Intervention
behavioral skills training
Design
single case other
Population
not specified
Finding
mixed

03Original abstract

The observer effect refers to an increase in a trainee’s performance following an observation in which the trainee records data on the behavior of another individual performing that skill. We examined three variations of observation and data recording to teach guided compliance implementation. Experiment 1 investigated the effects of data recording during video observation. Experiment 2 examined whether recording the performance of a peer and providing feedback improved performance. Experiment 3 required participants to accurately collect data during video demonstrations. Across all experiments, we observed performance improvements for some but not all participants. Experimenter feedback was necessary for most participants to meet criterion levels. Hence, the observer effect was not consistently found. We discuss implications of these findings as they relate to previous research and apply to staff training.

Journal of Organizational Behavior Management, 2022 · doi:10.1080/01608061.2021.1979708