Applying signal-detection theory to the study of observer accuracy and bias in behavioral assessment.

Chou et al. (2010) used signal-detection theory to test what changes observer accuracy. Adults without disabilities watched short clips and scored behaviors. The team gave different feedback, rewards, or clear definitions to see who would record correctly and who would drift.

Feedback and reward pushed observers to over- or under-count. Clear definitions helped a little, but money and praise shaped the numbers far more.

Meltzer (1983) showed the same pattern in pigeons: change the grain payoff and the birds’ bias swings. C et al. moved the idea to human staff.

Yaw et al. (2014) took the next step. They gave real residential staff feedback after training. Accuracy jumped, proving the lab finding works on the floor.

DeRoma et al. (2004) adds a twist. Their observers improved their own safety habits while watching others. C et al. focused on data drift; M et al. show watching can also change the watcher’s behavior.

Virues-Ortega et al. (2022) compared paper and app scoring. All tools were fairly accurate, so the tool matters less than the feedback the observer receives.

Your graph is only as good as the person holding the clicker. Schedule random reliability checks. Rotate data collectors. Give brief, neutral feedback right after the session. These small moves guard the numbers that guide your treatment decisions.