The effect of some environmental factors on interobserver agreement.

English et al. (1995) watched how well two observers agreed while scoring the same behavior.

They changed small things in the room: sometimes a peer sat beside the client, sometimes the view was blocked, sometimes the room was noisy.

Then they checked which setups gave the highest agreement scores.

Agreement jumped when a peer was present and when observers could clearly see and hear the client.

Poor sightlines or loud background noise dragged scores down.

Jones et al. (1977) had already warned that raw percent agreement hides errors; the 1995 data prove that room layout is one hidden source of those errors.

Hausman et al. (2022) later re-examined big IOA data sets and found that once observers are well-trained, adding more sessions does not raise agreement. Their finding extends the 1995 message: fix the environment first, then stop worrying about session count.

Wolfe et al. (2023) moved from physical space to graph space. They showed that steep trends and high variability on graphs now become the “environment” that lowers visual agreement, updating the 1995 lesson for the digital age.

Before you collect even one trial, scan the room. Move chairs so both observers see the client’s face, silence the TV, and, when possible, let a peer sit in. These zero-cost tweaks give you cleaner data and fewer reliability headaches later.