Methodological problems in the use of participant observers.

Four classrooms let teachers act as participant observers. Teachers counted student behavior while still teaching.

Researchers used a multiple-baseline design across rooms. They watched if the counting itself changed behavior.

In one room, both kids and the teacher acted differently once the teacher started recording. Scores shifted up, then drifted back.

The other three rooms showed no clear change. Reactivity can happen, but it does not always strike.

Paff et al. (2019) built the EBP-COM tool so outsiders can score teacher moves without asking the teacher to count. This extends the 1980 warning: if you must have data, use an outside eye.

Grodberg et al. (2012) and Mandell et al. (2016) show quick screens like the AMSE can be both short and solid. Their clean numbers came from trained outsiders, not busy teachers, matching the 1980 call to check who is watching.

Kyonka (2019) adds that even tiny single-case studies need power planning. Pair that with a reactivity check and your small N stays believable.

Before you trust any classroom data, run a quick probe: have the teacher count for two days, then stop. If numbers jump, reactivity is real. Swap to an outside observer, video coder, or tool like EBP-COM. Clean data means better decisions for kids.