Practitioner Development

An evaluation of the observer effect on treatment integrity in a day treatment center for children.

Howard et al. (2013) · Behavior modification 2013
★ The Verdict

Have supervisors collect integrity data on staff—it can instantly boost their own adherence to protocols.

✓ Read this if BCBAs running day-treatment or clinic teams who want a zero-cost staff fix.
✗ Skip if Practitioners who already use daily fidelity checks with real-time feedback.

01Research in Context

01

What this study did

McGeown et al. (2013) watched supervisors in a day-treatment center.

While the supervisors collected performance-feedback data on staff, the researchers quietly scored how well the supervisors themselves followed the treatment plan.

The design reversed and repeated so the team could see if any change stuck.

02

What they found

The moment supervisors started gathering feedback data, their own treatment integrity jumped up.

When they stopped, it dropped; when they started again, it rose again.

Two supervisors showed the same see-saw pattern, so the effect looked real.

03

How this fits with other research

Staddon (2013) found the same lift when peers, not supervisors, did the watching. Staff who scored a co-worker’s DTT session with a checklist pushed their own fidelity from about 40% to over 80%.

Bhaumik et al. (2008) tested teachers and saw no difference between observer-present and observer-absent phases once feedback was given. Together these studies say the key is feedback, not who is in the room.

Morawska et al. (2007) seems to disagree: mothers did better when they were watched. The clash disappears when you see they watched parents, not staff; parents may like an audience, but supervisors improve because they review data, not because someone stares at them.

04

Why it matters

You can raise your own integrity tomorrow by filling out a quick fidelity sheet on your staff. No extra trainer, no cost, just five minutes of scoring and a glance at the numbers. Try it during one session, post the score, and watch your own accuracy climb.

Free CEUs

Want CEUs on This Topic?

The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.

Join Free →
→ Action — try this Monday

Hand the BCBA a 5-item fidelity sheet for today’s session; have her score the therapist and then chart her own steps—watch both scores rise.

02At a glance

Intervention
other
Design
single case other
Sample size
5
Finding
positive
Magnitude
large

03Original abstract

Treatment integrity is an important concern in treatment centers but is often overlooked. Performance feedback is a well-established approach to improving treatment integrity, but is underused and undervalued. One way to increase its value to treatment centers may be to expose unrealized benefits on the observer who collects the performance feedback data. This "observer effect" could increase the value of performance feedback and promote more consistent evaluation of treatment integrity. The purpose of this investigation was to evaluate the observer effect on treatment integrity. Five supervisors who worked in a day treatment center were asked to collect performance feedback data on staff members' integrity in following a standard treatment protocol that supervisors were also expected to follow. Results showed an immediate and marked improvement in treatment integrity in three supervisors who collected but never received performance feedback. For two supervisors, this effect was reversed and replicated. Implications are discussed.

Behavior modification, 2013 · doi:10.1177/0145445513486801