Fidelity Assessment in Community Programs: An Approach to Validating Simplified Methodology

The team tested a 3-point Likert fidelity checklist in community autism programs. Staff scored each step as done, partly done, or not done. They compared these scores to trial-by-trial coding, the gold standard.

Twenty community sessions were double-scored. The goal was to see if the quick checklist matched the long method.

The simple checklist hit a large share exact agreement with trial-by-trial coding. Inter-rater reliability was perfect. It took a fraction of the time.

Community staff could now monitor fidelity without stopping to code every single response.

Ausenhus et al. (2019) and Zhu et al. (2020) extend this idea. They show remote feedback can also keep fidelity high when you can't be on site.

Paff et al. (2019) validated a classroom tool similar to the checklist. Both papers prove short forms can be trustworthy.

Jessel et al. (2020) seems to disagree at first. They found that shortening an IISCA to 3-5 minutes hurt control. The key difference is what was shortened. Jessel cut assessment time; Suhrheinrich cut scoring time while keeping the full session.

You can swap your long trial-by-trial sheets for the 3-point Likert checklist today. It saves time, keeps reliability, and works in busy community clinics. Use it during supervision or peer reviews to keep programs on track without extra staff hours.