A Preliminary Assessment of the Validity and Reliability of the Performance Diagnostic Checklist-Human Services
The PDC-HS gives BCBAs a reliable, valid shortcut to pinpoint environmental reasons for staff errors.
01Research in Context
What this study did
Wilder et al. (2019) tested a new checklist called the PDC-HS. It helps supervisors find why staff make mistakes at work.
Three experts watched short work videos. They filled out the checklist for each clip. The team then checked if raters agreed and if scores matched real problems.
What they found
Validity topped 93 percent. Reliability beat 95 percent. These numbers show the tool is both accurate and consistent.
In plain words, different people using the checklist reached the same answers. Their answers also lined up with the true causes of poor performance.
How this fits with other research
Suhrheinrich et al. (2020) also used short videos to check a simpler 3-point fidelity scale. Both studies hit above 99 percent agreement, proving video vignettes can give rock-solid data in busy human-service settings.
Boudreau et al. (2015) tell a caution tale. Their informant checklist for autism showed mixed validity; some scores matched other tests while others did not. Wilder’s much stronger numbers suggest the PDC-HS questions are simply clearer or the raters had better info.
Iosa et al. (2012) likewise validated an informant checklist for psychiatric disorders in adults with ID. Both papers support the wider point: well-written checklists save time without sacrificing accuracy when direct observation is tough.
Why it matters
You now have a quick, trustworthy way to spot why staff slip up. Use the PDC-HS during supervision, find the environmental fix, and skip long guessing meetings. The tool is free, fast, and ready for Monday morning.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Pick one recent staff error, watch a 5-minute clip together, and complete the PDC-HS to choose your first fix.
02At a glance
03Original abstract
The Performance Diagnostic Checklist-Human Services (PDC-HS) is an informant-based tool used to identify the environmental variables contributing to employee performance problems in human service settings. Although previous research has suggested that the tool is useful to identify an intervention which can be used to improve performance, no studies have formally examined the validity and reliability of the tool. In this study, we used video vignettes to assess the validity and reliability of the PDC-HS. We created three videos; each described a performance problem in one or more of the four PDC-HS domains. Twenty-one participants then watched the videos and evaluated the performance problem depicted using the PDC-HS. We measured the validity of the tool by calculating the percentage of participants who correctly identified the PDC-HS domain responsible for the performance problem. Two to four weeks later, we had participants re-watch the videos and complete the PDC-HS to obtain a measure of test re-test reliability. We also compared PDC-HS item responses across participants to evaluate inter-rater reliability. Results show that all participants correctly identified the problematic PDC-HS domain(s) in each video. PDC-HS validity values were above 93% across all three videos and domains; mean test re-test and interrater reliability values were above 95%. The results are discussed in terms of the preliminary nature of this analysis; although these data are limited by the amount and clarity of the information provided in the videos, this study represents a necessary first step in the assessment of the reliability and validity of the PDC-HS tool.
Journal of Organizational Behavior Management, 2019 · doi:10.1080/01608061.2019.1666772