Assessment & Research

Inter-rater reliability of the Pennsylvania independent monitoring for quality program's essential data elements scale.

Goreczny et al. (2005) · Research in developmental disabilities 2005
★ The Verdict

The IM4Q survey is a reliable quality-monitoring tool for adults with ID, so one rater is enough.

✓ Read this if BCBAs who collect program-evaluation or state-report data for adults with intellectual disability.
✗ Skip if Clinicians who only run skill-acquisition sessions and never file quality-of-life reports.

01Research in Context

01

What this study did

The authors tested how well two interviewers agree when they use the IM4Q Essential Data Elements scale.

They checked 74 survey items that ask adults with intellectual disability about life quality, services, and needs.

Different raters listened to the same interviews and scored each item to see if they matched.

02

What they found

Raters agreed on 73 of the 74 items more than 85 percent of the time.

That level of match shows the survey gives steady, trustworthy data no matter who scores it.

03

How this fits with other research

Capio et al. (2013) found the same high agreement between mothers and fathers filling out the Social Responsiveness Scale.

Both studies show one careful rater can give you a solid picture, saving time in busy clinics.

Xu et al. (2022) also worked to prove an IDD tool, but they looked at validity, not reliability.

Together the papers tell us: check both reliability and validity before you trust an assessment.

04

Why it matters

You can use the IM4Q survey with confidence when you need state or funder reports.

One trained staff member can score interviews, so you do not need double staff hours.

Strong reliability means data trends over time are real change, not scorer drift.

Free CEUs

Want CEUs on This Topic?

The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.

Join Free →
→ Action — try this Monday

Train one team member to score this month’s IM4Q interviews and schedule the next quarterly review.

02At a glance

Intervention
not applicable
Design
other
Population
intellectual disability
Finding
strongly positive
Magnitude
large

03Original abstract

The purpose of the independent monitoring for quality (IM4Q) program is to bridge communication between individuals with mental retardation and the service providers on whom they rely. The IM4Q program uses an interview (essential data elements survey) to gather information about the lives of individuals with mental retardation. Collaboration between individuals with mental retardation, family members, and community members provides diverse interviewing teams. This study evaluates inter-rater reliability of the essential data elements (EDE) survey. Findings show that 73 of the 74 questions had monitor agreement scores of 85% or higher Inter-rater reliability of the Pennsylvania independent monitoring for quality program's essential data elements scale.

Research in developmental disabilities, 2005 · doi:10.1016/j.ridd.2004.10.002