Inter-rater reliability of the Hayes Ability Screening Index in a sample of Australian prisoners.
The HASI gives the same score no matter who uses it, so you can trust the result after only brief training.
01Research in Context
What this study did
The team tested how well two raters agree when they use the Hayes Ability Screening Index (HASI).
They gave the short IQ screen to prisoners in Australia.
Two staff members scored the same people to see if they would get the same results.
What they found
The raters matched almost perfectly.
The study found excellent inter-rater reliability, with ICCs between 0.90 and 0.97 and Kappa at 0.95.
Even the cutoff score for possible intellectual disability stayed the same between raters.
How this fits with other research
The good news stands out because the Motivation Assessment Scale (MAS) keeps failing. Rojahn et al. (1994) and Ferrari et al. (1991) both found near-zero agreement when staff used the MAS with adults who have severe ID.
The MAS flop is not about the raters — it is about the tool. The MAS asks people to guess why someone is aggressive, a harder call than the HASI’s yes-or-no skill items.
Other screens for adults with ID do pass the reliability test. Lerman et al. (1995) got 95% rater agreement with the Sexual Knowledge Interview Schedule, and Matson et al. (1999) found high reliability with the MESSIER social-skills scale. These studies show a pattern: when the questions are concrete and the training is short, raters agree.
Why it matters
You can add the HASI to your intake packet without long training. One quick briefing is enough for two staff to score the same person and get the same result. That saves time in jails, courts, or residential homes where you need a fast, cheap screen for possible ID.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Print the HASI, run a five-minute practice with two staff, and start double-scoring your next intake.
02At a glance
03Original abstract
BACKGROUND: Reliable ascertainment of intellectual disability (ID) is important to identify those with special needs, in order for those needs to be met in the criminal justice system. Although the Hayes Ability Screening Index (HASI) is valid and widely used for the identification of possible ID, the risk of inter-rater bias between researchers when scoring the HASI has not yet been established. The current paper estimates the inter-rater reliability of the HASI in a sample of Indigenous and non-Indigenous prisoners in Western Australia. METHODS: We estimated intra-class correlation coefficients (ICC) for the consistency of agreement among three blinded raters using a two-way random-effects model assessing the inter-rater agreement of the HASI. Kappa was also estimated for the dichotomous HASI screening threshold outcome between the raters. RESULTS: The HASI exhibited very good within-subject consistency of agreement for Section B (ICC = 0.95; 95%CI:0.94-0.96), Section C (ICC = 0.97; 95%CI: 0.96-0.98) and Section D (ICC = 0.90; 95%CI: 0.87-0.92) subscales and for the total scaled score (ICC = 0.97; 95%CI: 0.96-0.98). The inter-rater reliability of the dichotomous adult ID screening threshold (<85) was also very good (Kappa = 0.95). CONCLUSIONS: The current study provides new evidence that the HASI has a low risk of bias from between-rater scoring and can be reliably scored by both non-clinicians and clinicians with little training, when administered in prison settings. Pre-scoring training should focus on the more subjective 'clock-drawing' section, in order to maximise inter-rater reliability.
Journal of intellectual disability research : JIDR, 2015 · doi:10.1111/jir.12198