Interrater reliability of the Motivation Assessment Scale: failure to replicate with aggressive behavior.
The MAS gives near-zero agreement on why adults with ID are aggressive, so always back it up with direct data.
01Research in Context
What this study did
Two raters filled out the Motivation Assessment Scale for aggressive adults with severe ID. They wanted to see if different staff agreed on the function of each behavior.
The study took place in a state facility. No extra training was given beyond the usual MAS instructions.
What they found
The correlation between raters was almost zero (mean r = .034). Staff rarely picked the same function for the same aggressive act.
The authors call the result a 'failure to replicate' earlier claims that the MAS is reliable.
How this fits with other research
Ferrari et al. (1991) saw the same failure three years earlier. Only 16 of 55 staff pairs agreed on function, so the new data are a direct replication, not an outlier.
Chiviacowsky et al. (2013) compared the MAS side-by-side with the QABF and again found poor item-level agreement. Their newer evidence supersedes the 1994 warning: both checklists are weak alone.
Reid et al. (1999) used the QABF with 417 adults in the same type of setting. They did not report reliability numbers, but their large survey shows the field has already moved toward a different tool.
Embregts (2000) showed that brief reinforcer surveys are only 57 % accurate. The pattern is clear: quick paper checklists without direct observation give shaky data.
Why it matters
If you still keep the MAS in your FBA kit, treat it as a first draft only. Collect ABC data or run a brief experimental analysis before you write the behavior plan. Your treatment will stand on firmer ground and you will avoid costly mistakes with aggressive behavior.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Add one direct observation session or brief alone/attention test before finalizing the next MAS-based FBA.
02At a glance
03Original abstract
The Motivation Assessment Scale (MAS) was used to identify the variables maintaining aggressive behaviors exhibited by 18 adolescents and adults with severe to profound intellectual disability. Each client was rated by two staff members. A variety of measures were calculated to assess interrater reliability. Pearson coefficients across the 18 pairs of raters ranged from -.667 to .722 with an overall correlation of .034. Five of the 12 positive correlations were significant at the .05 level. Correlations across each of the 16 questions of the MAS ranged from -.337 to .425. None of these correlations were significant. Similarly low reliability was obtained when percentage of agreement measures were calculated, although 8 of the 18 pairs of raters (44.44%) did agree on the source of reinforcement maintaining the client's aggressive behavior. These results suggest that for some individuals the MAS may not represent a viable alternative to more formal functional analysis procedures.
Research in developmental disabilities, 1994 · doi:10.1016/0891-4222(94)90020-5