Inter-Rater Agreement for the Milestones and Barriers Assessments of the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP).
VB-MAPP totals are trustworthy, but always double-check shaky subdomains before you lock an individual goal.
01Research in Context
What this study did
Two board-certified behavior analysts watched the kids with autism take the VB-MAPP. They scored the same videos on their own. The team then ran ICCs to see how close the two sets of scores were.
They checked the total Milestones score, the total Barriers score, and every single subdomain.
What they found
The overall Milestones number was solid (ICC = 0.87). The overall Barriers number was okay (ICC = 0.66).
But when they zoomed in, most subdomains landed in the poor-to-moderate zone. Only Listener and Visual Perceptual held up well.
How this fits with other research
Himuro et al. (2017) also tested an autism-related scale and found high intra-rater numbers. Their good news matches the VB-MAPP total-score story: big totals can look fine even when single parts wobble.
Eussen et al. (2016) gave Movakic to kids with severe ID and saw the same pattern. Construct validity looked strong, yet single items moved around. The pattern repeats: total scores hide weak spots.
Ferron et al. (2017) warn us to mask our eyes when we judge graphs. Their point: small visual cues can fool us. The VB-MAPP data back them up—if you trust one low-reliability subdomain to set a goal, you might chase noise, not skill.
Why it matters
You can still use the VB-MAPP totals for team meetings or billing. Before you write an individual goal, flip to the subdomain ICC. If it is below 0.60, run a quick second assessment or pick a different probe. This two-step check keeps your treatment plan tied to real behavior, not rater drift.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Open last week’s VB-MAPP, circle any subdomain with ICC < 0.60, and re-test that area with a second observer before the next team meeting.
02At a glance
03Original abstract
We determined inter-rater agreement for the VB-MAPP, an instrument sometimes used in planning educational goals and evaluating intervention effects for young people with autism. A pair of raters independently rated each of 32 children diagnosed with autism. Intraclass correlation coefficients for the total Milestones and Barrier scores were 0.876 and 0.629, respectively, indicating good and moderate reliability. There was variability in reliability in the different domains of the Milestones Assessment, with most indicating moderate reliability, and most of the individual Barriers Assessment domains indicating poor reliability. These are the first data relevant to the reliability of the VB-MAPP, they suggest that further evaluation of its reliability is merited and that a high reliability for individual domains should not be assumed.
Journal of autism and developmental disorders, 2019 · doi:10.1007/s10803-019-03879-4