The diagnosis conundrum: Comparison of crowdsourced and expert assessments of toddlers with high and low risk of autism spectrum disorder.
Anonymous raters can match expert rankings of social communication in 18-month-old toddlers, offering a rapid remote screening option.
01Research in Context
What this study did
The team asked anonymous online raters to watch 20-second clips of 18-month-old toddlers.
Some toddlers had high autism risk, others had low risk.
Experts also rated the same clips. The goal was to see if the crowd matched the pros.
What they found
Crowdsourced scores lined up tightly with expert scores.
The match means everyday people can spot social-communication differences in short videos.
A quick remote screen is possible without a clinic visit.
How this fits with other research
Sutherland et al. (2025) went further and ran full telehealth diagnoses. They still hit 90 % agreement with in-person results, showing the idea keeps working when real clinicians run the show.
Meimei et al. (2022) pooled 17 telehealth studies and found most screens catch autism 70–100 % of the time. Myers et al. (2018) sits inside that range, giving the review another friendly data point.
Bogenschutz et al. (2015) used interactive TV for remote diagnosis years earlier. Their success set the stage for letting non-experts try the same job through shorter clips.
Why it matters
You can speed up early screening tomorrow. Record a brief play clip, upload it to a secure crowdsourcing platform, and get risk ratings within hours. Use the result to decide who needs a full telehealth or in-person evaluation first. Families skip long wait-lists and you still keep expert time for the kids who need it most.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Film a 20-second play interaction, blur the child’s face if needed, and pilot a crowd-rating request to triage your wait-list.
02At a glance
03Original abstract
As the diagnosis and treatment of Autism Spectrum Disorder (ASD) becomes a greater public health concern, access to professionals and expert assessment becomes increasingly more challenging. Novel techniques to improve efficiency of assessment of young children with social communication concerns are desperately needed to address significant barriers to accessing diagnostic evaluations. Utilizing crowdsourcing technology, we compared ratings by experts and crowdworkers of social communication behaviors in videos of 18-month-old children. Of note, 563 crowdworkers and 24 expert responses were collected in approximately 7 hr and 57 days, respectively. Summed scores of social communication behaviors observed by crowdworkers showed high correlation to those of experts. These data introduce a novel and perhaps efficient way in which to examine of social communication impairments in toddlers. Autism Research 2018, 11: 1629-1634. © 2018 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: This research compared crowdsourced assessments of social communication behaviors to assessments by experts, for toddlers whose behaviors ranged in severity from typical to atypical. Results showed that crowdsourced rankings of social communication behavior significantly correlate with those of experts.
Autism research : official journal of the International Society for Autism Research, 2018 · doi:10.1002/aur.2030