Deep Neural Network Reveals the World of Autism From a First-Person Perspective.
A deep-learning camera log can spot autism better than expert eyes by reading what the person does not point at.
01Research in Context
What this study did
Ruan et al. (2021) trained a deep-learning model on first-person photos. The photos were taken by autistic and neurotypical people as they moved through daily life.
The model learned to spot tiny differences in what each group aimed their camera at. It then tried to guess who had autism from new photos it had never seen.
What they found
The computer beat expert clinicians. It labeled the photos correctly more than 80 percent of the time.
Autistic photographers aimed away from faces and other eye-catching objects in the center of the shot. The model used this quiet center as its main clue.
How this fits with other research
Wan et al. (2019) got a similar hit rate with a 10-second eye-tracking clip. Both studies show machines can flag autism from where people look.
van der Geest et al. (2002) saw no gaze difference when kids viewed cartoon people. The new study used real-world photos, explaining the clash: cartoons are too simple to trigger the social-scanning gap.
Williams et al. (2002) first mapped messy face scanning in autistic adults. Mindi’s work extends that idea into everyday snapshots, proving the pattern holds outside the lab.
Why it matters
You may soon have an app that screens for autism from a child’s point-of-view photos. Until then, notice what your learner points a tablet or phone at. Fewer faces or objects dead-center can guide you to embed more social targets during natural environment teaching.
Get CEUs on This Topic — Free
The ABA Clubhouse has 60+ on-demand CEUs including ethics, supervision, and clinical topics like this one. Plus a new live CEU every Wednesday.
During play, note if the learner aims the camera or eye gaze away from faces; use that moment to prompt a social orient.
02At a glance
03Original abstract
People with autism spectrum disorder (ASD) show atypical attention to social stimuli and aberrant gaze when viewing images of the physical world. However, it is unknown how they perceive the world from a first-person perspective. In this study, we used machine learning to classify photos taken in three different categories (people, indoors, and outdoors) as either having been taken by individuals with ASD or by peers without ASD. Our classifier effectively discriminated photos from all three categories, but was particularly successful at classifying photos of people with >80% accuracy. Importantly, visualization of our model revealed critical features that led to successful discrimination and showed that our model adopted a strategy similar to that of ASD experts. Furthermore, for the first time we showed that photos taken by individuals with ASD contained less salient objects, especially in the central visual field. Notably, our model outperformed classification of these photos by ASD experts. Together, we demonstrate an effective and novel method that is capable of discerning photos taken by individuals with ASD and revealing aberrant visual attention in ASD from a unique first-person perspective. Our method may in turn provide an objective measure for evaluations of individuals with ASD. LAY SUMMARY: People with autism spectrum disorder (ASD) demonstrate atypical visual attention to social stimuli. However, it remains largely unclear how they perceive the world from a first-person perspective. In this study, we employed a deep learning approach to analyze a unique dataset of photos taken by people with and without ASD. Our computer modeling was not only able to discern which photos were taken by individuals with ASD, outperforming ASD experts, but importantly, it revealed critical features that led to successful discrimination, revealing aspects of atypical visual attention in ASD from their first-person perspective.
Autism research : official journal of the International Society for Autism Research, 2021 · doi:10.1002/aur.2376