Using gaming technology to help teens on the autism spectrum

People with the autism spectrum often have difficulty interpreting facial signals, in part because they are unable to recognize how the direction of gaze is used to predict the actions and intentions of others. This can have huge implications for understanding behavior, language learning, and social interaction.

A Penn State study is developing a targeted intervention method using gaming technology designed to improve sensitivity to gaze signals, which could treat key symptoms of autism and help improve social skills.

The work appears in JCPP advances, the Journal of Child Psychology and Psychiatry Advances, published under the aegis of the Association for Child and Adolescent Mental Health.

The intervention, Social Games for Adolescents with Autism (SAGA), has been in development since 2013, when Suzy Scherf, associate professor of psychology and Social science research institute Co-funded faculty member and her team received an International Collaborative Online Learning (COIL) grant. “We wanted to develop a computer game intervention to improve the sensitivity to gaze signals for people with the autism spectrum,” she said.

In 2015, the work was featured on the Big Ten Network’s LiveGrand spotlight segment shown during football and basketball games. Jason Griffin, a doctoral student in the Department of Psychology who works in the Developmental Neuroscience Laboratory directed by Scherf, took into account feedback from attendees who played SAGA during filming of the segment to further refine the game.

During the game, participants progress through a narrative storyline and interact with animated characters using a range of non-verbal social cues, including gaze, to guide goal-directed behavior and problem-solve in the game.

“We built a narrative about being a pet sleuth and told the attendees they had to find the lost animal,” Griffin said. “As they progress through the game, they learn to respond to non-verbal social cues from humanoid figures that feature slightly enlarged heads and eyes. “

Scherf said game strategy is a new approach in this type of intervention because skill learning is separate from the game scenario. “Participants can make their own choices in the game narrative, while the technology The game automatically calibrates itself based on their performance to keep them engaged and learning. This means that participants spend hours devoting themselves to the task of gazing, which would be very difficult to do otherwise. “

According to Scherf, the game is the product of an interdisciplinary collaboration between Penn State, a team of undergraduate and graduate visual arts students who worked with the lab programmer to design the graphics for the game.

“Each character’s gaze had to be exactly straight and mimic human movement, which was very delicate. Jason tested the game iteratively and provided feedback, so the team could figure out what we wanted to do as scientists. It was a total collaborative effort that also fell within the resource and time constraints of the project. “

Results of the SAGA study

During the study, 40 adolescents on the autism spectrum were randomized to the standard treatment or care control group. Teens in the treatment group were asked to play SAGA for 30 minutes at home three times a week for 10 weeks.

According to Griffin, after the 10-week period, adolescents in the treatment group developed an increasing sensitivity to human gaze signals, unlike the standard care group. “Participants who experienced a sufficient dose of gameplay showed greater processing-related improvements, such as the ability to perceive and interpret cues from other people’s gaze.”

The improvements to human gaze cues also translated outside of the game. “We showed participants photos and movies of people and noted an increase in gaze cues among our treatment participants,” Scherf said.

However, the most relevant finding from the study was that adolescents in the treatment group demonstrated improvements in their social skills, as documented by their parents. “We found that participants who showed the greatest improvement in understanding human gaze cues in-game had corresponding improvements in their social skills. Parents had no knowledge of their children’s performance in the game, so their observations were completely unbiased, ”Scherf said.

Additionally, the researchers indicated that they were able to retain all participants throughout the study. “Often the procedures take place in a clinic, and it can be difficult to get children to want to participate. The kids found the game very engaging and were excited to play it, ”Griffin said.

Scherf appreciates Griffin’s commitment to the project, which spans four years. “Jason was the only graduate student collecting data and he also coordinated a team of full-time staff and students. Jason also took the lead in coordinating the tests, in which families came every weekend for months. “

In the second phase of the project, researchers will develop improvements for the game to determine whether the skills learned translate into real conversations.

Other researchers on the project include ISSR co-funded faculty members Charles Greier, Professor Frances Keesler Graham Early Career and Associate Professor of Human Development and Family Studies; and Joshua Smyth, distinguished professor of bio-behavioral health and associate director of the SSRI.

The work is supported by a grant from the National Institute of Mental Health.