How do you measure what someone perceives if they can’t tell you? It’s a fundamental challenge in autism research, where individuals with severe symptoms – particularly those who are nonverbal – have been largely excluded from studies requiring verbal communication or complex instructions.
The solution came from an unexpected source: animal cognition research. A video game designed without instructions, where players learn through gameplay alone, has successfully measured perception and learning in over 200 adolescents with autism, including those who have significant communication difficulties.
Published in Science Advances, the study identifies specific differences in how autistic individuals integrate sensory information while demonstrating that inclusive research design is possible. We spoke with lead researcher Benjamin Scott about the work, the collaboration with Helen Tager-Flusberg and Joseph McGuire, and what this means for families who’ve been shut out of research.
What made you think a video game could solve the problem of including nonverbal autistic individuals in research, and how did your background in animal cognition influence this approach? How did that translate into designing a game for children?
This came out of a brainstorming session with colleagues at Boston University, specifically Helen Tager-Flusberg. I invited Helen for coffee to discuss her research and to ask her what she thought about using animal models to study how autism risk genes alter brain function. My lab studies brain function in part by training rats and mice in complex cognitive tasks, inspired by a branch of psychology called psychophysics. To get animals to learn these tasks we use reward-based learning and behavioral shaping, tricks our field has picked up over many years to get rodents to learn tasks without verbal instructions, which of course they cannot follow. Helen then suggested we apply our behavioral shaping approach to measure perception and learning in adolescents with ASD. She knew there was an important gap in research in this area, namely that individuals who were more severely affected were being excluded from the kind of precise measurements that psychophysics makes possible. So we built the game around the principles we use for our animal studies, and upgraded it to make it fun and playable online by humans.
Walk us through what happens when a nonverbal autistic individual first encounters Geode. What does it look like in those first minutes? And what does it look like when they ‘get it’?
This is a difficult question to answer. There is a lot of heterogeneity in this population. There is a saying, “if you have met one individual with autism, you have met one individual with autism.” We find that players approach the game in different ways. However, one common theme is that the player is at home in a comfortable environment and, if the player is more severely affected, they are typically with a caregiver, often a parent.
We don’t have the precision to measure the moment of insight, exactly when the player figures out the rule. We rely on the statistics of the game over many trials, so mapping out their thoughts, guesses and beliefs moment to moment cannot be done precisely.
Autistic teens took longer to learn and had reduced accuracy, but they could still play. Is this difference about cognitive ability itself, or is it more about the tools we use to measure it?
The deficits we see in video game play correlate with survey scores provided by parents and caregivers. These survey scores assess real-world abilities including adaptive function and behavioral flexibility. In general we find that individuals with lower adaptive abilities tend to score lower in our game. This suggests that the reduced accuracy we have seen in autistic individuals is not merely about their ability to play video games but relates to important real-world skills.
The term ‘noisy evidence integration’ is technical. Can you explain what’s actually happening when these individuals play?
The short answer is that we don’t know. We know that many brain circuits are altered in autism, but we don’t know which brain changes produce the differences in cognition, learning and behavior. This is an unsolved problem in autism research. Our goal in this study was to help take a step toward answering this question. The purpose of our video game is to make precise, objective and mathematical descriptions of the behavior of players. Now that we have this objective measurement of behavior, we can better track down the neural circuits involved, and this is exactly what we are working on next.
Helen Tager-Flusberg and Joseph McGuire are coauthors on this paper. How did their expertise – hers in autism research, his in decision science – shape what you built? What does this kind of collaboration make possible that you couldn’t do alone?
Helen Tager-Flusberg and Joseph McGuire were essential in getting this study off the ground. My background is in animal cognition; I had never worked with humans before, let alone individuals with neurodevelopmental disorders. Helen’s background in autism was crucial in helping define the concept of the study and in recruiting participants to it. Joe was essential too. His lab helped design the statistical tests and generate the preliminary data which demonstrated that back-translated animal tasks could be used to produce video games. This kind of collaboration is a powerful spark for scientific discovery. It produces new ideas through the cross-pollination of multiple fields and brings together tools rarely found in single labs.
You referred to the participants and their families as “the fourth collaborator.” What did you mean by that, and how did they shape the research?
They shaped our work in three main ways. First, the kids and families provide the data. The parent or caregiver tells us how they think their teen learns, adapts and perceives through surveys. The teens tell us how they see the world by playing the video game. Second, as we developed GEODE, some players gave us feedback, this feedback and the challenges encountered have led to refinements of the game. In fact, during an early phase we had a team of autistic individuals develop their own version of the game and incorporated some of their ideas into our pipeline. Finally, we have presented our work to families through conferences, and the questions and feedback from parents and caregivers have helped us refine our thinking and approach, and to better understand its relevance to their lives.
Autism research is incredibly active right now – maybe more than ever. There are countless studies, some groundbreaking, some controversial. Parents are hopeful but also cautious, because they’ve seen flashy findings that don’t translate into real-world change. Where does Geode stand in that journey? And realistically, what do you see this becoming in the future?
This is important, so thank you for the opportunity to clarify this point. GEODE was not designed as an intervention or a clinical tool. There is no evidence in our study that it can diagnose, cure, treat, or alter the progression of autism. What GEODE is now is a research tool: an instrument to measure aspects of perception and cognition directly from individuals in a way that increases accessibility across the spectrum, including for those with more severe intellectual disabilities or communication deficits.
What excites us is the research that this tool opens up, and it is this future research that could eventually matter for individuals and families. From that perspective, GEODE is at the very beginning of its path to relevance for families affected by autism. With that goal in mind, there are two directions we want to pursue.
First we want to use GEODE to better leverage preclinical animal models. In laboratory animals such as rats and mice, we can produce the same genetic mutations found in humans with a high prevalence of autism. These animals allow us to study the consequences of mutations in autism risk genes for brain function and behavior. The persistent problem with this approach has been knowing which behaviors to study, and whether the changes we see in animals are relevant to what we see in people. GEODE makes that comparison more straightforward: not only was GEODE directly inspired by tasks used in mice and rats, these tasks are also intentionally synchronized (so that the human and animal tasks use similar stimuli, mechanics and training methods). This cross-species approach allows us to compare behavioral changes in humans with those in animals much more directly than if we were to use different behaviors in different species. That should let us get far more out of animal models in discovering how autism risk genes alter brain function and behavior.
The second is the development of more precise outcome measures for clinical trials. As you mentioned, there is a great deal of work in ASD and related neurodevelopmental disorders, and in some cases new treatments are in the pipeline, some in or nearing clinical trials. Objective, precise measures of how drugs improve function are critical to demonstrating that these treatments are effective. In future work we hope to determine whether GEODE could meet the criteria for an outcome measure in clinical trials. Now is the right time to do this, as the tools available to treat neurodevelopmental disorders are expanding, particularly new approaches from gene therapy.
