VR Experiment: Children With ASD Spatial Category Training
Luden.io team is looking for fields in which the best achievements in the video games industry can be useful to people. This is how the Rewire project was born. We wanted to figure out if video games and VR technologies can help in the world of educating people with ASD.
For a year and a half, we have been communicating with educators, psychologists, and therapists from all over the world and often hear very similar stories. For example: to teach a child with ASD some basic (for us, typical people) skills, it is necessary to prepare a special learning environment by choosing materials, objects and toys, printing instructions and hoping that the child will become interested in this, that they will be motivated and will not sweep things around the table. All this, of course, takes a lot of time and effort.
Our previous experiments have proven the fact that virtual reality can be a great motivational tool. But it was still very interesting to us if students were able to transfer the skill from virtual reality to the real world. This is why we teamed up with ABA therapists from ‘Our Sunny World’ rehabilitation center for children with ASD and related disorders, and decided to conduct a little experiment.
Note: At the current stage we wanted to get a measurable summary of our VR experiments and decide where to go next, not to conduct research according to all the rules of the scientific method. These are the results of our experiment. If you want to conduct an experiment of your own, don’t hesitate to contact us at Rewire.Education to get a code and teacher’s guide to VR Lessons
Hypothesis
Virtual Reality (VR) provides a learning environment free of distractions and extraneous sensory stimuli. At the same time, virtual reality works as a motivational tool, especially for children familiar with gadgets: tasks are held in a colorful, cartoony atmosphere, and for completing assignments the student receives a reward — they can draw in the air using a pointer, which is followed by an airplane.
We assumed that such an environment would help children feel more confident in completing tasks similar to those that they are offered in real life. And, as a result, in this environment, they will be able to quickly master the skill and transfer it from a virtual environment to reality.
Environment
We used Oculus Go to carry our experiment out. This model is autonomous (i.e. does not require a PC), but it is equipped with a controller that allows you to interact with a virtual environment, which is not available in other stand-alone VR solutions like Google Cardboard. Besides, this headset is quite durable, it can withstand a blow to the wall (please try to avoid checking if that’s true in practice).
We took the tasks from the guide to assessing the milestones of the development of verbal behavior, the VB-MAPP (in this case, ‘Listener Behavior’ skill, cell 11-d), as a basis for the experiment. The purpose of the exercise is to educate children in such spatial categories as ‘on’ and ‘under’, ‘behind’ and ‘in front of’, ‘inside’ and ‘outside’.
The exercise is available in two modes — ‘sorting’ and ‘guessing’ (commenting). In both cases, one of the 5 available surfaces appears in front of the player (table, chair, toy bridge, house, etc.). In the sorting mode, an object and instructions on where to put it (on the table, under the chair, behind the house, etc.) appear in front of the player. After that, the player must navigate the controller pointer at the highlighted item, press the button, bring the item to the desired surface and press the button again. If the player is mistaken, the item returns to their ‘hand’. After several mistakes are made, a hint is given — the target zone is getting highlighted and an instruction appears on the screen.
In guessing mode, a character located in a virtual space in front of the player lays an object on the desired surface. After that, the player is invited to choose from two options where the item is located (on or under the table, etc.). If the answer is incorrect, a new item appears and the player is invited to try again.
Both modes are flexibly customizable: you can work on prepositions individually (for example, only ‘on’ or only ‘inside’) or in pairs. Pairs of prepositions can be studied both in isolation and in any combination of pairs. In one exercise, the player can complete from 5 to 25 tasks. You can also choose one of two environment options (room or ranch) and one of three companion characters options (girl, boy, or a robot).
At the end of the lesson, statistics of correct and incorrect answers, random button presses, and the total duration of the session are displayed.
Process
Three participants were selected for the experiment — S. (13 years old), E. (7 years old) and D. (17 years old). The experiment was conducted in the form of short sessions 3–5 minutes each as part of regular weekly classes, an average of 2 sessions per lesson. At the beginning of the experiment, all participants’ number of independent answers in the exercise to distinguish between ‘on/under’, ‘behind / in front of’ and ‘inside / outside’ were measured in a virtual reality helmet and in a natural environment — that is, with ordinary toys. The choice of specific exercises, the number, and duration of sessions were left to the discretion of the therapist. In the case of obvious progress in virtual reality (70% or more successful attempts), control testing was conducted in ordinary reality at the end of the experiment.
Natalia Malinova, Head of Applied Behavioral Analysis (ABA) department at ‘Our Sunny World’ center (curator — Yulia Erz, BCBA) tells:
The purpose of the experiment was to understand whether it is easier for a child to learn using virtual reality compared to learning in a natural environment or using pictures, as it is usually happening.
Previously, we have not been working on this topic [prepositions] with this children at all. Maybe, someone has previously tried training it with them, but during testing they did not show any knowledge of prepositions. [As part of the experiment,] we tried to teach them to distinguish between these 3 pairs of prepositions and arrange objects in space according to instructions.
We chose these children because their education level suggested that they were ready to learn prepositions. Unfortunately, the zone of proximal development of most children in our center does not allow teaching prepositions.
A boy names S. had an interesting trait: it was easier for him to complete commenting tasks. The percentage of correct reactions in sorting tasks was lower. According to the data we’ve received, there is no clear trend line to be found, but there is a gradual increase in the percentage of correct reactions. Unfortunately, it’s not uncommon when one day the child does not come in the mood, hasn’t slept well, or feels bad, and some evidence suggests that there was some certain progress, and then — boom! — and a not very successful lesson.
We’ve been working with S. for about 3 years. Overall, he is easily taught, he has a good memory, but exercises with physical objects were not easy for him. The fact that commenting exercises in VR were easier for him to do confirms this theory — it was quite difficult for him to physically interact with objects according to the instructions. But according to the data we have, we see that this skill was well fixed for him when working in virtual reality — he was even able to generalize and transfer this skill to the natural environment. The distinction between ‘on/under’ and ‘inside/outside’ has been well consolidated.
It seems to me VR classes helped him a lot, he grew more motivated by it over time. If it was simply not interesting for him to do these exercises earlier, but in virtual reality, it became much more interesting and more understandable for him, he began to feel more successful and therefore came the improvements when working in the natural environment with almost the same exercises.
When he was able to complete a task, he often said: ‘I did it’! Previously, he did not use such words at all, he did not comment on his activities. Then he began to comment and what is more, he began doing so exactly when he was exercising in virtual reality. I had never heard him comment on his actions in a natural environment. Maybe virtual reality acts as an intermediary for him, it’s kind of a play pretend for him, and therefore, relieves anxiety.
Another boy, E., 7 y.o. Unfortunately, we did not have much data about him since he came to our center only once a week. Our training in VR was going pretty well since he loves various gadgets, it made him a lot more interested in classes. He began to recognize [a Luden.io employee] Georgii, asked where he was, and if he was absent in his office, he said ‘I want a car’ — he really liked the graphomotor skills exercises. For him, it was like getting a reward for completing some tasks.
He got used to this exercise with prepositions, up to the point that when he began to succeed, he began to say ‘I made it! I made it!’ and got excited and happy about that. I think that was cool. And the most interesting thing is that both S. and E. began to make comments on their actions in the process of completing tasks in virtual reality.
According to his mother, E. began to understand the prepositions very well. He managed to perform tasks with them successfully and even comment on where the things are. Mom said that he remembered ‘on/under’ well, but the use of ‘inside / outside’ remained confusing to him. At the end of the experiment, we began teaching ‘behind/in front of’ prepositions. In virtual reality, we reached 70% success rate and about 60% success rate in the natural environment, but we will continue to work on it. We [me and the boy’s mother] believe and see that it turned out to be very cool, interesting and motivational for E. judging on his behavior. It would be great to continue the studies with him in virtual reality.
Third boy, 17 y.o. He’s an observing type: he likes to look out the window, sit still and do nothing, and it was very difficult to interest him at all. Overall, he is very contemplative. He is not a fan of gadgets, so in this case, the results were not as visible as with the other boys. In the beginning, we immediately took prepositions in rotation, assuming that he was already familiar with these prepositions since he was already almost a grown-up and probably someone had already been learning these prepositions with him… But it turned out that we did not have good results in the rotation. When we studied ‘on/under’, in virtual reality we had 50% success rate, 40% with ‘behind/in front of’, and 60% with ‘inside / outside’. Moreover, in a natural environment, he did not show any results with ‘on/under’. ‘Behind / before’ recognition was the same as in VR, and ‘inside / outside’ had only 40% success rate. But in any case, we see that in virtual reality he was more successful.
Perhaps the result was affected by the boy’s special trait since he did not pay attention to the instructions while wearing a headset. We had the impression that he simply did not look closely and poked wherever he could. So far, we have not yet been able to teach him to read what is written before answering.
Control
For comparison we suggest the results of a girl named V. (8 y.o.). In this case, the training has continued for 4 months following similar methods to those used in the experiment.
Conclusions
The data we’ve collected does not allow us to consider our experiment a full-fledged research. Partly due to the small number of subjects and the absence of a simultaneously observed control group, partly since the sessions in virtual reality were included in a strictly limited time frame. Nevertheless, our observations allow us to assume that a combination of factors (various exercises + comfortable environment + immediate reward in the form of a game) contributes to the accelerated learning progress of some children.
Natalia Malinova: Virtual reality allows us to choose materials, environment, objects, and surfaces to work with, which is not always possible to quickly arrange in a room in a natural environment. Since our children are not like typical children, they will not sit quietly and patiently wait for us to carefully prepare the environment. It is much easier to do in virtual reality. Finally, with the help of virtual reality, we motivate them to perform tasks that they previously had difficulties with and did not like. Virtual reality motivates them more because everything is colorful there, everything is beautiful. After each successful session the child gets an opportunity to draw — by the way, they really liked to draw. I believe that this is an additional motivational condition for successful learning.
It is more interesting and easier for the child to do things in virtual reality since it is easier to correct your mistake than in real life. Our goal is to make the child feel successful, to feel capable of completing tasks, and this will motivate them to continue their studies. If they have learned to feel successful in virtual reality, our task is to transfer this success to the natural environment.
Judging by the first and second boys, we see that improving the results in virtual reality has affected the improvement in the natural environment. Of course, we had a small group to run the experiment on and we would like to test it out on a larger one so that some statistical patterns could be seen, but I think we can do that in the future.
Rewire VR Lessons are available for everyone (along with a teacher’s guide) at Rewire.Education website. Sign up and give them a try!
