Keep your eyes on the road kid
The VR Bicycle project (Keep your eyes on the road kid), or my Creative Technology Graduation Project (Bachelor), forever holds a special place in my heart. This was one of those projects that I just had to have from the start, worked really hard on and, in the end, I got the appreciation I deserved for this hard work. This makes the whole experience of working on such a project so much better. And it helps a lot when trying to remain patient when trying to fix a frustrating bug. Although this page will be rather long, compared to those of other projects, I encourage you to read it thoroughly, because this project influenced my development as a Creative Technologist like no other. This project is the reason I got hired for The Boerenkerkhof AR Project and I was also asked to present it to the first year Creative Technology students during the 'Open Days' in September 2019 and March 2020 (But that one was cancelled due to the Corona virus outbreak). I also got nominated for the best GP of my year by my supervisor Robby van Delden. I ended on the third place out of 80-something projects.
Introduction
Choosing a Graduation Project can work in a number of different ways. There is a list of potential projects, proposed by companies or other employers that reach out to the university with an assignment for us. This is what happened in my case. The Roessingh Centrum van Revalidatie asked the university for someone to create a feedback system on the gaze behavior of children in a VR environment. They wanted to analyze exactly what the children are looking at, for how long, etc. This is of course very important information for them, since this project aims to teach children with DCD (Developmental Coordination Disorder) how to ride a bicycle in traffic. DCD is a handicap that is predominantly noticeable at a young age. Boys are more likely to have it than girls are. Children with DCD have a backlog in the development of motor skills and have trouble coordinating their movements. They’re often described as being ‘Clumsy’. As a result, they have more trouble learning basic everyday skills such as riding a bike. I have DCD. And it sucks! Although cycling wasn't too big of a problem for me, I had a lot of trouble learning how to swim. So, I had a lot of motivation to help these children. Add to that the fact that I could work with VR, and you have a project that I'm very passionate about. Unfortunately, so was another student: Maaike Keurhorst. She wanted to do this project as well. Luckily, the Roessingh had enough means to assist both of us. Maaike would work on the feedback on the gaze behavior, and I would work on reducing the amount of 'Motion Sickness' induced by this simulation. More on that later.
Situation
This project aims to teach children with DCD how to ride a bicycle. The Roessingh has been doing this for a long time and is very good at it. However, the therapists deemed the step from practicing in this safe environment to practicing in traffic too large. So, they developed a VR cycling simulation to bridge this gap. This simulation would allow for a realistic yet safe practice environment for the children. See the video below or have a look at My Graduation Project Description for a more in-depth explanation of the simulation and the current situation.
Although this sounds nice, the simulation still had a mayor problem: motion sickness. Motion sickness is a strange, hard to define phenomenon, but is nevertheless extremely common. The best definition that I've found was "A collection of symptoms in which akin to classical motion sickness, without the presence of physical motion". Classical motion sickness being something very common. It's when you feel sick on a long car ride for example, or after a rollercoaster. VR induced motion sickness comes with a wide range of symptoms, but the most common ones are nausea, disorientation, headaches, and dizziness. The symptoms, as well as one's resistance to VR induced motion sickness, are highly personal as well. I was tasked by the Roessingh to reduce this effect as much as possible.
Cause
The exact cause of (VR induced) motion sickness is unknown. Scientists aren't even sure whether regular motion sickness and VR induced motion sickness are caused by the same thing. The lack of understanding of the exact cause makes it extremely difficult to solve this problem at the root. Nevertheless, there are theories about the cause of VR induced motion sickness. I'll quickly describe the most popular, generally-accepted one. For a more in-depth analysis, have a look at the 'Ideation: The cause of VR induced Motion Sickness' section of the report. Sensory Conflict Theory is the most popular one at the moment. This theory states that motion sickness is caused by a mismatch between signals from your body and signals from your eyes. This is best illustrated with an example:
Imagine you're in a car. The car doesn’t seem to move relative to you, so your eyes tell your brain that you're not moving. But the car is moving, and your body feels this, so your body gives your brain a different signal. Sensory Conflict Theory states that this mismatch is both required and sufficient for the occurrence of motion sickness. It also follows that the greater the mismatch, the more severe the experienced motion sickness will be. In VR this would be the opposite way around. Your eyes perceive that you're moving through a virtual environment, but your body doesn’t experience this motion, since you are, in fact, standing still. It is unknown whether this has the same effects as the previously described scenario.
Method
So, how do you solve a problem of which you do not even understand what causes it? Well, here's what I did: I could not address this problem at its root, but since it's a heavily researched topic, a lot of literature and experiments about this topic already exist. I collected a ton of scientific and non-scientific literature on the topic. I focused on experiments that simply measured the effect that changing one specific factor would have on the development of (VR induced) motion sickness. If this change showed a significant reduction in the experienced (VR induced) motion sickness, I could try to implement a similar change to this simulation and hopefully achieve the same effect.
I deducted 13 potential solutions from this literature. 12 of which originated from scientific experiments or were based on grounded scientific theories, and one simply originated from a blog post, but I (considering myself somewhat of an expert on the subject at this point), believed this solution also had potential. However, I did not have the time to implement and experiment all of the solutions. So, I ranked them based on 2 factors. One was an estimation of how much time it'd take to implement a solution, the other was my estimation of their potential to reduce VR induced motion sickness. So, a solution that might work, but takes a lot of money, time or effort to implement would still be discarded.
I chose to implement the 5 highest ranking solutions. I would continue to experiment with 4 of these. One did not need experimenting since we were sure to keep it, regardless of its effect on VR induced motion sickness. This solution was to change the acceleration and deceleration from discrete continues. Before, the speed of the bike was either 0 or 100, and now it could be anywhere in between. This is of course more realistic.
The other 4 solutions were:
- Pointing a fan at the user (this is actually the non-scientific solution)
- Making the field of view (FOV, the angle with which the user can see) smaller
- Adding a point / points of reference
- Adding waypoints
The rationale behind each solution and different potential implementations of said solutions is described in more detail below.
Solutions
Fan
The fan is quite simple. It is placed in front of the user and blows air into their face as they cycle. I programmed it such that the speed of the fan is related to the speed with which the user is cycling, so the faster the user goes, the harder the fan blows. Click on the video to see the fan in action. I apologize for the poor quality, it was recorded last-minute on a phone. Although not scientifically proven, this solution fits in with 3 principles that have been proven.
Firstly, it provides the user with fresh air. Fresh air helps to reduce the symptoms of motion sickness, as well as to delay their occurrence. Secondly, the fan allows the user to maintain a link to the real, physical world. The user can experience a feeling that does not originate from the simulation. This is said to help as well. Thirdly, the fan reduces the sensory mismatch. Especially since the air intensity is related to the cycling speed. The airflow feels more like air resistance. This gives your body, even if just slightly, a feeling of movement. This makes for a smaller sensory mismatch and therefore reduced symptoms.
Field of view
The effect of a smaller field of view is probably the most researched topic relating to VR induced motion sickness. The general idea behind this solution is that a smaller field of view means less visual input (a weaker signal, so to say), and therefore a smaller mismatch with the signals from your body. I've implemented this solutions via different methods: Smaller: The FOV is simply made smaller Blurred: Making the peripheral vision increasingly blurrier Vignette: Making the peripheral vision increasingly darker
Point of reference
The presence of a steady point of reference has also shown to help reduce motion sickness. Some theories even state the lack of a proper point of reference as the sole cause of (VR induced) motion sickness. The 3 implementations I found were: A virtual nose: Adding some shape resembling a nose inside of the VR goggles. A grid: Adding some horizontal and vertical lines. A steady horizon: This one was not implemented or tested since it was already present in the simulation.
Waypoints
Waypoints are dots or items that give you an indication as to where you're supposed to go. They are very common in racing games, and they could apparently reduce VR induced motion sickness. The idea behind this is that they give the user an indication as to where they're supposed to go and, thus, help their brain predict this motion. This solution was eventually removed as the waypoints were too distracting from the actual simulation and thus defeated the purpose.
The SSQ
After implementing these solutions, the details of which are described in the report, I could start to experiment and measure the effect the changes had on the development of VR induced motion sickness. I used the Simulator Sickness Questionnaire (SSQ) to measure the severity of the experienced VR induced motion sickness. The SSQ is somewhat of a golden standard in this field of research. It's very nice to have such a proven, commonly-used tool. Especially in a field of research that struggles with a great number of variables, high personal differences in both resistance to motion sickness and experienced symptoms, ethical concerns and many other challenges. The SSQ lets users rate 16 symptoms that fall into 3 distinct categories: Nausea, Oculomotor & disorientation. I can use the ratings to calcuate a score for each category as well as a total score for VR induced motion sickness. The higher the score, the more severe the experienced VR induced motion sickness is.
Results
I first did a pilot test (N = 7) to further decrease the number of solutions that needed to be tested. I used participants who were very experienced with VR and / or closely related to this project. I used their feedback to limit the number of solutions back to 2: The FOV via vignetting and the fan.
The differences in SSQ scores over 3 trials. The trial order was randomized to counter both the build up of VR induced motion sickness and the learning effect.
With these solutions I conducted a larger experiment (N = 27). The results of this experiment showed that the fan actually significantly reduced the development of VR induced motion sickness. I'm now the first person to ever have shown this effect and I think that's very cool. The vignette actually scored slightly worse than the Null test. This could be coincidence or due to other reasons, but for both the fan as well as the vignette, I advised more experimenting to more accurately determine their exact effect on VR induced motion sickness. The graph on the left (or above) shows the average difference in SSQ scores over 3 trials. The higher the score, the more severe the experienced VR induced motion sickness. It is possible to score below zero, which would mean that a person feels better despite the longer exposure time.
Before I end, I'd like to thank the Roessingh Research & Development (Especially Roos Bulthuis), as well as my supervisor Robby van Delden for their guidance. I'd like to thank TwinSense for their promotional video. And lastly, I'd like to thank all the other people who helped me by giving advice or participating in experiments!