Author: Marc Steiner

Since dealing with blog posts and the master thesis preparation course has been far and inbetween this post is meant more for myself to get me a little further back on track and to collect my thoughts and findings up until now.

Preliminary title: “Effects of virtual reality training environments for large scale emergency operations in comparison to real life training”

Here the exposé I created for class:

Although to me it did feel like I narrowed down my scope quite a bit during the creation of my exposé, I realise that it is still very broad and my topic will need to be defined more clearly until I can send the topic assignment / approval form. I know which general direction I am taking, though I am lacking in understanding of what actually constitutes a masters thesis and how deep I can and should go, and consequently how much I am able to cover.

Next Steps:

1. Search through current literature and find what the latest status on research in my relevant area is and organise my findings in a sorted collection for later use.

I have already set up something like my own personal blog and resource collection space in form of a Notion page, which I plan on using as a guide throughout the process. I hope that I will be able to expand on it and properly organise everything to speed up my workflow. On this page I will collect thoughts, links, images, contacts and whatever else I might need.

2. Engage in conversation with other professionals from my field who deal with the topic or something similar on a daily basis.

I do already have a few connections from previous projects at work in mind, which I will contact in order to find out more about the topic but also gain new perspectives and maybe input on different aspects I can cover in my thesis.

3. Contact research partners and institutions to interview them about the topic.

As an extension of the previous points I will, once I have a better idea of what exactly I will be tackling, interviewing research partners and asking for permission to reference their work during my own.

4. Find possible cooperation partners for projects through step 3.

If the opportunity arises during step 3, I could possibly find a project which I can analyse and work towards during and for my thesis specifically.

5. Define a temporary structure of my work.

Self explanatory, once all the previous steps are done I would like to come back to my Notion and organise everything into a preliminary structure for me to follow.

6. Define a timeline based on all my collected information.

Once that structure is defined I will lay out more concrete plans on how to move forward from that point.

Interesting literature:

Here are some sources I found that I will further investigate for a solid foundation.

https://ieeexplore-1ieee-1org-18gcrmwtl00a2.perm.fh-joanneum.at/xpl/conhome/1000791/all-proceedings

https://ieeexplore-1ieee-1org-18gcrmwtl00a2.perm.fh-joanneum.at/document/10108413

https://www.sciencedirect.com/science/article/pii/S096599781300166X

https://www.sciencedirect.com/science/article/pii/S0379711212000136

https://link.springer.com/article/10.1007/s00530-023-01102-0

As is the case with all new technology, it is easy to lose oneself in the excitement of the most recent developments and fascination of the possibilities when looking at VR/AR. However, it is important to also address parts that might prove to have a negative impact, be it in terms of emotional and physiological responses, environmental concerns, data safety or something else.

So, for this blog post, I decided to look into safety concerns that come with the usage of VR hard- and software and possible dangers connected to it, as a fitting wrap-up to my previous ventures.

Since it is a relatively new sector, augmented reality is rather vulnerable to cyber attacks like spoofing, data manipulation or snooping.¹

VR-headsets are basically variations of a computer, and VR-experiences are essentially software applications, which means that a VR system is just as susceptible to issues of cyber criminality as phones or computers. A VR headset could be just as easily falling victim to a cyber attack as every other computer, possibly resulting in data breaches that can leak personal information, lead to identity theft or cause damage to the hard- and software, amongst other things. One fact that stood out to me in particular was how severely personal information might be endangered when we are talking about virtual reality: VR-systems need to track the user’s movements in order to even work properly. What most people don’t know, however, is the fact that a person’s movement is just as individual as their fingerprints, thus possibly enabling companies to always identify a person based on their movement data and without their consent.²

Due to a person’s unique movement pattern, it is almost impossible to anonymize VR- and AR-tracking data. Scientists already managed to identify users very precisely, which would be a real issue if a VR-system were to be hacked.³

VR-applications collect a lot more data than conventional technologies, as they can listen to all of the users conversation via the live mic, collect biometric data and even record eye-tracking data, thus determining what the user might look at.⁴

So, one of the most imminent dangers of using augmented reality technologies is the fact that the applications and hardware collect a lot of information about who the user is and what they are doing. With this, questions such as „what do AR-companies use the acquired user data for“ or „Does the company share the data with third parties“ arise.³

Generally, when big brands offer certain technologies, the public has a much higher level of trust for these applications or devices without really addressing concerns about them or questioning the product. Over-trusting is a real issue, as it is important for users and designers of VR-products to concern themselves with topics like the previously mentioned dangers of hacking, impact of user age on their experience, or the response of a user to unexpected issues such as hurting oneself while using a VR-appliance.⁵

Sources

1. Thetechrobot. “What are the Security and Privacy Risks of VR and AR” medium.com. Published September 21, 2023. https://medium.com/@thetechrobot609/what-are-the-security-and-privacy-risks-of-vr-and-ar-264896d290f3

2. IEEE. “Virtual Reality Security” IEEE Digital Reality. 2022. https://digitalreality.ieee.org/publications/virtual-reality-security

3. Kaspersky. “VR und AR: Risiken für die Sicherheit und Privatsphäre” Kaspersky. n.d. https://www.kaspersky.de/resource-center/threats/security-and-privacy-risks-of-ar-and-vr

4. Awais, Maham. “What are the potential risks of virtual reality?” Educative.io. n.d. https://www.educative.io/answers/what-are-the-potential-risks-of-virtual-reality

5. Kenwright, Ben. “Virtual Reality: Ethical Challenges and Dangers. Physiological and Social Impacts” IEEE Technology and Society Magazine, vol. 37, no. 4, p. 20-25, Dec. 2018, doi: 10.1109/MTS.2018.2876104.

I have looked at different aspects of VR and its uses, but this post wants to take a closer look at the basis for everything that I have talked about up until now: How do our bodies even respond when being exposed to virtual reality?

First of all, of course VR has its limitations. This is due to the fact that it mostly covers only one sense: seeing. Often times hearing is also involved, but things such as touching, smelling or even tasting are almost always left out, as there is simply no mainstream applicable solution available yet. However, those that have already tried a VR headset know that the immersion is still considerable. But how does it compare?

Experiments on rats have shown that the frequency of electrical spikes between neurons drop by around two thirds when experiencing VR content as opposed to the real world. Activity of cells responsible for navigation also drops to 30%, compared to 80% in real life. Furthermore, 60% of hippocampal neurons, responsible for information retention and learning, are simply turned off during VR sessions. Of course these numbers are not directly applicable to humans, as this was not fully researched yet, but the results show interesting differences nonetheless.¹

Of course there are efforts made to research specific parts of how humans interact with VR. For example, a study about the difference in impact between physical and visual perturbations on users’ balance was conducted. The study participants were tasked to walk along a treadmill, while one group was being physically disturbed in their balance, whereas the other group was visually disturbed. The results showed that the physical group managed to improve their balance by 10% through the training, while the visual group increased it by 40%.²

Another interesting approach was used in a study for the Psychology of Sport & Exercise journal. During that study, participants had to lift weights in form of a dumbell using one arm. One group was looking at their real arm while doing so, and the other group was in a virtual twin of the current room, with a 3D hand holding a 3D weight, all made to mimic the exact same field of vision the non-VR users had. The arm movement was also tracked during exercise, in order to increase immersion. What is interesting is that, doing the exact same exercise, the VR group was found to have a 11% lower rating of perceived exertion, and also a 10-13% lower pain intensity rating than the people doing the “real” exercise. The study lists a mental diversion from the pain stimuli, or a lack of visual signs of exertion on the virtual hand, which the user identified to be their own, as possible reasons for the results.³

These first studies of how our bodies respond to VR, or what VR makes our bodies do, hold very promising connotations as to what it could be used for in the future, as the technology continues to rapidly develop. I will be sure to follow that evolution closely!

Sources

1. Virtual Times. “How does our brain respond to Virtual Reality?” Virtual Times. Published November 29, 2021. https://virtualtimes-h2020.eu/how-does-our-brain-respond-to-virtual-reality/

2. J Crayton Pruitt Family Department of Biomedical Engineering. “Virtual Reality Trains The Mind To Balance The Body” University of Florida. Published September 17, 2018. https://www.eng.ufl.edu/newengineer/news/virtual-reality-trains-the-mind-to-balance-the-body/

3. Matsangidou, Maria, Chee Siang Ang, Alexis R. Mauger, Jittrapol Intarasirisawat, Boris Otkhmezuri, and Marios N. Avraamides. “Is Your Virtual Self as Sensational as Your Real? Virtual Reality: The Effect of Body Consciousness on the Experience of Exercise Sensations.” Psychology of Sport and Exercise. 2019. Elsevier BV. https://doi.org/10.1016/j.psychsport.2018.07.004.

After talking about where and how VR can be used, the logical next step is taking a look at what actual difference the technology makes.

In the past, companies have utlized in-person training as their primary way of educating employees. Traditionally, this method made sense and often times there was simply no alternative to experiencing things in person. However, this approach can often be quite costly, as the employee might need to be relocated to a different location and maybe also accomodated for multiple days during the training. This becomes more of an issue the further said employees need to travel to and from the training site. This is where VR training can help cut down costs, both concerning money and time. Especially industries with complex or very specific training scenarios can benefit a lot from this, as they are proportionally much easier to simulate than meet the reqiurements in real life.¹

Due to this obvious benefit, a large number of professionals are alrady shifting budgets to VR training. But the benefits are not only monetary, as there are many more soft-skill or employee comfort related advantages. For example, a study conducted by assessing the impact of VR on training shows that managers who experienced training through VR were 40% more confident to apply new skills than the other two groups, which had consumed similar content through e-learning or in traditional classrooms. The study also showed that users were 3.75 times more emotionally connected to the content than classroom learners, which resulted in them completing the training four times faster.²

Though some people may certainly prefer traditional training, this aspect of user satisfaction can not be ignored, as most users prefered virtual learning environments. The increase in attention, satisfaction and effectiveness is most likely a result of the entire experience being much more pleasant. Being able to learn at their own workplace, without having to travel hours to training facilities, meetings or workshops and maybe even having to stay there multiple days just sounds like a better alternative to most people. But of course, the two ways of teaching can always co-exist, and compliment each other in areas where they are at their best, respectively.

Sources

1. Facilitate. “Virtual Reality vs. Traditional Training Methods: Which is More Effective?” LinkedIn. Published February 15, 2023. https://www.linkedin.com/pulse/virtual-reality-vs-traditional-training-methods-which

2. Future Visual. “VR vs Traditional Training & When You Should Adopt it” FutureVisual. n.d. https://www.futurevisual.com/blog/vr-training-traditional-training/

In my previous post about use cases of VR I briefly touched on its educational aspect. In this next post I will further elaborate on this specific sector, and what VR can mean for the education of the future.

What has always been a topic and also in this case ends up being the most important factor for VR is immersion. When using VR glasses, anything is done while being fully immersed into the digital environment, and when it comes to learning, that also has a positive impact. Especially since the pandemic, teaching remotely has struggled with retention rates of students. This is where VR appliations can be used to deliver content to students in a much more engaging manner, as visual learning is proven to be more effective for most students, when compared to traditional teaching methods. This was proven in a study at 3M Corporation, where humans were found to process visual information 6000 times faster than plain text.¹

Virtual reality classrooms foster active learning by enabling students to participate directly, enabling them to learn by doing. This approach requires full attention and participation from students, which is presented in a fun way. Through the use of such methods, VR can help students develop critical thinking and technological skills, which are sure to be useful in their later careers leading into an ever-evolving technological future.²

On top of that, VR engages more of the users senses, which again lends itself to be more engaging and less boring: An important factor concering young students. These virtual experiences can transform classrooms or even the students homes into a laboratory or a museum, even going back in time for history classes. Through VR, access to scenarios that used to be cumbersome is easier than ever before, be it just a day trip to a foreign country without leaving the room, or a visit to distant planets in physics class. The result stays the same: VR removes physical limitations.³

Of course, this level of technology and teaching comes with its own set of challenges. The first of these is certainly the cost of acquiring enough VR glasses for an entire classroom, as it can get quite costly very quickly. Another issue is the steep learning curve for teachers, as they need to be well-versed and trained in this area, to be able to properly deliver educational VR content.⁴

Seeing how fast VR has developed in recent years, however, these problems are being tackled and realising the benefits of incorporating VR in classrooms can lead to proper steps being taken. In the same way students used to get excited about being shown a movie in class, in the not so distant future, students can be excited to be taken on an educational, virtual journey to anywhere imaginable.

Sources

1. Spilka, Dmytro. “How VR And AR Are Revolutionizing eLearning For Learners Of All Ages” eLearning Industry. Published May 18, 2023. https://elearningindustry.com/how-vr-and-ar-are-revolutionizing-elearning-for-learners-of-all-ages

2. Intel Corporation. “Active Learning Fosters Technical and Innovative Learning” Intel. n.d. https://www.intel.com/content/www/us/en/education/teaching-strategy/active-learning.html

3. Intel Corporation. “Virtual Reality (VR) in Education” Intel. n.d. https://www.intel.com/content/www/us/en/education/transforming-education/vr-in-education.html

4. Siddiqui, Wahaj. “Virtual Reality (VR) in Education: The Future of Learning” LinkedIn. Published September 12, 2023. https://www.linkedin.com/pulse/virtual-reality-vr-education-future-learning-wahaj-siddiqui

Now that we know what Virtual Reality is, it is time to dive into the broad world of VR and find out in which fields, jobs, situations or settings it can be used. The first thing that comes to mind when thinking of VR is probably the video gaming or entertainment industry, as those were initially responsible for the development of the technology towards public use. While that still holds true today, more and more companies have adopted this way of communicating, researching and testing different ways of utilising VR for promoting, showcasing or bringing their products to life. Through this, VR has, in recent years, made its entrance into the mainstream, revolutionising many sectors with its fresh and extremely involving take on user experience.¹

But what are these use cases, precisely? This post aims to provide some overview and short descriptions of relevant cases.

Health Care

The first and arguably one of the most important uses for VR is health care. Surgeons are using VR during their education to simulate their future work spaces and get to know procedures comfortably in a repeatable manner. This way they can practice all kinds of surgeries without the need for real patients, something that was just not as easily possible before.²

The technology also facilitates medical procedures which might require expertise that can not be provided locally, depending on a patients whereabouts and needs. This is where VR can be used to connect to doctors and specialists around the world for remote consultation and treatment.¹

Lastly, VR is also being used during the treatment of patients. CBT (cognitive behavior therapy) profits immensely from this, as it can provide a controlled and safe environment, which helps patients work through anxiety or phobias. Depending on the needs of the patient, the program can also be easily adjusted and personalised. This approach is also perfectly reproducable, recordable and able to be monitored by medical staff.¹

Education

Similar to how it is being used in health care, VR can help provide virtual environments for almost all other fields aswell. Many jobs work in dangerous environments, which make it harder to train new employees. VR offers a safe solution to this, as it can simulate any environment or task, which provides immersion, realism and most importantly risk free training, which is also cost effective and easy to use. Employees can be taught and stay up to date with a very fast iterative process, which can be used anywhere in the world.²

Architecture

In architecture, VR has two main use cases. It can provide a space for visualising and simulating future projects and buildings, which gives a much better insight into how the object might look and feel, making dimensions much more palpable than on a piece of paper. More advanced setups also offer the opportunity of making real-time changes in the planning phase. The second use case is then to use these existing 3D models and visualisations to provide the customer with a realistic walk-through of their future home without having to physically be there, or before the object is even fully built.³

Online Shopping

In a similar vein to showcasing buildings, VR can also be used much closer to the customer, providing visualisations of products they might want to purchase online, from the comfort of their home. Visualisations might be used to even let the user place certain products in their real life environment to understand the dimension of a new wardrobe they are thinking of purchasing.⁴

Tourism

This virtual try-before-you-buy trend also extends to tourism. VR glasses can show a preview of what the planned holiday might look like in person. Further than that it can also serve as a full alternative through guided virtual tours and visits to distant places for a much cheaper price. This approach also makes it easier to deal with accessibility issues, as it can be done from anywhere, with a much smaller carbon footprint as an added bonus.¹

These were just some of the most common uses of VR in our daily lives. Hopefully this post provided a nice overview, and who knows what else is to come in the next few years of development.

Sources

1. Expert Panel, Forbes Councils Member. “17 VR Applications That Can Provide A Powerful User Experience” Forbes. Published August 22, 2023. https://www.forbes.com/sites/forbestechcouncil/2023/08/22/17-vr-applications-that-can-provide-a-powerful-user-experience/?sh=3379fc503030.

2. Expert Panel, Forbes Councils Member. “13 Productive And Creative Uses For VR That Impress Tech Experts”. Published December 5, 2022. https://www.forbes.com/sites/forbestechcouncil/2022/12/05/13-productive-and-creative-uses-for-vr-that-impress-tech-experts/

3. Alcanja, Daniel. “10 Industries Utilizing Virtual Reality in 2024”. Trio Blog. Published February 8, 2021. https://www.trio.dev/blog/virtual-reality-applications

4. Leonard, Kimberlee. “Top 5 Virtual Reality Business Use Cases” Business.com. Last modified February 21, 2023. https://www.business.com/articles/virtual-reality-business-use-cases/

This post will explain the differences between augmented and virtual reality. Post 7 already explained the basics of VR, which is why this one will start off by explaining augmented reality in detail.

Augmented reality is the process of, as the name suggests, augmenting the users real world view by overlaying relevant data on some form of display. This can be easiest understood by taking a look at the earliest application of augmented reality: Heads-up-displays (HUDs). As most new technology, they were first utilised in a military setting, more specifically in the displays of airplanes and tanks, by projecting information onto the cockpit in order to provide information while still being able to observe their real surroundings. More recently, augmented reality is being used in video game HUDs, where the player’s health, ammunition and other status is being displayed. The army is experimenting with implementing this technology for real world soldiers as well, through the use of personal head-mounted visors.¹

More every uses are for example AR Glasses like the HoloLens or just a smartphone with AR apps and filters. Using these, the real environment is expanded on with mostly 3D objects that are superimposed to appear as they would if they were physically there. This has many uses outside of gaming or the military, as it can help visualise products or furniture during the shopping experience. AR is also used in medical settings, to practice operations or other procedures, as well as in architecture and archeology, to visualise buildings that don’t exist anymore, or buildings that are to be built.²

The main difference between augmented and virtual reality is the immersion factor. Augmented reality keeps the user in the real world, and only adds a few elements to that. This interaction with reality is often the key factor and main reason for choosing AR for a project. This in itself can however also be a drawback, as the added visuals always have to compete with what we see, which can be visually jarring next to one another. On top of that, the AR geometry has to be tracked to our head movement, so that everything stays in the right place relative to the real geometry that it is placed upon, which can also result in some immersion-breaking visuals.

In comparison, virtual reality blocks the users vision of the real world and fully immerses them. This of course does not allow for the same use cases as stated above, but can have other benefits. When using VR, the environment is usually created in 3D and can represent whatever is needed, without having to actually be there. This provides a clear advantage when thinking of large scale operations and training scenarios for users such as the military, police and medical personnel. More on that in a future post.

As this post illustrated, AR and VR are not really directly competing against each other and rather have a different area of use. For future posts towards the topic of my master’s thesis, I will be focussing on VR applications.

Sources

1. Hosch, William L.. “augmented reality.” Encyclopedia Britannica. Last modified September 8, 2023. https://www.britannica.com/technology/augmented-reality.

2. Technikum Wien. “Wie funktioniert Augmented Reality?” Technikum Wien Academy. Accessed January 10, 2024. https://academy.technikum-wien.at/ratgeber/was-ist-augmented-reality/