Category Archives: Definition

Describes how virtual and augmented reality works and what it means.

Application Areas of Virtual Reality

With this article the different application areas of virtual reality are introduced. Other than augmented reality virtual reality is not extending the physical environment with virtual elements. The aim is to create a completely independent virtual reality a user is able to interact with. The sensory experience can include sight, touch, hearing and even smell. [1]

Application Areas

The most popular fields in which virtual reality is present are:

  • Entertainment
  • Urban Design and Architecture
  • Training and Education
  • Medicine

The following chapters cover example applications of virtual reality to give a high level overview.

Entertainment

The first product which really caught the attention of the public eye was definitely the crowdfunding campaign on Kickstarter for the Oculus Rift in 2012. [2]

In the meantime there have been many different products introduced to the market. In Tokyo, April 15th 2016 the first virtual reality exception opened for a limited time. It allows anyone to experience the fascination of virtual reality.  [3]

Urban Design and Architecture

Virtual reality can be used to plan large scale urban project as well as individual buildings. The core element is a three dimensional digital model of the planned development. This is especially useful for long term project and has been used for the development of the campus of the University of Cambridge. [4]

3D Model University Campus
3D Model University Campus [4]
Training and Education

The US Army is using virtual reality setups to train combat situations in an enclosed environment. This minimizes potential risks and allows to reset and simulate specific situations in a short amount of time. This minimizes costs and increases efficency.[4]

US Army – Virtual Reality Training [5]
Medicine

Virtual Reality is used to help individuals with a stroke or neurological disorder to recover. A patient is able to manipulate objects in a virtual environment which can be structured to provide audio-visual feedback. This promotes motor learning to improve the rehabilitation process. Specific living tasks are being practiced within the virtual environment, e.g. making a cup of coffee in a kitchen. Real-time motion capture helps the therapist to document change with the support of a software. [6]

Virtual Kitchen Setup [6]
US Army – Virtual Reality Training [5]

[1] https://en.wikipedia.org/wiki/Virtual_reality

[2] http://video.golem.de/games/8816/oculus-rift-virtual-reality-brille-%28kickstarter%29.html

[3] http://gematsu.com/2016/03/bandai-namco-opening-vr-zone-tokyo-april-15

[4] S. Roudavski, “VIRTUAL ENVIRONMENTS AS SITUATED TECHNO- SOCIAL PERFORMANCES Virtual West Cambridge case-study,” 2010.

[5] http://www.army.mil/article/84453/

[6] D. White, K. Burdick, G. Fulk, J. Searleman, and J. Carroll, “A virtual reality application for stroke patient rehabilitation,” IEEE Int. Conf. Mechatronics Autom. 2005, vol. 2, no. July, pp. 1081–1086, 2005.

 

 

Application Areas of Augmented Reality

Augmented reality can be applied in numerous areas. While the first AR systems were developed for military, industrial and medical fields, they also appear in commercial and entertainment. [3] suggest to classify the possible areas of application of AR as follows:

  • Personal information systems
  • Industrial and military applications
  • Medical applications
  • AR for entertainment
  • AR for the office
  • Education and training

Personal information systems

Personal wearable computing is one of the biggest potential markets for AR [3]. Firstly, they can be used for personal assistance and advertisement. Devices to automatically store inputs from the real world like conversations, people or just any information which might be useful for the user are stored and displayed at a convenient time. Navigation is another field of appliance for AR. For example, virtual signs lead the driver of a car or provide warnings when potential hazards arrive.

Industrial and military applications

Besides design, assembly and maintenance, AR is useful for combat simulation, too. To experience a product before building a prototype is an advantage of augmented reality. Using this technology in the design phase of a car therefore helps to cut costs. It is also used in the manufacturing process of cars as well as aircrafts. Important information is displayed to workers in real time and without interrupting the working progress.

Medical applications

As stated in the section before, information can be displayed to medical personnel, too. AR can be used during surgeries, showing information about a patient to a nurse or for visualizing CT and MR scans.

Augmented-Reality-in-Healthcare
[4]
AR for entertainment

There are two main areas in entertainment where AR can be applied. These areas are games as well as sports broadcasting. While VR is mainly used for indoor gaming, AR games can also take place outdoors. Examples for such games are “Ingress” where the competitors have to hack and build up virtual portals in the real world or “Zombie run!” in which the player has to find a way to get away from zombies. [2] In sports broadcasting objects like players or a ball can be tracked and highlighted with AR technology. This technology is used especially for analyzing certain key plays and tactical behavior.

AR for the office

In the office, AR can be useful for collaboration. Working together on a subject might be more comfortable if the workers see a 3D model of the object. Also remote collaboration becomes easier and more efficient.

Education and training

The last area of application is education and training. As mentioned in the previous section, virtual models also support the learning process and help to understand complex matters. Math, geometry or engineering are only a few courses which could benefit from AR technology.

 

Game-based learning is only one example of how AR is used in different of the above mentioned areas and shows that there are no exact boundaries between these areas of application. As augmented reality is being developed further, the possibility for its usage become more and more diverse.

 

[1] D.W.F. van Krevelen, R. Poelman. A Survey of Augmented Reality – Technologies, Applications and Limitations. The International Journal of Virtual Reality, 2010, 9(2):1-20

[2] http://www.androidauthority.com/best-ar-apps-and-games-for-android-augmented-reality-584616/. Accessed 28.03.2016

[3] T. H. Höllerer and S. K. Feiner. Mobile Augmented Reality. In H. Karimi and A. Hammad, editors, Telegeoinformatics: Location-Based Computing and Services. CRC Press, Mar. 2004. ISBN 0-4153-6976-2.

[4] http://www.augmentedrealitytrends.com/augmented-reality/healthcare-industry.html. Accessed 28.03.2016

Definition Virtual Reality (VR)

The best way to get a definition of the term ‘Virtual Reality’ is to investigate both of these two words:[1]
Virtual is originally a French word and means: apparently, the possibility to exist
Reality is an English word and means: Fact, Actuality

That means VR is a seeming, an apparent actuality. When we look for a definition that fits also for the informatic we can find one from Astheimer or from Dörner.
Astheimer says that “The term Virtual Reality means, that there are techniques which allow a person, to integrate directly into a computer-generated world. In this man-machine-interface more human senses are considered at the same time.”[2]
Dörner says that“Virtual Reality (VR) describes the displacement of an observer to another, seeming reality using a computer system. Through special immersive VR technologies, the viewer should feel a presence in an apparent reality.”[3]

To stimuli the human minds with the virtual sensations there are different interfaces:
Optical Interfaces: Interfaces for the visual perception system. These interfaces are the most important ones. There are different models like glasses or Head-Mounted-Displays.[4]
optical interface

Acoustic Devices: Devices which are to address the acoustic sense of the people are in an advanced development, but it is still complicated to implement the three-dimensional listening without the use of many speakers.

Haptic Devices: This can be devices like a data glove. This devices are not very high developed, the problem is, that devices needs to adapted to each separate user.[5]
Haptic Device

Possible UseCases for VR:
Car industry: Particularly in the development phase of a new car, the VR technologies plays a major role. If a new design of a car is developed, a three-dimensional computer-model of the car will be created that the design can be examined.[6]

Medicine industry: In medicine, the VR technology is already in use for daily business. For example, organs are simulated virtually and serve as a training facility for young surgeons.[7]

Entertainment industry: The greatest potential of VR is maybe in the entertainment industry. VR is increasingly used in cinemas, theaters, amusement park and the game industry. [8]

[1] A. Preisser, 2010
[2] Astheimer et. Al., 1994
[3] Dörner, 2009
[4] [eMargin 10] 3DVISOR.com. Pdf-Online-Quelle. , 2010.02.22.
[5] CyberGlove Systems 10] CyberGlove Systems. Online-Quelle. , 2010.02.21
[6] TecCHANNEL IT EXPERTS INSIDE. Online-Quelle. (2010.02.22)
[7] Prof. Dr. med. Linus Geisler Private Homepage. Online-Quelle. (2010.02.17)
[8] Digital.World. Online-Quelle. (2010.02.22)

Definition of Augmented Reality (AR)

They are different definition of Augmented Reality (AR), simply does it means adding things like pictures, text, 3D-objects or sounds into the real world with a support device such as smartphones or special glasses. Wikipedia[1] describes Augmented Reality in March 2016 as “a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data”. In this definition, the direct view of a physical world means to add objects to the front of the real-world environment. An example is the BMW colour Head-up Display in a Car.

Indirect view of a physical world means that there is a video or a live feed with added information (eg. Score by sports event on TV.)

In the Handbook of Augmented Reality [2] defined Julie Carmigniani and Borko Furht “Augmented Reality (AR) as a real-time direct or indirect view of a physical real-world environment that has been enhanced/augmented by adding virtual computer-generated information to it.” Compared to the Wikipedia definition, Carmigniani and Furht have not closer describe the computer-generated information. The goal of AR is simplifying the user’s life by bringing virtual computer-generated information in his immediate surrounding and add also an indirect view of the real-world environment like live-video streams said Carmigniani and Furht.

Cruz-Neira et al. [3] and Milgram, Kishino [4] have called the continuum that spans between the real environment and the virtual environment as mixed reality. The Mixed Reality has been split into the Augmented Reality (AR) close to the real environment and into Augmented Virtuality (AV) close to the virtual environment. This definition is known as the Milgram’s Reality-Virtuality (RV) continuum.

Augmented Reality is a physical real-world environment with virtual objects.
Augmented Virtuality is a virtual world merged with real objects.
The picture below shows the different between AR and AV. [5]

 

Nowadays the terms Augmented Reality, Mixed Reality and Enhanced Reality often used as synonyms for a presentation of additional information in a foreground.
Azuma et al. [6] says that AR has not to be considered as restricted to a particular type of technologies, nor is it limited to the sense of sight. AR can potentially apply to all senses, augmenting smell, touch and hearing.

 

[1] https://en.wikipedia.org/wiki/Augmented_reality
[2] J. Carmigniani, B. Furht. (2011). Handbook of Augmented Reality. Springer: New York. S.3
[3] C. Cruz-Neira, D.J. Sandin, T.A. DeFanti. (1993). Surround-Screen Projection-based Virtual Reality: the Design and Implementation of the CAVE. Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, S.135–142
[4] P. Milgram and F. Kishino. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems, Vol E77-D, No.12
[5] https://valeonti.com/2010/05/10/getting-to-know-augmented-reality/
[6] Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, Blair MacIntyre. (2001). Recent Advances in Augmented Reality

Virtual Reality (VR): 50 Years of innovation!

This post describes important milestones in the development of virtual reality (VR). Of course these milestones are not concluding, but these milestones are important in the development of the Virtual Reality.

The Sensorama (1962)

In 1962, Morton Heilig built a prototype of a multi-sensory theater. This theater was called Sensorama and was able to play five short movies. The movies were displayed in a wide-angle view and provide body tilting. Additional it supplied stereo sound and the use of aromas (tastes). So the Sensorama was the first passive virtual reality machine. Unfortunately, the Sensorama was not a financial success story, so the Sensorama was built only once. [1][2]

Sensorama
Sensorama from 1962 [A]
The Sensorama patent from 1962 is linked here.

The computer program Sketchpad (1963)

In 1963, the computer scientist Ivan Sutherland wrote the first interactive computer program named Sketchpad. This program was a pioneer of today’s computer-aided design (CAD) programs and it was a major breakthrough in the evolution of computer graphics and human-computer interaction. Sketchpad was the corner stone for graphical user interfaces in our today’s personal computers. [3][4]

Here is the link to the technical report of the Sketchpad program.

Head mounted display: The Sword of Damocles (1968)

The first virtual and augmented reality head mounted display (HMD) was called “The Sword of Damocles”. It was also developed by Ivan Sutherland in 1968, with the help of one of his students, Bob Sproull. The unit was bulky and extremely heavy, but it has a binocular display and was able to display a computer-displayed perspective view of a room or a cube. [5][6]

Sword of Damocles HMD
Sword of Damocles HMD [B]
Here is the description of the “Sword of Damocles”.

The term “Virtual Reality” was born (1982)

It is not exactly known if the term “Virtual Reality” and his meaning was used for the first time in the science fiction novel “The Judas Mandala” written by Damien Broderick in 1982. But only five years later, in 1987, the term “Virtual Reality” was registered in the “Oxford English Dictionary”. [7][8]

Cave Automatic Virtual Environment (1992)

The Automatic Virtual Environment (acronym CAVE) was firstly presented in 1992. The Cave was a room-size visualization solution, where projectors creates on the different walls in a room a virtual 3d environment. The CAVE technology can be used for example in flight simulators or supports architects to visualize their constructions. The construction of a CAVE system remembers partly on a Holodeck from the science fiction movies Star Trek. [9][10][11][12]

Cave Automatic Virtual Environment
Cave Automatic Virtual Environment (Today) [C]
Nintendo Virtual Boy (1995)

The first portable head mounted console which are able to display true 3d graphics was the Nintendo “Virtual Boy”. It was released in 1995 and persisted of a head mounted display (HMD) and a controller. The display color was only monochrome, because Nintendo uses only rad LED’s. Other LED’s were at this time not efficient enough. Unfortunately, the Nintendo Virtual Boy was not successful, because there were only a five game titles which supports this new hardware. On the other hand, at this time, there were others game consoles on the market, like the Sega Saturn or the Sony PlayStation. So Nintendo sold only almost 77’000 pieces of the Virtual Boy. [13][14]

Nintendo Virtual Boy
Nintendo Virtual Boy [D]
The Matrix Movie (1999)

In 1999 the movie “The Matrix” hits theaters all over the world. The movie says that the human beings live in a virtual world, controlled by machines. With this movie, the topic of virtual reality is better known by the mainstream. [15]

Virtual Reality Head Mounted Display (2000)

In the year 2000, the US company Micro Vision developed for the US AirForce a HDTV head mounted display with a resolution of 1920×1080 pixels. [16]

First Data Glass from the Fraunhofer Institute (2009)

In 2009, the Fraunhofer Institute in Dresden presented the first data glass. This data glass contains an Eye-Tracker CMOS chip, so the glass knows where the eyes are looking for. The data glass can display information on the two screens and can give in that way the right information to the wearer. [17]

Google introduced Google Glass (2012)

The US internet company Google introduced in 2012 the optical head mounted display (OHMD) Google Glass. This glasses are combined with a small computer. The wearer communicates with the glass via voice, the wearer talks to the Google Glass. One year later, in 2013, Google started selling a prototype of Google Glass to qualified “Glass Explorers”. Also one year later, in 2014, Google started to sell the Google Glass to the public. Of course, the Google Glass or other Data Glasses belongs more to the topic of augmented reality instead of virtual reality. But these glasses are very close to the use of head mounted displays HMD. [18][19]

Facebook buy Oculus VR (2014)

Oculus VR, simply know as Oculus, is a virtual reality technology company from California in the united states. The company was founded in 2012 and developed, with the support of a Kickstarter campaign, the virtual reality head mounted display Oculus Rift. In 2014, Facebook acquired Oculus for $2 billion and in 2015, Oculus partnered with Samsung to develop the Samsung Gear VR. [20]

Oculus Rift HMD
Head Mounted Display (HMD) from Oculus [E]
Microsoft introduced HoloLens (2015)

In 2015, Microsoft presented together with Microsoft Windows 10 his new Data Glass HoloLens. This HoloLens beds 3d holograms into the real world. It also an augmented reality Glass. But it shows, where we are going in the future: The borders between reality and virtual object disappears little by little. [16][21]

Microsoft HoleLens together with Skype.
Sources:
[1] https://en.wikipedia.org/wiki/Sensorama
[2] http://www.mortonheilig.com/InventorVR.html
[3] https://en.wikipedia.org/wiki/Sketchpad
[4] Sears A., Jacko J. A. (2006), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Taylor & Francis Group: New York
[5] https://en.wikipedia.org/wiki/The_Sword_of_Damocles_(virtual_reality)
[6] http://www.roadtovr.com/watch-the-godfather-of-vr-ivan-sutherland-speak-at-the-2015-proto-awards/
[7] https://de.wikipedia.org/wiki/Damien_Broderick
[8]http://www.heise.de/tp/artikel/40/40103/1.html
[9] https://en.wikipedia.org/wiki/Cave_automatic_virtual_environment
[10] https://de.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment
[11] https://en.wikipedia.org/wiki/Holodeck
[12] https://www.christiedigital.com/en-us/3d/virtual-simulation/virtual-reality-simulator/cave-automatic-virtual-environment
[13] https://de.wikipedia.org/wiki/Virtual_Boy
[14] http://www.vrs.org.uk/virtual-reality/history.html
[15] https://en.wikipedia.org/wiki/The_Matrix
[16] https://de.wikipedia.org/wiki/Head-Mounted_Display
[17] http://www.fraunhofer.de/de/presse/presseinformationen/2009/06/Mediendienst062009Thema3.html
[18] https://en.wikipedia.org/wiki/Google_Glass
[19] https://de.wikipedia.org/wiki/Google_Glass
[20] https://en.wikipedia.org/wiki/Oculus_VR
[21] https://www.microsoft.com/microsoft-hololens/en-us

Image sources:
[A] http://www.mortonheilig.com/InventorVR.html
[B] http://www.roadtovr.com/watch-the-godfather-of-vr-ivan-sutherland-speak-at-the-2015-proto-awards
[C] https://www.christiedigital.com/en-us/3d/virtual-simulation/virtual-reality-simulator/cave-automatic-virtual-environment
[D] https://de.wikipedia.org/wiki/Virtual_Boy
[E] http://www.mtv.com/news/2467386/ps4-oculus-rift/

The inspiration to this post came from here.

Definition of Virtual Reality (VR) – Addition

At least with the movie “The Matrix” in 1999 [1] everybody knows what Virtual Reality, as briefly known as “VR”, is. In this Movie, humans live in a virtual world. This virtual world is controlled and designed by intelligent machines. But almost everybody means, that the world they live in it, is real.

The term “virtual reality” means in other words a “computer-simulated reality” [2]In this virtual generated environment, a virtual world, the user can move and interact with the objects. The perspective and views changes with the movements of the user [3].

According to Mazuryk and Gervautz [4], Virtual Reality systems could be grouped in three different levels:

Level 1: “Desktop VR”: In this level, a virtual world is presented to the users on monitors, for example, in video games. It is the simplest type of Virtual Reality and almost everybody has already seen it.

Level 2: “Fish Tank VR”: This system is based on the desktop VR. In addition, the head movements of the user will be adapted to the displayed image (head tracking). This can be realized for example with a simple Webcam or Microsoft Kinect. This YouTube video demonstrates the differences between an image with and an image without head tracking.

Level 3: “Immersive Systems”: The user sees only the computer generated virtual world and can fully interact with it. So the perspective changes according to the users’ position or moves. In this case, more technical equipment, for example a Head Mounted Displays (HMD), are necessary.

Head Mounted Display
http://www.tested.com/tech/concepts/464299-samsung-and-oculus-vr-announce-gear-vr-innovator-edition-hmd/

A special point of virtual reality is the use of computer technology [5]. Input and output devices are required for the representation of the virtual world or virtual objects. The easiest way to dive into a Virtual Reality world is to play video games with a console like a Xbox or a PlayStation. Today it is possible to use a state-of-the-art smart phone as a Head Mounted Display (HMD). Google currently offers so-called “Cardboards“. With one of these Cardboards and some appropriate apps, an Android or an iOS smart phones can be transformed easily into a HMD.

Google CardBoard
http://www.augmentedrealitytrends.com/virtual-reality/google-vr.html

Probably one of the “most real” Virtual Reality system came from the fictional “Star Trek: Next Generation” movies [6]. In this movies, there were rooms with a Virtual Reality system so-called “holodeck”. In an article in the New York Time [7], Nick Bilton wrote, that technology companies work on the realization of something like a holodeck. But it needs a lot of time and money, so a holodeck will not be available bevor 2024.

[1] https://en.wikipedia.org/wiki/The_Matrix
[2] https://en.wikipedia.org/wiki/Virtual_reality
[3] M. Mihelj, D. Novak, S. Begus. (2014). Virtual Reality Technology and Applications. Springer:London
[4] T. Mazuryk, M. Gervautz Virtual Reality, History, Applications, Technology and Future. University of Technology, Austria: Wien
[5] R. Doerner, W. Broll, P. Grimm, B. Jung. (2013). Virtual und Augmented Reality (VR/AR). Springer: Berlin
[6] https://en.wikipedia.org/wiki/Holodeck
[7] http://bits.blogs.nytimes.com/2014/01/26/disruptions-the-holodeck-begins-to-take-shape/?_r=1

How does AR work? Different approaches

The aim of augmented reality is to bring graphics, audio etc. to the real world in real time. For example for apps on a smartphone are two different approaches in use, the marker-based or image recognition and location or GPS-based approach.[1]

Marker-based

Marker-based applications are using a camera to overlay the image with content or information. This is happening in real time. The camera feed is processed and the real image is augmented by some content or information. Markers are for example QR codes. If you are scanning an QR-code and you are getting additional information on your screen than you are using a marker-based augmented reality application. Marker based applications are most common in augemented reality because it is much more simple to recognize something what is hard coded in your application.

For example Nissan made a application where you can see one of their cars in augmented reality by just using your webcam and a printed wheel. The demo is under the following link:

http://takethewheel.nissan.com.au/

Location-based[2]

Location-based applications are using the GPS tools in the device to determine the users position. The position and the direction of the user are the input parameter for the app to show at what the user is looking at.

For example:

[1] https://www.appfutura.com/blog/whats-augmented-reality-and-how-does-it-work/

[2] Location-based augmented reality on mobile phones, R. Paucher ; M. Turk IEEE, DOI 10.1109/CVPRW.2010.5543249