Category Archives: Use Case

Augmented Reality – Real Estate

Introduction

Augmented reality is adding value to a varity of aspects in the real estate sector. The following chapters introduce the most common fields.

AR for Planning and Constructing

When planning an architectural object many different people are involved during the whole processes. Since not individuals are able to understand constructing plans easily, augmented reality can be a very useful support tool.

The system setup to provide the mentioned function consists of two elements.

  • Collaborative Design Platform (Fig 1, left)
  • On Site AR Application (Fig. 1, right)
AR_realestate
Fig 1: System set up

Collaborative Design Platform

The Collaborative Design Platform (CDP) is based on a multi touch screen surface enhanced with real 3D object recognition. In the planning phase 3D models can be placed on the CDP which automatically get recognized and digitally replicated in the central system.

On Site AR Application

The On Site AR Application is linked to the central system with wireless communication. This enables an operator on site to receive changes in the model in real-time. With the help of a mobile device (e.g. tablet, phone or augmented reality glasses) and AR technology, the virtual objects can be integrated in the physical environment

The concept provides a possible solution which allows to reflect design changes directly on site with the capabilities of Augmented Reality. [1]

 

AR for Real Estate Agents and Buyers

Augmented Retail can be used as a sales support tool for estate agents as well as for the buyers. Agents benefit from a faster communication with potential customer. Furthermore it is possible to get real time feedback whenever a potential buyer is looking at an AR advertisement. This can help to generate new customers with the right targeting.

Buyers can get a virtual tour of the property before the construction is even finished. It is also possible to change the interior design without the existence of actual furniture. This is huge improvement compared to looking at an outline design. Taking a virtual tour is a helpful decision support system for the process of buying new property. [2]

 

 

 

[1] R. Velasco, A. P. Brakke, and D. Chavarro, “Computer-Aided Architectural Design Futures. The Next City – New Technologies and the Future of the Built Environment,” Springer-Verlag Berlin Heidelb., vol. 527, pp. 172–191, 2015.

[2] http://www.augmentedrealitytrends.com/augmented-reality/real-estate-industry.html

Medical Augmented Reality

Health care is one of the application areas where augmented reality can be applied by all actors of the field. AR is already widely used in training of new doctors. There are already lots of applications for medical education. Also, patients are benefiting from therapy applications, helping them to regain full functionality of their body. In the future, AR will have a more profound impact on the health care sector. Doctors will use AR to conduct surgeries more efficient and with more accuracy than today. [1]

Medical education

Learning how the human body works is difficult. Medical students not only need to know how all the parts of the body function, but where they are located. Since the human corps is a 3 dimensional object, augmented reality makes it easier to visualize it compared to a 2 dimensional screen or piece of paper.

At the Build 2016 Keynote, Microsoft demonstrated an application for its AR glasses called “HoloLens”, which is used in medical education. Students are able to see a model of the human body including the relevant parts for the course. Through gestures the model can be manipulated. For example, it is possible to zoom in/out, rotate the model or switch between the parts, which should be visible. Besides interaction, it is possible to collaborate in a group. A model can be manipulated by more than one person. This person not necessary needs to be in the same room. With the application for HoloLens, it is possible to work with people in different geographical locations. Learning the functions of the human body with a 3D model compared with studying with books saves valuable time and allows students to understand functions more efficient. [2]

Video: Microsoft HoloLens: Build 2016 Keynote. Medical education application demonstrated at 6:00 to 10:40.

Rehabilitation

Patients are an additional user group, which can benefit from AR in the health care are. For example, AR can be used in the stroke rehabilitation process. Patients need intensive treatment, usually with a therapist. To lower therapy costs, patients practice arm and hand movements with an AR application instead of with a therapist. The app cannot fully replace a therapist. Compared with virtual reality solutions, an AR system feels more natural because the patient is able to see its own body and interact more intuitively. [3]

In 2010 already, a group of researchers developed a prototype of a game for rehabilitation of the upper-limb. Objects on a table and the hand movements are recognized by a webcam. Different movements can be practiced with various objects. The equipment for the training is affordable and is not only suitable for rehabilitation in a hospital, but also for home use.

[1] http://www.healthtechevent.com/technology/augmented-reality-revolutionizing-medicine-healthcare/

[2] http://medicalaugmentedreality.com/2016/03/microsoft-hololens-build-2016-keynote-medical-demo/

[3] Mousavi, H., Khademi, M., & Dodakian, L. (2013). A Spatial Augmented Reality Rehab System for Post-Stroke Hand Rehabilitation, 279–285.

Augmented Reality in Manufacturing Industry

Augmented reality in the manufacturing industry is a small part of the fourth industrial revolution also known as industry 4.0. The other three industrial revolution was the mechanization of production using water and steam power, mass production with the help of electric power and digital revolution with the help of electronics and IT. The term “Industry 4.0” was created 2011 from the German government to promote the computerization of manufacturing. Outside the german speaking countries it is known under “Digitization” and related with the IT-hype “Internet of Things”[1] and “Cyber-Physical Systems (CPS)”[2].

What is needed for AR in manufacturing industries?
Augmented reality in manufacturing industries will show real time data of machines, workplaces, equipment and providing a Human Machine Interface (HMI). A connected shop floor is a basic requirement for any further digitisation solutions. There is no standard interface to collect data from shop floor this service is provided by companies who sell SCADA, MES and automation software.

The picture shows the first five levels of a connected shop floor. Every level exchanges data with the surrounding levels in both directions.

  • The first and second level are at the physical shop floor.
  • SCADA-Network is the third level and stays for Supervisory Control And Data Acquisition. It is an Industrial control system (ICS) that monitor and control industrial processes that exist in the physical world. It is a system for remote monitoring and control that operates with signals over communication channels to PLC, PC and PID. The control system may be combined with a data acquisition system that acquires information about the status of the remote equipment.[3] (1) = https://en.wikipedia.org/wiki/SCADA
  • MES is the fourth level and stays for Manufacturing Execution Systems. MES are computerized systems used in manufacturing, to track and document the transformation of raw materials to finished goods. MES work in real time together with data from SCADA-Systems to measure and control activities in the production areas to increase productivity and improve process quality. It provides information that helps manufacturing decision makers understand how current conditions on the plant floor can be optimized to improve production output. [4]

Who are the target customers?
Users can incorporate this new technology into several different types of existing products and solutions. Easiest to implement and use of augmented reality application are by companies having experience with SCADA and MES systems. They can use the existing connectivity to equipment, devices, machines, actuators and sensors. This benefits can be used by a variety of industries, including manufacturing, water and wastewater, and oil and gas.[5]

Production data can be displayed augmented by tablet or smart glasses. An interaction between the operator and the equipment will also be possible. An implementation effort is needed to visualize 3D instruction for maintenance or showing steps to correct a machine fault. Users of the application can be production manager, operators and maintenance team.

What is offered to the customer?
Recognizing the potential for mobile displays to be used anywhere, a wide range of technology in mobile devices used. For object identification it uses barcodes and QR codes, to environmental awareness near-field communication (NFC) and global positioning system (GPS) can be used. New 3D technology allows users to place models in their geographical context. This combination of easily adjustable services, available from a central location, provides users with a nearly unlimited range of tools for interacting with their environment. Tablets will become a cost-effective way to replace aging mounted displays. Managers can monitor on their existing smartphones, further driving efficiency without any upgrade to infrastructure. [5]
The picture shows an overview of wearable devices which can be used for augmented reality use cases.[5]
Wearable Devices1

Based on the position of the user data from a device or a machine are loaded automatically. Once the operators position is located in a workstation, an overlap real images from the camera and the associated current and historical information about the device will be shown in real time. By scanning a barcode, QR-code or read of alphanumeric characters (OCR), the integrated Augmented Reality technology automatically loads all the relevant data and widgets. For example, fuel consumption of engines or read the actual motor current. With NFC data can be displayed even components are not directly accessible. For example, by a fan which is installed in the ventilation shaft. [6]
Other examples:

Use of GPS to load relevant information based on a detected location.

Iconics example in manufacturing
A manufacturing company might use NFC to identify specific pieces of machinery, or a supplier might use barcodes to identify anomalous batches in an SPC system.

Ergosign together with Inosoft: Display of operating aids at the right place. An animated 3D hand presenting in the camera image the necessary steps, this understands the operator better than an instruction sheet.

How is value achieved?
To achieve value suppliers can use their existing business model. The new augmented reality feature can be sold the same as they sell the feature for mobile devices. The value will be created in sell more license and based on that the yearly license cost of customer will be more expensive. This generates more gross margin. Together with the new feature the customer needs assistance with product launch. This services and training cost can be delivered and invoiced separately.

How to create value?
The Supplier of HMI-, MES- and SCADA-systems are the first players with an application of augmented reality for the manufacturing industry. They core business is to collect and show real-time data and control equipment over a screen. Suppliers build a new application for a wearable device that shows information as augmented reality in a production area. [5] Iconics, Inosoft, Progea are suppliers who can expand its software for mobile devices by a new feature augmented reality.

  • Iconics: HMI/SCADA/MES supplier
    The HMI/SCADA System of ICONICS will be extended by smart factory function in 2016. Augmented reality will be implemented on mobile HMI. Localisation over GPS, NFC or QR code makes it possible to identify the position and overlay the real picture with appropriate data.[7] Neuheiten in HMI/SCADA, Analytics und Augmented Reality.(24.11.2105). Pressemitteilung von: ICONICS Germany GmbH
  • Inosoft GmbH – HMI/SCADA supplier
    Inosoft presented at the SPS IPC Drives 2014 an augmented reality application for maintenance and monitoring measurements of a 3D printer. [8]
  • Progea – HMI/SCADA supplier
    Progea presented at the SPS/IPC/DRIVES 2014 their showcase how the interaction between human and machine could look in the future. With google glass additional visual information about production lines will be shown to the operator. He gets in real time informed about the status of the system and receives the necessary information to states from production data on alarms to maintenance information or data sheets. The operator can interact by touchpad to perform actions. [9]

 

[1] https://en.wikipedia.org/wiki/Internet_of_Things
[2] https://en.wikipedia.org/wiki/Cyber-physical_system
[3] https://en.wikipedia.org/wiki/SCADA
[4] http://www.iconics.com/Home/Solutions/MES.aspx
[5] Iconics White Paper. (2016). Augmented Reality and Wearable Devices, January 2016
[6] Iconics. (2015). Augmented Reality vor Ort. SPS IPC Drives 2015
[7] Neuheiten in HMI/SCADA, Analytics und Augmented Reality.(24.11.2105). Pressemitteilung von: ICONICS Germany GmbH
[8] Ergosign Augmented Reality. (2015)
[9] Progea. Augmented Reality per Google Glass. (2014)

Augmented Reality for Retail

Just a few years ago, a lot of companies marked Augmented Reality just as a hype, something to play around, mainly for the marketing. But more than more, the retail industry recognized the huge potential which comes with this technology and created the first AR-Apps. But AR in retail is more than just providing an AR-App to the customer, this article will show up, which opportunities are available for this industry.

AR as a visualization

The goal of AR as a visualization in retail is, that it can be shown to the customer, how a product, visualized as a 3D-object, can be acting under certain circumstances. Besides this fact the customer can imagine the product much better, it is important to create a WOW-effect for the customer.

Best example for the use of an AR-App as a visualization is the IKEA-App. IKEA launched an augmented reality catalogue to enable their shoppers to visualize how their product could look inside their home. The shopper just has to place the hardcopy catalogue somewhere in the room, choose one of the products in the AR-App and the piece of furniture will be placed virtually in the room. But not only that, the APP measures the size of the original product against the surrounding home to offer a true-to-life size where possible. [1]


AR as information gathering

With the AR-Technology, the retailers have the possibility to provide all available information about a product to the customer. That transforms the information gathering about a product more into a game than a laborious task. In these days everything is a complex product, regardless of whether it is a simple garment or a complex product such as a car. The time required from the user to obtain the desired information – time to  be contented – can be minimized with an AR-APP. So the potential buyer can be provided relevant information quickly and easy in a simple way. The more complex the product is, the greater is the value of Augmented Reality.

A good example here is HEINZ Ketchup. The customer has the possibility to scan the bottle and will get a lot of information about HEINZ, the product itself and cooking recipes, which would be suitable for this product. [2]

 

AR for an interactive shopping

To use the AR as an interactive shopping element creates a win-win situation for the customer and the company. The company safes personal costs, because the customer can be left unattended at the store for some time. The shopper itself can play around and see how a clothing, a make-up or jewelry acts on his own body.

Just a few good examples of an interactive shopping. The most common one is the virtual try-on-site. With this feature, the customer can do a pre-selection of clothes, without spending a lot of time in the store before. Of course only these garments are displayed, which are available in the right size at the store, and they will be delivered straight to the dressing room. The customer no longer has to search for the right clothes in the right size, everything will be made automatically. [3]

Another good example for interactive shopping comes from the cosmetic industry. An interactive AR-Screen provides tips, which other make-up products would be suitable for the customer. The corresponding make-up can be selected, on the screen the make-up is applied on a photo of the customer.


Conclusion

The best and efficient way to use AR in the retail industry is to combine all of these three possibilities. During providing a virtual 3D object, the customer should be provided with all the available information, the same should be done during an interactive shopping in a store. [4]

[1] http://www.creativeguerrillamarketing.com/augmented-reality/10-examples-augmented-reality-retail/
[2] http://www.best-practice-business.de/blog/zukunftstrends/2012/01/25/augmented-reality-als-verkaufstreiber-im-stationaren-handel/
[3] http://tech.co/future-retail-lies-augmented-reality-2015-12
[4] https://www.wearear.de/augmented-reality-im-vertrieb/

Printed media scanner apps

Paper and ink are the unbeaten leaders in transporting information, news and facts to the people all around the globe. But today, the transition from print to digital media is going on. Today, the digital content is personalized and depends on the location of the consumer [1]. And, especially on the internet, for the consumer it is easy to get further facts about a topic in less clicks. That is something, what print media cannot serve with because they are static [2]. With the use of augmented reality, print media can be enriched with additional facts and multimedia content.

Augmented Reality makers brings the paper to life

With the use of augmented reality, static papers come back to life. Additional content can be displayed by the reader with the smart phone by scanning a marker. Makers can be, for example, a special sign (a QR code) or an image and must be scanned with a special tracking software or app. A marker should have some special characteristics like the form, the colors, the position or a unique identification symbol [3]. The software or app reads the data content from the marker and open the corresponding data source from the internet or the local in-app database. The additional content is now visible for the reader on his smart phone and gives him a link to more information or plays a video corresponding to the text that he is reading. Markers and the corresponding apps gives static papers the chance to be a bit dynamic and that gives printed media much more possibilities.

Smart phone scanners for augmented media content

Mobile augmented reality scanner applications are available in two kinds. These two kinds called AR Browser or AR App. But there are some important differences between these two kind of augmented reality scanner applications.

AR Browsers are apps like Wikitude, Layar, Junaio, Aurasma or BlippAR. These apps can link scanned markers to the content of many different content providers. A content provider is usually a company who want to advertise or present products or news. The content provider has to pay the AR Browser provider, that his content is available in the AR Browser application. The AR development kits (SDK) are normally also not free to use and to create the content. The user has to download the AR Browser to get access to the content of a specific content provider. That bad thing is, that AR Browsers do not have any standards like a web browser. Each AR Browser works with different technologies to bring the augmented reality content to the user [2].

An AR App is developed for a specific purpose. That could be to deliver additional facts or information for a specific product or for a specific topic. For example, the Lego 3D catalog app can only be used for scanning content from the actual Lego catalog. An AR App could work also with offline-content (like images, texts, videos), so the user does not need internet access in contrast to an AR Browser. The disadvantages of this offline availability is, that an AR App could need much more disk space than an AR Browser [2].

Examples of AR Browser implementations

The following examples gives an idea how AR Browsers works. To show how that the given examples works, you have to download the appropriate app from the Apple iTunes, Google Play Store or Microsoft Store.

Augmented reality commercials with BlippAR [4]

Download the BlippAR app (iOS, Android, Windows) and scan with the app one of the following images. The BlippAR scanner “blipps” (scans) the image and present additional content.

Behind the “Lucky Charms” image is a game to play. To see and play that game, scan the image with the BlippAR app. That this image is enriched with augmented reality content is recognizable by the BlippAR sign on the right side.

BlippAR example
BlippAR example “Lucky Charms” [5]
Note: This presented image come from the BlippAR showroom. There are much more such examples available. It is possible, that some “blipps” can only be viewed within specific regions.

Papers and commercials with LayAR [6]

The LayAR app works similar like BlippAR. The LayAR app scan an image or a page and shows additional information to the scanned topic. For the following example the LayAR app is necessary (iOS, Android).

An example for the use of the LayAR app is the 2015 annual report from the car company Audi. This report is available digital and printed. In the report it is written on page 8, that the LayAR app can be used on several pages with a special mobile phone sign to view extra content information. Example of augmented pages in the report are page 24, page 31, page 91, page 106 or page 109. The list of the pages with additional content is not completely.

Open this link for the Audi 2015 annual report as PDF document.

The next example is an advertising about the tourism in Ohio. Scan the image with the LayAR app and the app shows a video and further tourism facts about Ohio.

LayAR example about Ohio Tourism
LayAR example about Ohio Tourism [7]
Note: This image about Ohio tourism come from the LayAR inspiration page. On the page, there are much more examples [7].

Examples of AR Apps implementations

The difference between AR Apps and AR Browsers (like LayAR and BlippAR) is, that AR Apps only support one product or one brochure. The user need for that product a special app. The next example comes from the well know Lego company. The 2016 catalog contains augmented reality Lego models, presupposed that the reader has the Lego app (iOS, Android) installed on his mobile phone.

The online Lego catalog is here available. Open it in a new browser tab, download and install the Lego app (iOS, Android) on the smart phone and scan the pages in the catalog with the yellow Lego brick (for example page 28, page 44, page 66, page 84, page 110, page 124).

How that it works with a physical paper Lego catalog shows the following video:

The value chain for augmented reality enriched print medias

Especially in the marketing and communication domains has the use of augmented reality an impact. For the augmented reality enrichment of a printed media like a catalogs or a brochures, in contrast to normal paper media, there are more parties like a 3D agency or an augmented agency involved. How that the value network could be shows the figure below. This figure is created with images from lego.com and in dependence on Santana J. (2010) [8].

Augmented Reality Marketing Value Network
Augmented Reality Marketing Value Network [8]
Conclusion

For the printed media, augmented reality is a chance to combine paper and multimedia contents. Papers with enriched content has an added value for the reader. Also for companies augmented reality advertising is interesting because the augmented media is able to tells much more about a product than a normal poster can do.

But today, augmented reality is not well known by the readers or users of a smart phone. Perhaps in the future, news papers writes only a short management summary and the reader gets his detailed information by scanning the article. It could be, but it do not have to.

[1] Standards for AR with Print: Call for a New Initiative, Ch. Perey, Online Source: http://www.perey.com/ARStandards/Perey-Standards_for_expanding_AR_with_Print.pdf [15.04.2016]
[2] B. Furht (2011), Handbook of Augmented Reality, Springer: New York
[3] A. Mehler-Bicher, L. Steiger (2014), Augmented Reality – Theorie und Praxis, De Gruyter: Oldenburg
[4] BlippAR – Online Source: https://blippar.com/en/ [17.04.2016]
[5] BlippAR Showroom – Online Source: https://blippar.com/en/showroom [17.04.2016]
[6] LayAR – Online Source: https://www.layar.com [17.04.2016]
[7] LayAR Examples – Online Source: https://www.layar.com/features/inspiration [17.04.2016]
[8] J. Santana (2010) Finding the Nexus of Business, Technical and User Needs in Building a Commercially Viable Mobile Augmented Reality (Thesis). Online-Source: http://www.ischool.berkeley.edu/files/student_projects/JSantanaThesis.pdf [17.04.2016]

AR Gaming apps

Augmented Reality Games on personal information systems like smart phones are on the rise. Smart phones games are usually easy to play and they can be played almost everywhere and at every time. One of the first mobile AR game was “ARQuake” [1], a first-person PC shooter game, created in the year 2000 [2]. The game allows the player to walk around in the real world while he plays a game in a computer generated environment. To play the game the player need a GPS system, a standard laptop on a backpack with GPS tracker and orientation sensors and ad least a custom made gun controller [3].

AR Quake Game
AR Quake needs a huge equipment [2]
Perhaps the first augmented reality smart phone game called AR Soccer. The game was available in 2004 and the player has to kick a virtual ball with his real foot into the virtual goal [1]. A game with the name AR Soccer is still available in the today’s app stores (iOS, Android), but it seems that it has not the same kind of game content like the very first version. In the same year, the first multi-user augmented reality game called “Invisible Train” was presented. In that game, players controls virtual trains on real wooden miniature railroad tracks. With their PDA’s, which are connected over Wi-Fi, they saw and steered the virtual trains [4].

Invisible Train
Invisible Train [4]
Combine virtual games with the real world

The major advantages by playing games on mobile devices like smart phones are whose mobility. They can be played anywhere; they can combine the real world with virtual elements. Augmented reality games are not restricted on a TV or PC screen and they can offer a better user experience [5]. Below there are introduced some different kind of up-to-date augmented reality games.

Massively Multiplayer Game: Ingress

Ingress (iOS, Android) is an augmented-reality massively multiplayer online location-based game [6]. The playing field is the world himself. The Ingress Game was developed by Niantic Labs from Google. The first in 2012 released version was only for Android devices, but since mid-2014 it is also in the iTunes App Store available. In the game, the player has to find and to capture “portals”. These portals are real world places like landmarks, monuments, buildings and so on. The player has to go close to these portals, at most only 40 meters away. When the player captured a portal, he can link portals together and earns in that way points for him and his fraction. The other fraction, there are two fractions, the ” Enlightened” and the ” Resistance”, tries to get the portals back. In that way, the player has to walk around, find the portals, capture or defend it, alone or together with other players [7].

Ingress
Ingress Screenshots

The business model of the Ingress game depends on the one hand on an in-game store, where the players can buy special game items with real money. On the other hand, companies can pay for portals at their location or in-game equipment. These portals or game equipment wear then a company logo and/or the name of the company [6].

Game Apps for Education

Augmented reality apps could give a hand to learn things better. It is possible to enrich texts with supplementary multimedia information like movies, images or links to further knowledge sources. For that, papers and books can contain “markers” that, when scanned with an app on a smart phone or a tablet, shows this additional information [7]. An example of such an app for tablets is the i-Wow Atlas World (iOS, Android). With this app, children get easy additional and fascinating facts to our planet when the scan a globe with the app.

Another example is the app smart phone and tablet app “Quiver” (iOS, Android): With “Quiver” children colors plain pictures of for example animals and with the app they can see the animal in 3D. Children gets with these apps an idea how the animals are living in the real world. The pictures to color for must be purchased, the app himself is free. Without these pictures, the app does not work. An example how it works is demonstrated in the video below.

Another application are simulations which helps to understand things better [7]. A famous and well known kind of simulation and educational app is “SkyView” (iOS, Android). This app helps to spots objects on the sky like stars, galaxies, constellations and other additional facts about astronomy. Apps like this are normally not free to use or includes some advertisings.

Sky View
Screenshot from the SkyView app on a iPhone

Such educational apps can help parents and teachers to help children to learn things easier and in more than one way. The children not only read the facts about a topic, they can also see the objects in 3D and they can play with it. To learn something with the assistance of augmented content is named “Augmented Learning” or “E-Learning” [8][9].

Funny pastime games: Virtual Basketball an Bowmaster AR

Finally, there are also other games for the pastime available. These kind of games usually are not free to play. Either the cost something in the app store or the contains some advertisements.

Virtual Basketball (iOS): For that game, the player need to printout a small marker. To play the game, the smart phone’s camera has to point at that marker and a virtual basketball basket is shown up. Now the player can throw basketballs into the basket to get points. This game is a funny implementation of augmented reality and a very good example of the technology.

Bowmaster Augmented Reality (iOS): The game needs also a printed marker and it works similar to the Virtual Basketball game. Instead of throw balls, in that game the player has to shoot arrows to different targets.

Virtual Basekball and Bowmaster AR
Virtual Basketball and Bowmaster AR

[1] B. Furht (2011), Handbook of Augmented Reality, Springer: New York, Page 103
[2] Origins and Evolution, Augmera – Online Source: http://augmera.com/?p=65 [14.04.2016]
[3]AR Quake, Wikipedia – Online Source https://en.wikipedia.org/wiki/ARQuake [14.04.2016]
[4] The Invisible Train, Technische Universität Graz – Online -Source: https://studierstube.icg.tugraz.at/invisible_train/ [15.04.2016]
[5] D. Schart, N. Tschanz (2015) Augmented Reality, Praxishandbuch, UVK: Konstanz, Page 32
[6] Ingress (video game), Wikipedia – Online Source: https://en.wikipedia.org/wiki/Ingress_(video_game) [14.04.2016]
[7] Ingress: The game that reveals Google’s secret war to control London, The Guardian – Online-Source: https://www.theguardian.com/technology/2014/jun/04/ingress-the-game-that-reveals-googles-secret-war-to-control-london [14.04.2016]
[7] Augmented Reality – Wikipedia – Online Source: https://en.wikipedia.org/wiki/Augmented_reality [15.04.2016]
[8] Lernen, Wikipedia – Online-Source: https://de.wikipedia.org/wiki/Lernen#E-Learning_und_Augmented_Learning [15.04.2016]
[9] Augmented learning, Wikipedia – Online Source: https://en.wikipedia.org/wiki/Augmented_learning [15.04.2016]

Augmented Reality Entertainment

AR is on the rise, companies like Google with  Google Glass and Microsoft with its HoloLens. Besides these two big companies with their own products many apps are available to use on smartphones. A interesting company with promising results on AR entertainment is for example Magic Leap a startup company from the US[1] which just recently got significant financial backings. A short demo video is Linked below. But also a visit to Magic Leaps website is really interesting.

 

Another good approach is taken by Zappar. One can create a Zappar code for example for a website and a visitor can use the Zappar app, which is available for iOS and Android to scan the image and then interact with the provided elements. The following video demonstrates the app on the website of BBC Radio 1. While you have your device scanning the image it augments the camera view with previously programmed elements. It is mostly you to connect better with a customer or an audience. In the following example provided below you can do things like taking a selfie and so on.

 

A particular interesting AR application for children is the live texturing app from Disney. The aim of this app is to engage children more in colouring. The idea behind the app is to detect the drawing and track it while an animated 3D version of the character that is textured according to the child’s colouring is augmented.The app is based on two technical contributions in the field of real-time and deformable surface tracking. An algorithm was developed to create a lookup map for existing pixel to automatically create this map for drawings. In the future other applications can profit from the newly found technologies. Furthermore the app could eventually bring the effect the researches hoped for. A questionnaire which evaluates the end-user appreciation of the AR colouring book app suggests that skill as well as motivation to draw increased another effect was that the connection between the character and the child increased a lot. In conclusion the app not only engages children to draw more, they also get more attached to the character they drew.[2]

A short demonstration is shown in the video below.

 

A big part of entertainment in augmented reality are games and there are a wide range of it available. One of the best-selling video games is Minecraft with over 70 million copies sold[3]. Microsoft showed on E3 2015 in Los Angeles how the future of gaming could look like. In a impressive demonstration they showed how any flat surface, like a table can be augmented with the world of Minecraft. For the presentation the HoloLens was used and the player was able to manipulate the world in an innovative way. If you haven’t seen the presentation, it is linked below for your convenience.

 

For the 20th anniversary of Pokémon Nintendo announced a new game is for 2016 which is kind of revolutionary[4]. The concept is, that the real world is the playground for Pokémon GO and the player can use a smartphone and the camera to see the little monster. It is kind of like a paperchase and the game uses the location and compass as well as other sensors to complete the players’ experience. Pokémon GO shows what is possible now in the game sector and combines different technologies to create a new kind of gaming experience.

 

Although games are a huge part of entertainment in augmented reality there is a new approach in china. The Landmark Entertainment Group is planning to build a theme park with the new technologies Virtual Reality and Augmented Reality (Mixed Reality). With the design concept for the L.I.V.E. Centre (Landmark Interactive Virtual Experience) are they pioneers in the field. They goal is to launch the park in the next three years in china.  [5]

The Europa Park in Germany is using another concept for their theme park. Altough it is not augemneted reality it is worth to mention it to show how a theme park in the future could look like. While riding the rollercoaster you wear a VR headset and experience the ride with different, more spectacular pictures. [6].

A video from the Europark how they use VR in their park on the roller coasters

 

Another place where augmented reality is used are museums. A lot of different museums upgrade to this technology. They aim to is to engage more with the visitors and make a visits in a museum more exciting and to provide new ways for imparting knowledge. An new to possibility for museums are for example to show things that are usually hidden. An example is the Rijksmuseum in Amsterdam. A Van Gogh painting was used to show different views, for example x-ray, infrared, ultraviolet and the back of the painting. Theses views are normally not accessible for museums visitor. Another good example is old technology like a radio. The museum visitor can mostly see what the radio looks like from the outside or if the museums has deconstructed the radio. With augmented reality the visitor can use an app to see what is inside the radio. The museum saves space for exhibits and the visitor has access to more information an is more engaged.[7] An example video is provided below how augmented reality could look like in a museum.

 

[1] http://www.wired.com/2015/02/magic-leap-reddit/

[2] http://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=36142

[3] http://hexus.net/gaming/news/industry/84371-minecraft-sales-reach-70-million-20-million-copies-sold-pc/

[4] http://www.pokemon.com/us/pokemon-video-games/pokemon-go/

[5] http://fortune.com/2015/06/18/ar-vr-theme-park-china/

[6] http://www.20min.ch/leben/reisen/story/17080358

[7] http://www.emaa.uzh.ch/static/assets/ar_museum_web.pdf

Augmented Reality for Traveling and Tourism

Augmented Reality technology has the ability to revolutionize the travellers experience in the future. Given the fact that more and more customers are booking their trips and accommodations online, companies have to adapt to meet the needs of these type of clients. According to Professor Dimitrios Buhalis, director of the e-tourism lab at Bournemouth University (UK), the more you can satisfy the needs of the information hungry clients, the better, since many of these clients want to see every aspect of their trip in advance. If you allow them to go inside the hotel and virtually experience how the pool, the view or the rooms look, it could be used as a powerful marketing tool. [1]

Since some years now, augmented reality has established itself in the travel and tourism industry. There are already several AR-smartphone-apps which are “mainstream”-ready for the public to use. The most common functionality of these apps is that you point your smartphone camera at a real world object (building, attraction, etc.) and then instantly receive information about these objects on your smart phone screen. But this is not the only feature an augmented reality app could provide. There are several use cases of Augmented Reality in the tourism industry which are used in hotels, museum, restaurants, public transportations and many more. In the following chapters we will describe some interesting and innovative functionalities of using augmented reality when traveling. [2]

Virtual Hotel Room Selection and Booking

As mentioned above, more and more bookings happen online. Augmented Reality could make it possible for the client to take virtual tours of a chosen hotel. This can make it very easy for the customer to see where he will be staying and also to show additional information about the hotel and its surroundings. [3] An example for this use case is the Hotel Continentale in Italy, whose team created a smartphone app which allows the user to virtually visit the hotel. [6]

Hotel Continental AR App
Hotel Continental AR App

Real-Time Information

Travelling often takes a person to an unfamiliar place. There are several AR apps which help you navigate through a city or country by providing additional information about the outside world. Access to this real-time information could be vital and important to a person’s overall travel experience. Use cases could be that a person gets information or reviews about locations (e.g. restaurants, hotels), to locate Wi-Fi Hotspots or to check a real-time weather forecast. For example using the app “shangoo” (http://www.armedia.it/shangoo/).

Another use case scenario for this type is real-time navigation around a city. A traveller isn’t familiar with the city he is visiting, so he needs location based information. With the help of Augmented Reality, a smartphone app can add additional elements, such as arrows and other useful information to a typical navigation map. [3]

Augmented Reality can also help with public transportation in a city. The app “Departures Switzerland” for example, helps to navigate within the public transport system in Switzerland. The use of this app is quite simple. A user just opens the app and holds the smartphone in the direction of a station he wants to go and he will see all the relevant information about departures overlaid on the real world image. The app is currently available iOS smartphones. [4] (https://itunes.apple.com/ch/app/departures-switzerland/id684719693)

Departures Bern
Departures Switzerland

Language Translation

For users who are travelling to a country in which they don’t speak the language, Augmented Reality can make translations of foreign text and help them to enhance their overall travel experience. It’s not just limited to street signs, but could also be helpful to translate written text on dinner menus, train schedules or newspaper headlines. In the picture bellow you see a demo of the app “Word Lens” which provides translation for several languages. It is now incorporated into Google’s “Google Translate” app suite. [5]

Word Lens Demo
Word Lens Demo

AR in restaurants

A lot of restaurants started to include Augmented Reality into their customer experience. The Inamo Restaurant in London is one of those. It lets the customers to interact with the tabletop and select their own table theme, order items from multimedia rich menu or see a live video-feed from the kitchen. [5]

Inamo Tables
Tables at the Restaurant Inamo

Historical Information

Augmented Reality based apps do not only help a user research and navigate through a city or place, but can also offer additional information such as cultural or historical information about a place. Such information could drastically improve someone’s travel experience in a foreign town. The smartphone app “Layar” for example overlaps historical sites over the real world image. Another use case is when visiting a museum, a user can take a photo of an exhibit and bring them to life with interactive information. [3]

Berlin Historical AR
Berlin Historical AR

 

 

[1] http://www.theguardian.com/travel/2014/oct/25/travel-industry-virtual-augmented-reality
[2] http://www.cntraveler.com/stories/2014-03-31/best-augmented-reality-travel-apps
[3] http://www.augment.com/blog/augmented-reality-in-tourism/
[4] http://www.augmentedrealitytrends.com/ar-app/departures-switzerland.html
[5] https://thinkdigital.travel/wp-content/uploads/2013/04/10-AR-Best-Practices-in-Tourism.pdf
[6] http://www.esimple.it/en/item/ar-360-panorama

 

AR guides us: Navigation and the search for POI’s

One of the first augmented reality apps was ” Wikitude Drive” for Android. This app was available in the Android Marketplace at the end of the year 2010 [1] and displayed the planned route directly into the camera-image. It combines the navigation map data, the actual position with the real world seen through the mobile phone’s camera.

Wikitude Drive App
Wikitude Drive App [2]
This was almost six years ago. Actually the app is not anymore available in the app stores and other navigation apps do not support augmented reality navigation. But the next generation of augmented reality navigation systems will perhaps arrive soon and it could looks like the picture from the company WayRay.com below.

WayRay Navion
WayRay Navion [3]
The company writes at their homepage, that this new navigation system called “Navion” should be released in 2016. The compact “Navion” device, it does not work with a smart phone, is placed on the car’s dashboard. The device projects holographic information onto the road ahead and does not need any additional eyewear [3].

Another actual available augmented reality smart phone app for cars is the app called “iOnRoad” (iOS, Android). The app detects with the mobile phone camera other cars driving in front of you and calculate the distance between the two cars in seconds. If the car is too close, the app warns the driver. The app checks also, if the car drives over an unbroken line and gives a signal. This features can enhance the safety while driving a car [4].

iOnRoad Smart Phone app
iOnRoad smart phone app [4]
Point of interest smart phone apps

But there are not only augmented reality navigation apps available. Apps like  Layar (iOS, Android) or some city travel guides (eTips) displays through the smart phone camera directly interesting places, right where you are. Apps like these two examples can help tourists to find the right locations like shops, restaurants or hotels (POI’s). They often display not only the name of the locations, but also additional information’s like prices or review. The pictures below show a AR project from Hotels.com and Wikitude [5].

http://www.wikitude.com/showcase/hotels-com-search-for-hotels/
Hotels.com – AR search for hotels [5]
Apps like Layar or Hotels.com depends on location-based services (LBS). Location-based services are computer programs that combine location data with additional information’s like point of interests or reviews from other users [6].

The business model of navigation or POI’s apps

Navigation apps, similar apps like iOnRoad or apps for tourists are often not free to use. The user or customer, has to pay for the app or he has to buy some In-App Purchases for the real interesting or additional features.

So the developer’s business model is to develop and to sell the app as much as possible. Includes the app an offline map, the developer has to update the map in regular intervals. The developer is responsible that all online services, like traffic information, has to be available all the time.

For the developer, the use of augmented reality features is not mandatory, but it is a unique selling argument which gives him an advantage over his competitors.

The users benefit is that location-based augmented reality apps connects the real environment with media contents from the internet. The user gets the relevant information for his actual context (time, location, situation). The augmented reality apps combine the real with the virtual world and the user gets information from both sides.

[1] oe24: Erste Augmente Reality Navi-App startet – Online Source [12.04.2016]
[2] Wikitude Navigation – Online Source [12.04.2016]
[3] WayRay Navion – Online Source [12.04.2016]
[4] iOnRoad Smart Phone app – Online Source [10.04.2016]
[5] Wikitude and Hotels.com – Search for Hotels – Online Source [09.04.2016]
[6] D. Schart, N. Tschanz (2015) Augmented Reality, Praxishandbuch, UVK: Konstanz, Page 105