All posts by Team SimonCo

Visual aspects of Augmented Reality

Vision Summit 2016

The International Conference on Augmented Reality, Virtual Reality and Computer Graphics (SALENTO AVR) takes place annually in the city of Otranto in Italy. The main goal of this conference is to bring together researchers, scientists, and practitioners to discuss key issues, approaches, ideas, open problems, innovative applications and trends on virtual and augmented reality, 3D visualization and computer graphics. The main areas of research are in the fields of medicine, cultural heritage, arts, education, entertainment, industrial and military sectors. Some of the currently discussed topics about Augmented Reality of this conference are in the field of the visual aspects of AR or research regarding perceptual issues in AR. [1]

Legibility Issues within Augmented Reality Applications

A lot of newly developed Augmented Reality applications in the industrial environment use Head-Worn Displays (HWDs) to provide the user with technical documentation by adding textual graphics, 3D Models and animations to the screen. A main problem of augmented reality in the industrial sector is the legibility and readability of the text which is displayed with HWDs. Because it is very important to display text-style, colour and illustrations properly, efficient text visualisation is therefore critical for industrial AR applications. Currently there is no standard which defines how text should be displayed via HWDs. Because of that a lot of developers don’t know the optimal text style for specific displays and applications.

According to the literature, legibility of a text depends mainly on the following aspects: background, display technology and text style including font, size and colour. There are currently four approaches how to increase text legibility in AR:

  • Move the text on the display or direct user’s gaze towards high contrast areas.
    • For assembly and maintenance operations, where the user must be focused on the task, this is not very feasible. Some workstations may not afford darker areas of the scene.
  • Modify the text contrast with hardware solutions in the display (e.g. LCD masking)
    • This solution, while the most promising approach, is still in the research stage.
  • Adapt the text color according to the background
    • For industrial applications this is not always feasible because it could violate color-coding guidelines that can be regulated by international standards (e.g. “ASME A13.1, 2007”, “ISO 3864”-Safety Symbols)
  • Employ outline/billboard technique
    • Based on the latest results it is suggested that this is a feasible solution for industrial applications because it the flexibility and the ease of implementation

plain text and billboard

Comparing the two text styles. Plain text on the left, billboard on the right.

The latest research suggests that enhancing text contrast via software, using outline or billboard, is an effective practice to improve legibility in many situations. Maximum contrast styles, like “black text and white billboard” or “white text only” is suggested when the reading time of a scenario is important. In conclusion billboards provide the best performances but at the slight cost of scene occlusion. [2]

 

[1] http://www.salentoavr.it/tracks/#augmented

[2] A. Uva, M. Fiorentino, G. Monno, “Addressing Legibility Issues in Industrial Augmented Reality”, 2014

 

Augmented Human International Conference

Focuses on scientific contributions towards augmenting humans capabilities through technology for increased well-being and enjoyable human experience. The topics of interest include, but are not limited to: Brain-Computer Interfaces, Muscle Interfaces and Implanted Interfaces; Wearable Computing and Ubiquitous Computing; Augmented and Mixed Reality; Human Augmentation, Sensory Substitution and Fusion; Hardware and Sensors for Augmented Human Technologies; Safety, Ethics, Trust, Privacy and Security Aspects of Augmented Humanity.[1]

  • First Held: 2010
  • Last Held: February 25-27, 2016 in Geneva

The topics below are chosen from presentation held at the Augmented Human Conference at 25-27 February in Geneva 2016 and with a relevance to Augmented Reality.

Wearability Factors for Skin Interfaces [2]
There are two aspects of wearable skin interfaces to consider.
Body Aspect: location, body movements and body characteristics
Device Aspect: attachment methods, weight, insulation, accessibility, communication, interaction, aesthetics, conductors, device care and connection, battery life
Skin interfaces Skin interfaces example

A Lifelog System for Detecting Psychological Stress with Glass-equipped Temperature Sensors [3]
In this presentation a lifelog system enable us to measure biological information at all times with wearable devices . This experiment was done by using a glass that measures nasal skin temperature and makes a video at the same time. With that information the team could identify stress situation.
stress diagram

Augmented Visualization for Guiding Arm Movement in the First-Person Perspective [4]
The motivation behind the Guiding Arm Movement is to learn physical activities Tai-Chi. It can also be AR body partsused to learn any other movements. The user wears an AR-glass and sees the movement of the body as an augmented shape of body parts.  Occlusion caused by other students or objects becomes irrelevant as there is no more a traditional trainer showing the movements.
Physical activities can be learned in two steps:
1. We learn the new movement roughly, i.e. roughly learn the complete form of moves.
2. We know the basic movements and we need to learn the detail by correcting the small deviation.

Exploring Eye-Tracking-Driven Sonification for the Visually Impaired [5]
The idea is that the user can decide which information is relevant. The control is done by tracking the eye movements of the users exploration field. The device can play sounds for color, text and facial expression.
By color sonification, the color will be mapped with instruments and the pitch represents the brightness.
If there appears text in the Eye-Tracker it will be mapped to spoken sounds. With the Pitch and with Stereo it can be located in a 2D position (See picture below).
Facial Expressions mapped similar as color recognition to instruments.

text AR

 

[1] http://augmera.com
[2] Wearability Factors for Skin Interfaces.Xin Liu, Katia Vega, Pattie Maes, Joe A. Paradiso. MIT Media Lab
[3] A Lifelog System for Detecting Psychological Stress with Glass-equipped Temperature Sensors. Hiroki Yasufuku, Tsutomu Terada, Masahiko Tsukamoto
[4]Augmented Visualization for Guiding Arm Movement in the First-Person Perspective. Ping-Hsuan Han, Kuan-Wen Chen, Chen-Hsin Hsieh, Yu-Jie Huang, and Yi-Ping Hung
[5] Exploring Eye-Tracking-Driven Sonification for the Visually Impaired. Michael Dietz, Maha El Garf, Ionut Damian, Elisabeth André

Smart eyewear and the opportunities of the future

In the past and the near future, on Augmented Reality and technology conferences, one circular topic is often discussed: The evolution of the wearable technologies and the opportunities for the future in the consumer and the enterprise areas.

One discussed and upcoming subject is the next generation of smart eyewear. After Google had presented his Google Glass, the important question is, what happens next with this mobile computing platform and what are the applications in enterprises.

These topics are actually discussed at the Augmented and Virtual Reality conferences. Below there is a selection of presentations and discussions about Augmented Reality wearables, especially about Smart Glasses or Smart Eye Wear, at 2016 technology conferences.

Conference Name: Wearable Technology Show, March 2016, London, UK [1]

Presentation title: Sony Smart Eyeglass – Designing for the Human Being
Description: Customers doesn’t know the possibilities of Smart Eyeglasses and the question is, what would be good solutions and applications for them. This keynote expands Sony’s philosophical design considerations for Smart Eyeglass and what are Augmented Reality application designs and use cases.

Presentation title: An Introduction to Smart Eyewear
In this presentation several of the leading protagonists in Augmented Reality eye wear present their smart glasses. And they discuss the possibilities for consumer and enterprises with these new mobile computing platforms.

Conference Name: Augmented World Expo, June 2016, Santa Clara CA, USA [2]

Presentation title: Smart Glasses – Opportunities for the Enterprise Market
The main question of this short presentation is “what are the valuable opportunities for smart glasses in the enterprise?” The presentation will review the evolution of the technology and how that the industry has adopt this change to force up the power of human productivity.

Presentation title: The butterfly dream: Smart eyewear in 2031
The population of Augmented Reality devices will increase in the next 15 years and the eyewear looks almost like regular glasses. That makes the Augmented Reality industry very powerful and important and they will also bear great responsibilities. These topics and the future design choices will be discussed in this presentation.

Presentation title: The business impact of smart glasses for work
This presentation shares real stories on how business are deploying smart glasses into production environments. The presentation also gives an outlook how smart glasses begins to transform the workplaces and how workers will use technologies like Internet of Things IoT and big data analytics. Smart glasses connect the human workforce to with the intelligence of machines and data.

Conference Name: ARVR Innovate, April 2016, Dublin, Ireland [3]

This conference has one presentation about the topic of smart glasses and Augmented reality. The presentations title is “Immersive AR & Smart Glasses – the opportunities on the road to consumer adoption“, but there is no description of the content of this presentation.

A short discussion about smart glasses and smart eye wear

Since Google launched his Google Glass project, it is clear, that the use of wearable eyewear will be in the near future inevitable. The question is not “if”, only “when” the technology is available and technically mature.

Smart glasses will link available information’s and humans closer together. They feed augmented live information to the normal view during the people’s activities. Another question are the application areas of this technology: Where is it help- and useful and where can enterprises generate money. It is only a matter of time before smart glasses become a part of our daily lives.

Here is a list of Augmented Reality and smart glasses which are in development or already available. The list of course not closing. [4]

Epson Moverio Smart Eyewear
Link: http://www.epson.com/cgi-bin/Store/jsp/Landing/moverio-augmented-reality-smart-eyewear-technology.do

Epson Moverio BT-200 Smart Glass
Epson Moverio BT-200 Smart Glass

Meta Augmented Reality Headset
Link: https://www.metavision.com/

Meta 2 Smart Glass (Dev Kit)
Meta 2 Smart Glass (Development Kit)

Vuzix Smart Glasses
Link: https://www.vuzix.com/

Vuzix M300 Smart Glass
Vuzix M300 Smart Glass

LaForge Shima
Link: http://www.laforgeoptical.com/

La Forge Optical - Shima Smart Glass
La Forge Optical – Shima Smart Glass

Optinvent ORA-2
Link: http://www.optinvent.com/produit/ora-2

Optinvent Ora-X Smart Glass
Optinvent Ora-X Smart Glass

 

[1] Wearable Technology Show, Augmented Reality & VR Show Conference 2016, http://www.wearabletechnologyshow.net/augmented-reality-vr [28.04.2016]
[2] AWE, Augmented World Expo, http://www.augmentedworldexpo.com[30.04.2016]
[3]ARVR Innovate: Where Augmented and Virtual Reality Get Down to Business, http://innovatereality.com [01.05.2016]
[4] Hongkiat, Ten Forthcoming Augmented Reality & Smart Glasses You Can Buy, http://www.hongkiat.com/blog/augmented-reality-smart-glasses/ [01.05.2016]

Augmented Reality in Manufacturing Industry

Augmented reality in the manufacturing industry is a small part of the fourth industrial revolution also known as industry 4.0. The other three industrial revolution was the mechanization of production using water and steam power, mass production with the help of electric power and digital revolution with the help of electronics and IT. The term “Industry 4.0” was created 2011 from the German government to promote the computerization of manufacturing. Outside the german speaking countries it is known under “Digitization” and related with the IT-hype “Internet of Things”[1] and “Cyber-Physical Systems (CPS)”[2].

What is needed for AR in manufacturing industries?
Augmented reality in manufacturing industries will show real time data of machines, workplaces, equipment and providing a Human Machine Interface (HMI). A connected shop floor is a basic requirement for any further digitisation solutions. There is no standard interface to collect data from shop floor this service is provided by companies who sell SCADA, MES and automation software.

The picture shows the first five levels of a connected shop floor. Every level exchanges data with the surrounding levels in both directions.

  • The first and second level are at the physical shop floor.
  • SCADA-Network is the third level and stays for Supervisory Control And Data Acquisition. It is an Industrial control system (ICS) that monitor and control industrial processes that exist in the physical world. It is a system for remote monitoring and control that operates with signals over communication channels to PLC, PC and PID. The control system may be combined with a data acquisition system that acquires information about the status of the remote equipment.[3] (1) = https://en.wikipedia.org/wiki/SCADA
  • MES is the fourth level and stays for Manufacturing Execution Systems. MES are computerized systems used in manufacturing, to track and document the transformation of raw materials to finished goods. MES work in real time together with data from SCADA-Systems to measure and control activities in the production areas to increase productivity and improve process quality. It provides information that helps manufacturing decision makers understand how current conditions on the plant floor can be optimized to improve production output. [4]

Who are the target customers?
Users can incorporate this new technology into several different types of existing products and solutions. Easiest to implement and use of augmented reality application are by companies having experience with SCADA and MES systems. They can use the existing connectivity to equipment, devices, machines, actuators and sensors. This benefits can be used by a variety of industries, including manufacturing, water and wastewater, and oil and gas.[5]

Production data can be displayed augmented by tablet or smart glasses. An interaction between the operator and the equipment will also be possible. An implementation effort is needed to visualize 3D instruction for maintenance or showing steps to correct a machine fault. Users of the application can be production manager, operators and maintenance team.

What is offered to the customer?
Recognizing the potential for mobile displays to be used anywhere, a wide range of technology in mobile devices used. For object identification it uses barcodes and QR codes, to environmental awareness near-field communication (NFC) and global positioning system (GPS) can be used. New 3D technology allows users to place models in their geographical context. This combination of easily adjustable services, available from a central location, provides users with a nearly unlimited range of tools for interacting with their environment. Tablets will become a cost-effective way to replace aging mounted displays. Managers can monitor on their existing smartphones, further driving efficiency without any upgrade to infrastructure. [5]
The picture shows an overview of wearable devices which can be used for augmented reality use cases.[5]
Wearable Devices1

Based on the position of the user data from a device or a machine are loaded automatically. Once the operators position is located in a workstation, an overlap real images from the camera and the associated current and historical information about the device will be shown in real time. By scanning a barcode, QR-code or read of alphanumeric characters (OCR), the integrated Augmented Reality technology automatically loads all the relevant data and widgets. For example, fuel consumption of engines or read the actual motor current. With NFC data can be displayed even components are not directly accessible. For example, by a fan which is installed in the ventilation shaft. [6]
Other examples:

Use of GPS to load relevant information based on a detected location.

Iconics example in manufacturing
A manufacturing company might use NFC to identify specific pieces of machinery, or a supplier might use barcodes to identify anomalous batches in an SPC system.

Ergosign together with Inosoft: Display of operating aids at the right place. An animated 3D hand presenting in the camera image the necessary steps, this understands the operator better than an instruction sheet.

How is value achieved?
To achieve value suppliers can use their existing business model. The new augmented reality feature can be sold the same as they sell the feature for mobile devices. The value will be created in sell more license and based on that the yearly license cost of customer will be more expensive. This generates more gross margin. Together with the new feature the customer needs assistance with product launch. This services and training cost can be delivered and invoiced separately.

How to create value?
The Supplier of HMI-, MES- and SCADA-systems are the first players with an application of augmented reality for the manufacturing industry. They core business is to collect and show real-time data and control equipment over a screen. Suppliers build a new application for a wearable device that shows information as augmented reality in a production area. [5] Iconics, Inosoft, Progea are suppliers who can expand its software for mobile devices by a new feature augmented reality.

  • Iconics: HMI/SCADA/MES supplier
    The HMI/SCADA System of ICONICS will be extended by smart factory function in 2016. Augmented reality will be implemented on mobile HMI. Localisation over GPS, NFC or QR code makes it possible to identify the position and overlay the real picture with appropriate data.[7] Neuheiten in HMI/SCADA, Analytics und Augmented Reality.(24.11.2105). Pressemitteilung von: ICONICS Germany GmbH
  • Inosoft GmbH – HMI/SCADA supplier
    Inosoft presented at the SPS IPC Drives 2014 an augmented reality application for maintenance and monitoring measurements of a 3D printer. [8]
  • Progea – HMI/SCADA supplier
    Progea presented at the SPS/IPC/DRIVES 2014 their showcase how the interaction between human and machine could look in the future. With google glass additional visual information about production lines will be shown to the operator. He gets in real time informed about the status of the system and receives the necessary information to states from production data on alarms to maintenance information or data sheets. The operator can interact by touchpad to perform actions. [9]

 

[1] https://en.wikipedia.org/wiki/Internet_of_Things
[2] https://en.wikipedia.org/wiki/Cyber-physical_system
[3] https://en.wikipedia.org/wiki/SCADA
[4] http://www.iconics.com/Home/Solutions/MES.aspx
[5] Iconics White Paper. (2016). Augmented Reality and Wearable Devices, January 2016
[6] Iconics. (2015). Augmented Reality vor Ort. SPS IPC Drives 2015
[7] Neuheiten in HMI/SCADA, Analytics und Augmented Reality.(24.11.2105). Pressemitteilung von: ICONICS Germany GmbH
[8] Ergosign Augmented Reality. (2015)
[9] Progea. Augmented Reality per Google Glass. (2014)

Augmented Reality for Retail

Just a few years ago, a lot of companies marked Augmented Reality just as a hype, something to play around, mainly for the marketing. But more than more, the retail industry recognized the huge potential which comes with this technology and created the first AR-Apps. But AR in retail is more than just providing an AR-App to the customer, this article will show up, which opportunities are available for this industry.

AR as a visualization

The goal of AR as a visualization in retail is, that it can be shown to the customer, how a product, visualized as a 3D-object, can be acting under certain circumstances. Besides this fact the customer can imagine the product much better, it is important to create a WOW-effect for the customer.

Best example for the use of an AR-App as a visualization is the IKEA-App. IKEA launched an augmented reality catalogue to enable their shoppers to visualize how their product could look inside their home. The shopper just has to place the hardcopy catalogue somewhere in the room, choose one of the products in the AR-App and the piece of furniture will be placed virtually in the room. But not only that, the APP measures the size of the original product against the surrounding home to offer a true-to-life size where possible. [1]


AR as information gathering

With the AR-Technology, the retailers have the possibility to provide all available information about a product to the customer. That transforms the information gathering about a product more into a game than a laborious task. In these days everything is a complex product, regardless of whether it is a simple garment or a complex product such as a car. The time required from the user to obtain the desired information – time to  be contented – can be minimized with an AR-APP. So the potential buyer can be provided relevant information quickly and easy in a simple way. The more complex the product is, the greater is the value of Augmented Reality.

A good example here is HEINZ Ketchup. The customer has the possibility to scan the bottle and will get a lot of information about HEINZ, the product itself and cooking recipes, which would be suitable for this product. [2]

 

AR for an interactive shopping

To use the AR as an interactive shopping element creates a win-win situation for the customer and the company. The company safes personal costs, because the customer can be left unattended at the store for some time. The shopper itself can play around and see how a clothing, a make-up or jewelry acts on his own body.

Just a few good examples of an interactive shopping. The most common one is the virtual try-on-site. With this feature, the customer can do a pre-selection of clothes, without spending a lot of time in the store before. Of course only these garments are displayed, which are available in the right size at the store, and they will be delivered straight to the dressing room. The customer no longer has to search for the right clothes in the right size, everything will be made automatically. [3]

Another good example for interactive shopping comes from the cosmetic industry. An interactive AR-Screen provides tips, which other make-up products would be suitable for the customer. The corresponding make-up can be selected, on the screen the make-up is applied on a photo of the customer.


Conclusion

The best and efficient way to use AR in the retail industry is to combine all of these three possibilities. During providing a virtual 3D object, the customer should be provided with all the available information, the same should be done during an interactive shopping in a store. [4]

[1] http://www.creativeguerrillamarketing.com/augmented-reality/10-examples-augmented-reality-retail/
[2] http://www.best-practice-business.de/blog/zukunftstrends/2012/01/25/augmented-reality-als-verkaufstreiber-im-stationaren-handel/
[3] http://tech.co/future-retail-lies-augmented-reality-2015-12
[4] https://www.wearear.de/augmented-reality-im-vertrieb/

Printed media scanner apps

Paper and ink are the unbeaten leaders in transporting information, news and facts to the people all around the globe. But today, the transition from print to digital media is going on. Today, the digital content is personalized and depends on the location of the consumer [1]. And, especially on the internet, for the consumer it is easy to get further facts about a topic in less clicks. That is something, what print media cannot serve with because they are static [2]. With the use of augmented reality, print media can be enriched with additional facts and multimedia content.

Augmented Reality makers brings the paper to life

With the use of augmented reality, static papers come back to life. Additional content can be displayed by the reader with the smart phone by scanning a marker. Makers can be, for example, a special sign (a QR code) or an image and must be scanned with a special tracking software or app. A marker should have some special characteristics like the form, the colors, the position or a unique identification symbol [3]. The software or app reads the data content from the marker and open the corresponding data source from the internet or the local in-app database. The additional content is now visible for the reader on his smart phone and gives him a link to more information or plays a video corresponding to the text that he is reading. Markers and the corresponding apps gives static papers the chance to be a bit dynamic and that gives printed media much more possibilities.

Smart phone scanners for augmented media content

Mobile augmented reality scanner applications are available in two kinds. These two kinds called AR Browser or AR App. But there are some important differences between these two kind of augmented reality scanner applications.

AR Browsers are apps like Wikitude, Layar, Junaio, Aurasma or BlippAR. These apps can link scanned markers to the content of many different content providers. A content provider is usually a company who want to advertise or present products or news. The content provider has to pay the AR Browser provider, that his content is available in the AR Browser application. The AR development kits (SDK) are normally also not free to use and to create the content. The user has to download the AR Browser to get access to the content of a specific content provider. That bad thing is, that AR Browsers do not have any standards like a web browser. Each AR Browser works with different technologies to bring the augmented reality content to the user [2].

An AR App is developed for a specific purpose. That could be to deliver additional facts or information for a specific product or for a specific topic. For example, the Lego 3D catalog app can only be used for scanning content from the actual Lego catalog. An AR App could work also with offline-content (like images, texts, videos), so the user does not need internet access in contrast to an AR Browser. The disadvantages of this offline availability is, that an AR App could need much more disk space than an AR Browser [2].

Examples of AR Browser implementations

The following examples gives an idea how AR Browsers works. To show how that the given examples works, you have to download the appropriate app from the Apple iTunes, Google Play Store or Microsoft Store.

Augmented reality commercials with BlippAR [4]

Download the BlippAR app (iOS, Android, Windows) and scan with the app one of the following images. The BlippAR scanner “blipps” (scans) the image and present additional content.

Behind the “Lucky Charms” image is a game to play. To see and play that game, scan the image with the BlippAR app. That this image is enriched with augmented reality content is recognizable by the BlippAR sign on the right side.

BlippAR example
BlippAR example “Lucky Charms” [5]
Note: This presented image come from the BlippAR showroom. There are much more such examples available. It is possible, that some “blipps” can only be viewed within specific regions.

Papers and commercials with LayAR [6]

The LayAR app works similar like BlippAR. The LayAR app scan an image or a page and shows additional information to the scanned topic. For the following example the LayAR app is necessary (iOS, Android).

An example for the use of the LayAR app is the 2015 annual report from the car company Audi. This report is available digital and printed. In the report it is written on page 8, that the LayAR app can be used on several pages with a special mobile phone sign to view extra content information. Example of augmented pages in the report are page 24, page 31, page 91, page 106 or page 109. The list of the pages with additional content is not completely.

Open this link for the Audi 2015 annual report as PDF document.

The next example is an advertising about the tourism in Ohio. Scan the image with the LayAR app and the app shows a video and further tourism facts about Ohio.

LayAR example about Ohio Tourism
LayAR example about Ohio Tourism [7]
Note: This image about Ohio tourism come from the LayAR inspiration page. On the page, there are much more examples [7].

Examples of AR Apps implementations

The difference between AR Apps and AR Browsers (like LayAR and BlippAR) is, that AR Apps only support one product or one brochure. The user need for that product a special app. The next example comes from the well know Lego company. The 2016 catalog contains augmented reality Lego models, presupposed that the reader has the Lego app (iOS, Android) installed on his mobile phone.

The online Lego catalog is here available. Open it in a new browser tab, download and install the Lego app (iOS, Android) on the smart phone and scan the pages in the catalog with the yellow Lego brick (for example page 28, page 44, page 66, page 84, page 110, page 124).

How that it works with a physical paper Lego catalog shows the following video:

The value chain for augmented reality enriched print medias

Especially in the marketing and communication domains has the use of augmented reality an impact. For the augmented reality enrichment of a printed media like a catalogs or a brochures, in contrast to normal paper media, there are more parties like a 3D agency or an augmented agency involved. How that the value network could be shows the figure below. This figure is created with images from lego.com and in dependence on Santana J. (2010) [8].

Augmented Reality Marketing Value Network
Augmented Reality Marketing Value Network [8]
Conclusion

For the printed media, augmented reality is a chance to combine paper and multimedia contents. Papers with enriched content has an added value for the reader. Also for companies augmented reality advertising is interesting because the augmented media is able to tells much more about a product than a normal poster can do.

But today, augmented reality is not well known by the readers or users of a smart phone. Perhaps in the future, news papers writes only a short management summary and the reader gets his detailed information by scanning the article. It could be, but it do not have to.

[1] Standards for AR with Print: Call for a New Initiative, Ch. Perey, Online Source: http://www.perey.com/ARStandards/Perey-Standards_for_expanding_AR_with_Print.pdf [15.04.2016]
[2] B. Furht (2011), Handbook of Augmented Reality, Springer: New York
[3] A. Mehler-Bicher, L. Steiger (2014), Augmented Reality – Theorie und Praxis, De Gruyter: Oldenburg
[4] BlippAR – Online Source: https://blippar.com/en/ [17.04.2016]
[5] BlippAR Showroom – Online Source: https://blippar.com/en/showroom [17.04.2016]
[6] LayAR – Online Source: https://www.layar.com [17.04.2016]
[7] LayAR Examples – Online Source: https://www.layar.com/features/inspiration [17.04.2016]
[8] J. Santana (2010) Finding the Nexus of Business, Technical and User Needs in Building a Commercially Viable Mobile Augmented Reality (Thesis). Online-Source: http://www.ischool.berkeley.edu/files/student_projects/JSantanaThesis.pdf [17.04.2016]

AR Gaming apps

Augmented Reality Games on personal information systems like smart phones are on the rise. Smart phones games are usually easy to play and they can be played almost everywhere and at every time. One of the first mobile AR game was “ARQuake” [1], a first-person PC shooter game, created in the year 2000 [2]. The game allows the player to walk around in the real world while he plays a game in a computer generated environment. To play the game the player need a GPS system, a standard laptop on a backpack with GPS tracker and orientation sensors and ad least a custom made gun controller [3].

AR Quake Game
AR Quake needs a huge equipment [2]
Perhaps the first augmented reality smart phone game called AR Soccer. The game was available in 2004 and the player has to kick a virtual ball with his real foot into the virtual goal [1]. A game with the name AR Soccer is still available in the today’s app stores (iOS, Android), but it seems that it has not the same kind of game content like the very first version. In the same year, the first multi-user augmented reality game called “Invisible Train” was presented. In that game, players controls virtual trains on real wooden miniature railroad tracks. With their PDA’s, which are connected over Wi-Fi, they saw and steered the virtual trains [4].

Invisible Train
Invisible Train [4]
Combine virtual games with the real world

The major advantages by playing games on mobile devices like smart phones are whose mobility. They can be played anywhere; they can combine the real world with virtual elements. Augmented reality games are not restricted on a TV or PC screen and they can offer a better user experience [5]. Below there are introduced some different kind of up-to-date augmented reality games.

Massively Multiplayer Game: Ingress

Ingress (iOS, Android) is an augmented-reality massively multiplayer online location-based game [6]. The playing field is the world himself. The Ingress Game was developed by Niantic Labs from Google. The first in 2012 released version was only for Android devices, but since mid-2014 it is also in the iTunes App Store available. In the game, the player has to find and to capture “portals”. These portals are real world places like landmarks, monuments, buildings and so on. The player has to go close to these portals, at most only 40 meters away. When the player captured a portal, he can link portals together and earns in that way points for him and his fraction. The other fraction, there are two fractions, the ” Enlightened” and the ” Resistance”, tries to get the portals back. In that way, the player has to walk around, find the portals, capture or defend it, alone or together with other players [7].

Ingress
Ingress Screenshots

The business model of the Ingress game depends on the one hand on an in-game store, where the players can buy special game items with real money. On the other hand, companies can pay for portals at their location or in-game equipment. These portals or game equipment wear then a company logo and/or the name of the company [6].

Game Apps for Education

Augmented reality apps could give a hand to learn things better. It is possible to enrich texts with supplementary multimedia information like movies, images or links to further knowledge sources. For that, papers and books can contain “markers” that, when scanned with an app on a smart phone or a tablet, shows this additional information [7]. An example of such an app for tablets is the i-Wow Atlas World (iOS, Android). With this app, children get easy additional and fascinating facts to our planet when the scan a globe with the app.

Another example is the app smart phone and tablet app “Quiver” (iOS, Android): With “Quiver” children colors plain pictures of for example animals and with the app they can see the animal in 3D. Children gets with these apps an idea how the animals are living in the real world. The pictures to color for must be purchased, the app himself is free. Without these pictures, the app does not work. An example how it works is demonstrated in the video below.

Another application are simulations which helps to understand things better [7]. A famous and well known kind of simulation and educational app is “SkyView” (iOS, Android). This app helps to spots objects on the sky like stars, galaxies, constellations and other additional facts about astronomy. Apps like this are normally not free to use or includes some advertisings.

Sky View
Screenshot from the SkyView app on a iPhone

Such educational apps can help parents and teachers to help children to learn things easier and in more than one way. The children not only read the facts about a topic, they can also see the objects in 3D and they can play with it. To learn something with the assistance of augmented content is named “Augmented Learning” or “E-Learning” [8][9].

Funny pastime games: Virtual Basketball an Bowmaster AR

Finally, there are also other games for the pastime available. These kind of games usually are not free to play. Either the cost something in the app store or the contains some advertisements.

Virtual Basketball (iOS): For that game, the player need to printout a small marker. To play the game, the smart phone’s camera has to point at that marker and a virtual basketball basket is shown up. Now the player can throw basketballs into the basket to get points. This game is a funny implementation of augmented reality and a very good example of the technology.

Bowmaster Augmented Reality (iOS): The game needs also a printed marker and it works similar to the Virtual Basketball game. Instead of throw balls, in that game the player has to shoot arrows to different targets.

Virtual Basekball and Bowmaster AR
Virtual Basketball and Bowmaster AR

[1] B. Furht (2011), Handbook of Augmented Reality, Springer: New York, Page 103
[2] Origins and Evolution, Augmera – Online Source: http://augmera.com/?p=65 [14.04.2016]
[3]AR Quake, Wikipedia – Online Source https://en.wikipedia.org/wiki/ARQuake [14.04.2016]
[4] The Invisible Train, Technische Universität Graz – Online -Source: https://studierstube.icg.tugraz.at/invisible_train/ [15.04.2016]
[5] D. Schart, N. Tschanz (2015) Augmented Reality, Praxishandbuch, UVK: Konstanz, Page 32
[6] Ingress (video game), Wikipedia – Online Source: https://en.wikipedia.org/wiki/Ingress_(video_game) [14.04.2016]
[7] Ingress: The game that reveals Google’s secret war to control London, The Guardian – Online-Source: https://www.theguardian.com/technology/2014/jun/04/ingress-the-game-that-reveals-googles-secret-war-to-control-london [14.04.2016]
[7] Augmented Reality – Wikipedia – Online Source: https://en.wikipedia.org/wiki/Augmented_reality [15.04.2016]
[8] Lernen, Wikipedia – Online-Source: https://de.wikipedia.org/wiki/Lernen#E-Learning_und_Augmented_Learning [15.04.2016]
[9] Augmented learning, Wikipedia – Online Source: https://en.wikipedia.org/wiki/Augmented_learning [15.04.2016]

Augmented Reality for Traveling and Tourism

Augmented Reality technology has the ability to revolutionize the travellers experience in the future. Given the fact that more and more customers are booking their trips and accommodations online, companies have to adapt to meet the needs of these type of clients. According to Professor Dimitrios Buhalis, director of the e-tourism lab at Bournemouth University (UK), the more you can satisfy the needs of the information hungry clients, the better, since many of these clients want to see every aspect of their trip in advance. If you allow them to go inside the hotel and virtually experience how the pool, the view or the rooms look, it could be used as a powerful marketing tool. [1]

Since some years now, augmented reality has established itself in the travel and tourism industry. There are already several AR-smartphone-apps which are “mainstream”-ready for the public to use. The most common functionality of these apps is that you point your smartphone camera at a real world object (building, attraction, etc.) and then instantly receive information about these objects on your smart phone screen. But this is not the only feature an augmented reality app could provide. There are several use cases of Augmented Reality in the tourism industry which are used in hotels, museum, restaurants, public transportations and many more. In the following chapters we will describe some interesting and innovative functionalities of using augmented reality when traveling. [2]

Virtual Hotel Room Selection and Booking

As mentioned above, more and more bookings happen online. Augmented Reality could make it possible for the client to take virtual tours of a chosen hotel. This can make it very easy for the customer to see where he will be staying and also to show additional information about the hotel and its surroundings. [3] An example for this use case is the Hotel Continentale in Italy, whose team created a smartphone app which allows the user to virtually visit the hotel. [6]

Hotel Continental AR App
Hotel Continental AR App

Real-Time Information

Travelling often takes a person to an unfamiliar place. There are several AR apps which help you navigate through a city or country by providing additional information about the outside world. Access to this real-time information could be vital and important to a person’s overall travel experience. Use cases could be that a person gets information or reviews about locations (e.g. restaurants, hotels), to locate Wi-Fi Hotspots or to check a real-time weather forecast. For example using the app “shangoo” (http://www.armedia.it/shangoo/).

Another use case scenario for this type is real-time navigation around a city. A traveller isn’t familiar with the city he is visiting, so he needs location based information. With the help of Augmented Reality, a smartphone app can add additional elements, such as arrows and other useful information to a typical navigation map. [3]

Augmented Reality can also help with public transportation in a city. The app “Departures Switzerland” for example, helps to navigate within the public transport system in Switzerland. The use of this app is quite simple. A user just opens the app and holds the smartphone in the direction of a station he wants to go and he will see all the relevant information about departures overlaid on the real world image. The app is currently available iOS smartphones. [4] (https://itunes.apple.com/ch/app/departures-switzerland/id684719693)

Departures Bern
Departures Switzerland

Language Translation

For users who are travelling to a country in which they don’t speak the language, Augmented Reality can make translations of foreign text and help them to enhance their overall travel experience. It’s not just limited to street signs, but could also be helpful to translate written text on dinner menus, train schedules or newspaper headlines. In the picture bellow you see a demo of the app “Word Lens” which provides translation for several languages. It is now incorporated into Google’s “Google Translate” app suite. [5]

Word Lens Demo
Word Lens Demo

AR in restaurants

A lot of restaurants started to include Augmented Reality into their customer experience. The Inamo Restaurant in London is one of those. It lets the customers to interact with the tabletop and select their own table theme, order items from multimedia rich menu or see a live video-feed from the kitchen. [5]

Inamo Tables
Tables at the Restaurant Inamo

Historical Information

Augmented Reality based apps do not only help a user research and navigate through a city or place, but can also offer additional information such as cultural or historical information about a place. Such information could drastically improve someone’s travel experience in a foreign town. The smartphone app “Layar” for example overlaps historical sites over the real world image. Another use case is when visiting a museum, a user can take a photo of an exhibit and bring them to life with interactive information. [3]

Berlin Historical AR
Berlin Historical AR

 

 

[1] http://www.theguardian.com/travel/2014/oct/25/travel-industry-virtual-augmented-reality
[2] http://www.cntraveler.com/stories/2014-03-31/best-augmented-reality-travel-apps
[3] http://www.augment.com/blog/augmented-reality-in-tourism/
[4] http://www.augmentedrealitytrends.com/ar-app/departures-switzerland.html
[5] https://thinkdigital.travel/wp-content/uploads/2013/04/10-AR-Best-Practices-in-Tourism.pdf
[6] http://www.esimple.it/en/item/ar-360-panorama

 

AR guides us: Navigation and the search for POI’s

One of the first augmented reality apps was ” Wikitude Drive” for Android. This app was available in the Android Marketplace at the end of the year 2010 [1] and displayed the planned route directly into the camera-image. It combines the navigation map data, the actual position with the real world seen through the mobile phone’s camera.

Wikitude Drive App
Wikitude Drive App [2]
This was almost six years ago. Actually the app is not anymore available in the app stores and other navigation apps do not support augmented reality navigation. But the next generation of augmented reality navigation systems will perhaps arrive soon and it could looks like the picture from the company WayRay.com below.

WayRay Navion
WayRay Navion [3]
The company writes at their homepage, that this new navigation system called “Navion” should be released in 2016. The compact “Navion” device, it does not work with a smart phone, is placed on the car’s dashboard. The device projects holographic information onto the road ahead and does not need any additional eyewear [3].

Another actual available augmented reality smart phone app for cars is the app called “iOnRoad” (iOS, Android). The app detects with the mobile phone camera other cars driving in front of you and calculate the distance between the two cars in seconds. If the car is too close, the app warns the driver. The app checks also, if the car drives over an unbroken line and gives a signal. This features can enhance the safety while driving a car [4].

iOnRoad Smart Phone app
iOnRoad smart phone app [4]
Point of interest smart phone apps

But there are not only augmented reality navigation apps available. Apps like  Layar (iOS, Android) or some city travel guides (eTips) displays through the smart phone camera directly interesting places, right where you are. Apps like these two examples can help tourists to find the right locations like shops, restaurants or hotels (POI’s). They often display not only the name of the locations, but also additional information’s like prices or review. The pictures below show a AR project from Hotels.com and Wikitude [5].

http://www.wikitude.com/showcase/hotels-com-search-for-hotels/
Hotels.com – AR search for hotels [5]
Apps like Layar or Hotels.com depends on location-based services (LBS). Location-based services are computer programs that combine location data with additional information’s like point of interests or reviews from other users [6].

The business model of navigation or POI’s apps

Navigation apps, similar apps like iOnRoad or apps for tourists are often not free to use. The user or customer, has to pay for the app or he has to buy some In-App Purchases for the real interesting or additional features.

So the developer’s business model is to develop and to sell the app as much as possible. Includes the app an offline map, the developer has to update the map in regular intervals. The developer is responsible that all online services, like traffic information, has to be available all the time.

For the developer, the use of augmented reality features is not mandatory, but it is a unique selling argument which gives him an advantage over his competitors.

The users benefit is that location-based augmented reality apps connects the real environment with media contents from the internet. The user gets the relevant information for his actual context (time, location, situation). The augmented reality apps combine the real with the virtual world and the user gets information from both sides.

[1] oe24: Erste Augmente Reality Navi-App startet – Online Source [12.04.2016]
[2] Wikitude Navigation – Online Source [12.04.2016]
[3] WayRay Navion – Online Source [12.04.2016]
[4] iOnRoad Smart Phone app – Online Source [10.04.2016]
[5] Wikitude and Hotels.com – Search for Hotels – Online Source [09.04.2016]
[6] D. Schart, N. Tschanz (2015) Augmented Reality, Praxishandbuch, UVK: Konstanz, Page 105

A short history of augmented reality

The very first appearances of augmented reality happened in the late 1960s and 1970s. Ivan Sutherland is credited with creating the very first head mounted display, which was introduced at the University of Utah in 1968. [1]work
First Head Mounted Display

In 1975 Myron Krueger creates Videoplace that allows users to interact with virtual objects for the first time. [2] The term augmented reality wasn’t really established until the early 1990s when the two Boeing employees Tom Caudell and David Mizell used the term to describe the extension of the range of view with work-specific information, which was done with a heads-up-display (HUD). The 1990s marked an important period for augmented reality, as several factors, such as the development of mobile computers and smartphones with more and more efficient hardware. In 1997 the first mobile augmented reality system (MARS) was developed at the Columbia University in New York. The system used GPS data to display information on a head-worn-display to show more information about the building and the campus to the person wearing it. The calculation power came from a laptop which was carried on the back.
LEA_NHNT_Schianchi
First Mobile Augmented Reality System

After the year 2000 the amount of augmented reality application increased drastically in several industries. The first augmented reality browser for smartphones, Wikitude, was introduced in 2008. Since then the development of new augmented reality apps and games has increased, also thanks to the breakthrough of the iPhone. In 2014 wearable augmented reality makes headlines thanks to Google Glass. Since then augmented reality has spread and more and more apps are being developed. [3]

[1] http://www.tomshardware.com/reviews/ar-vr-technology-discussion,3811-3.html
[2] https://educationar.wikispaces.com/History+of+Augmented+Reality
[3] http://media2mult.uni-osnabrueck.de/pmwiki/fields/wp13/index.php?n=AR.GeschichteDerAugmentedReality