Introduction To Augmented Reality Hardware: Augmented Reality Will Change The Way We Live Now: 1, #1

By Kaviyaraj R, Uma M and S. R. Jena

This book provides an introduction to augmented reality and covers many types of hardware that can be used in hardware reality to make the AR experience more immersive. Examples of such gear include processors, displays and sensors are explained briefly.

• Language: English
• Publisher: Soumya Ranjan Jena
• Release date: Sep 7, 2022
• ISBN: 9798847191791

Book preview

Introduction to Augmented Reality Hardware:

Augmented Reality Will Change the Way We Live Now

Kaviyaraj R Uma M

S. R. Jena

Authors Profile

Mr. Kaviyaraj Ravichandran

Doing Research on Augmented Reality in Computer Science and Engineering department, SRM Institute of Science and technology, SRM University, India. He did his Undergraduate and Postgraduate in Anna university Chennai. He specialized in the field of Augmented Reality and Internet of Things; he developed several applications models which are most useful for home automation and Society, and he designed and developed many 3D augmented reality models for education and e-commerce.

He filed 17 Patents in Intellectual Property India, One Australian Patent with grant and he has published a book along with 15+ papers in national and international journals and conferences. His research interest are Augmented Reality, Virtual Reality, Extended Reality, Data Science, and Internet of Things. He is a professional body member of IEEE, IAENG, Internet Society and SDIWC.

––––––––

Dr. Uma M

Dr. Uma Mohan received BSc and MCA degree from Bharathidasan university, Trichy. M. Tech degree from SRM University and Ph. D from Bharathiar university in Brain Computer Interface, Coimbatore, India. She has 20+ years of teaching and research experience. Currently, she is working as an Associate Professor in the Department of Computational Intelligence, School of Computing, SRM Institute of Science and Technology, Kattankulathur, Tamilnadu, India. She has published around 40+ papers in national and international journals

and conferences. She has published 6 patents. Her research interest includes Brain Computer Interface, EEG, P300, Data Mining, Recommendation System, Machine learning, Data Analytics.

––––––––

Mr. Soumya Ranjan Jena

Currently working as an Assistant Professor in Department of Computer Science and Technology at Madanapalle Institute of Technology and Science (UGC-

Autonomous), Madanapalle, Andhra Pradesh, India. He has completed M.Tech in IT from Utkal University, Bhubaneswar, Odisha, B.Tech in CSE from BPUT, Odisha and CCNA from CTTC, Bhubaneswar, Odisha. He has published 5 international books, 12 Scopus indexed research papers, 24 WoS indexed research papers, and 5 patents. His research interests include Cloud and Distributed Computing, IoT, Green Computing etc.

PREFACE

The term Augmented Reality (AR) was coined back in 1990 by Boeing researcher Tom Codell and one of the first commercial uses of this technology was in television and the military. As the world shifted towards becoming more technology-driven, AR became increasingly prominent in multiple fields, rolling out its second wave and drawing its connection towards the interactive concept. In layman's terms, AR is a technology that enhances the real world by affixing layers of digital elements onto it. These elements include computer-generated graphics, sound or video effects, haptic feedback, or sensory projects. The intention behind adding this digital information is to provide an engaging and dynamic customer experience that is enabled with the input received from varied hardware like smart glass, smart lenses, and smartphones.

AR is often mistaken with Virtual Reality. The main difference between the two is that while Virtual Reality replaces the entire real environment with an artificial one, AR is applied in a direct view of an existing real environment and adds elements like sounds, videos, or graphics onto it.

When we use a device or application enabled with the AR technology the hardware of the device or application captures the object's picture, sharing it with the computer vision program which then processes the image to gather all relevant details like the measurements of the object, any other objects which are present on the same surface, while also calculating how far these other objects are from the main object in focus.

In today’s world AR is applicable for almost all sectors. Leading tech giants such as Google, Amazon, and Facebook are making extensive use of the technology’s software. In the later years, we can observe a great degree of growth in industries that adopt AR, as they expand its application incorporating it into their everyday operations. By adopting AR, advancement can be brought in multiple fields especially if we consider the future of augmented application development.

- Authors

Contents

Section I – Introduction to Augmented Reality.......1

Chapter 1 What is Augmented Reality......................1

1.0  Augmented reality...............................................1

1.1  History of Augmented Reality....................................2

1.1.1 AR in 60s & 70s..........................................3

1.1.2 AR in 80s & 90s..........................................3

1.1.3 AR in 2000s & today......................................4

1.2  Relationship between AR, VR, MR and XR.......................8

1.3  Applications of Augmented Reality..............................11

Chapter 2 Concept of Augmented Reality..................21

2.0  How does Augmented Reality Works?...........................21

2.1  What is the technology behind it?................................23

2.1.1  Camera with a depth sensor...............................23

2.1.2  Registration tools.........................................23

2.1.3  Computer Vision.........................................23

2.1.4  Output device............................................24

2.2  Why Augmented Reality Needs Computer Vision?................24

2.3  How does Augmented Reality display digital content..............25

2.4  Working.......................................................26

2.4.1  SLAM...................................................28

2.4.2  Markers..................................................29

2.4.3  Location Based...........................................30

Section II – Augmented Reality Hardware...........31

Chapter 3 Display Fundamentals...........................31

3.0  Ocularity......................................................32

3.1  Display Types.................................................32

3.1.1  Optical See-Through Display..............................35

3.1.2  Video See-Through Display...............................36

3.1.3  Immersive displays and headsets...........................37

3.1.4  Caves and Walls..........................................38

3.1.5  Hemispheres and Domes..................................38

3.2  Augmenting Displays...........................................38

3.2.1  Binocular Augmenting Displays...........................39

3.2.2  Monocular Augmenting Displays..........................51

3.2.3  Fully Immersive Displays.................................61

3.2.4  Smartphone based displays................................69

3.3  Types of Augmented Reality Displays...........................79

Chapter 4 Audio..............................................82

4.0  Audio Display.................................................82

4.1  Conventional Audio............................................83

4.2  Monaural Sound...............................................84

4.3  Stereo Sound...................................................86

4.4  Surround Sound................................................89

4.4.1  Surround Sound Formats..................................90

4.5  Object-Based Surround Sound...................................92

4.6  Binaural Recording.............................................93

4.6.1  Real-Time Synthesis of Binaural Sound....................95

4.7  Technology behind Augmented Audio...........................96

Chapter 5 Haptics............................................98

5.0  Haptics........................................................98

5.0.1  High Resolution Haptic Display..........................101

5.1  Tactile Feedback Devices......................................102

5.1.1  Gloveone...............................................106

5.1.2  TeslaSuit...............................................109

5.1.3  Haptic Audio and Tactile Sound Transducers..............111

5.2  Force Feedback Devices.......................................112

5.2.1  CyberGlove Systems CyberGrasp.........................112

5.2.2 Geomagic Touch X Haptic Device........................114

Chapter 6 Processor........................................116

6.0  Processor......................................116

6.1  Role of Processor in Augmented Reality........................116

6.2  Heterogeneous Computing.....................................118

6.3  Heterogeneous System Architecture............................120

6.4  Why HSA.....................................120

6.5  Augmented Reality requires Heterogeneous Computing..........121

6.6  Programming for HSA.........................................127

6.6.1  OpenCL................................................128

6.6.2  CUDA..................................................131

6.6.3  hUMA..................................................137

Chapter 7 Sensors for Tracking...........................142

7.0  Sensors.......................................142

7.1  Optical Trackers..............................................143

7.2  Multicamera Optical Tracking..................................144

7.3  Optical Sensors...............................................147

7.3.1  Leap Motion Controller..................................148

7.3.2  Microsoft Kinect........................................152

7.3.3  Beacon Trackers........................................155

7.3.4  Electromagnetic Trackers................................157

7.4  Inertial Sensors...............................................158

7.4.1  How Inertial Sensors Function............................159

7.5  Acoustic Sensors..............................................160

Chapter 8 Navigation and Interaction Devices..........164

8.0  2D Versus 3D Interaction and Navigation.......................164

8.1  Hand and Gesture Tracking....................................166

8.2  Gloves.......................................167

8.3  CyberGlove III................................................168

8.4  Peregrine USB Glove..........................................169

8.5  Dual Wand/Paired Controllers..................................171

8.6  HTC Vive SteamVR Controllers...............................172

8.7  Oculus Touch.................................................172

8.8  Sony PlayStation Move VR Controllers.........................174

8.9  Navigating with Your Mind....................................176

Section I – Introduction to Augmented Reality

Chapter 1

What is Augmented Reality

––––––––

1.0  Augmented reality

The term augmented reality refers to an immersive experience of a real- world setting, in which the real-world items are augmented by computer- generated perceptual information.

Augmented reality (AR) is also known as mixed reality. Mixed reality and augmented reality are both names for the same thing (MR). A system that permits interaction in real time, reliably registers virtual and real things in three dimensions, and is referred to as augmented reality (AR) is meant to be understood as combining the real

world with a computer-generated

one figure 1. This improvement can

Figure 1.1 Augmented Reality

occasionally take place across a number of different sensory modalities, including the haptic,   the somatosensory, the visual, and the auditory. The added sensory information may either be constructive, which means that it contributes something to the natural environment, or destructive, which means that it detracts from the natural environment (i.e. masking of the natural environment). This experience has been skillfully integrated into the fabric of the real world to such a degree that it provides the appearance of being an immersive component of the setting in which it is taking place.

1.1  History of Augmented Reality

The way that we take in information and entertainment in the physical world has been fundamentally altered by the development of augmented reality technology over the past half century.

Figure 1.2 History of Augmented Reality

The technology has come a long way, and there is a growing number of applications for augmented reality (AR). Augmented reality makes tasks simpler and, without a doubt, more enjoyable, and it's already being used for everything from NASA simulations to immersive marketing experiences.

1.1.1  AR in 60s & 70s

Ivan Sutherland, a computer scientist, and professor at Harvard, invented the first head-mounted display in 1968. He called it The Sword of Damocles. The user had the opportunity to interact with computer- generated graphics that heightened their sensory perception of the outside world. 1974 was the year that Myron Kruger, a computer researcher, and artist, established a laboratory at the University of Connecticut known as Videoplace. This laboratory was completely devoted to the study of artificial reality. Within these walls, projection and camera technology were used to emit onscreen silhouettes that surrounded users for an interactive experience. This was done so that they could interact with the environment.

1.1.2  AR in 80s & 90s

Now that we have that out of the way, let's discuss how augmented reality made its way out of the lab and into a variety of industries and business applications. Tom Caudell, a researcher at Boeing, is credited with the invention of the term 'augmented reality' in the year 1990. 1992 saw the creation of 'Virtual Fixtures' by a researcher named Louis Rosenburg working in the USAF Armstrong's Research Lab. Virtual Fixtures was one of the first fully functional augmented reality systems. The system gave

military personnel the ability to virtually control and direct the movement of machinery, which enabled them to carry out tasks such as instructing US Air Force pilots on more secure flying procedures. Dancing in Cyberspace, a stage production that Julie Martin, a writer and producer, was responsible for creating in 1994, was the first time that augmented reality was utilised in the entertainment industry. Acrobats were featured in the show dancing alongside virtual objects that were projected onto the actual stage. 1998 was the year that Sportsvision became the first network to broadcast a live National Football League game using the virtual 1st & Ten graphic system, also known as the yellow yard marker. The technology displays a yellow line superimposed on top of the feed so that viewers can quickly determine where the team just advanced to in order to get a first down. This system is still in use today, despite the fact that it has significantly advanced since it was first implemented in the late 1990s. The yellow line marker and the other additional graphics have become so commonplace among viewers that the majority of them are unaware that this is an application of augmented reality technology. 1999 was the year that NASA developed a hybrid synthetic vision system for use on their X- 38 spacecraft. During their test flights, the system made use of augmented reality technology to assist in providing better navigation assistance.

1.1.3  AR in 2000s & today

In today's fast-paced technological landscape, augmented reality (AR) has already established a foothold. Let's watch how it eventually gets distributed to regular consumers and see what happens. In the year 2000, Hirokazu Kato created an open-source software library that was referred to as the ARToolKit. This package provides assistance to other developers

who are working on augmented reality software. The library superimposes simulated graphics on top of the physical world by utilising video tracking technology. In 2003, Sportvision improved the 1st & Ten graphic by incorporating the new Skycam system's feature into it. This allowed the company to present viewers with an aerial view of the playing field that had graphics superimposed on top of it. In 2009, Esquire Magazine was the first publication to use augmented reality in print media. Their goal was to make the pages appear as though they were alive. The issue of the magazine equipped with augmented reality showed a video of Robert Downey Jr. addressing the audience when the reader scanned the cover. 2013 marked the introduction of Volkswagen's Mobile Augmented Reality Technical Assistance (MARTA) app, which was primarily aimed at providing technicians with step-by-step repair instructions contained within the service manual. This application of augmented reality technology was a game-changer because it could be used in a wide variety of fields and would be used in the future to align and streamline processes. Users communicated with the internet using natural language processing commands while wearing the augmented reality technology. Users would be able to access a variety of applications using this device, including Google Maps, Google+, and Gmail, amongst others. 2016: Microsoft begins shipping its version of wearable augmented reality technology known as the HoloLens. Although it is more advanced than the Google Glass, the HoloLens comes with a hefty price tag. It is not the kind of accessory you would wear on a regular basis. 2014 was the year that Google made public its Google Glass products, which consisted of a pair of augmented reality glasses that users could put on to have more immersive experiences. The virtual reality headset functions as a wearable

computer and utilises Windows 10 as its operating system. Users are also able to scan their surroundings and create their own augmented reality experiences using this feature. 2017 saw the release of IKEA's groundbreaking augmented reality app, IKEA Place, which fundamentally altered the retail sector