Extended Reality: Tactile Intelligence in a Contactless World

One of the main value propositions of Information Technology lies in the digitization of physical processes. Enterprises leverage IT technologies to digitalize hundreds of physical processes from electronic management of documents and healthcare records to the remote monitoring and inspection of machines. The value of this digital transformation is reflected on increased automation, less error-prone processes, increased accuracy, and the ability to do things from remote i.e., without physical presence and physical contact. The importance of digitization has been recently highlighted during the COVID19 pandemic, where contactless processes replaced conventional physical processes.

An integral element of the digital transformation of physical processes is their proper visualization. End-users and other stakeholders of physical processes are asking for ergonomic and user-friendly visualizations of the context of the physical world objects that are involved in the process. To this end, a variety of visualization modalities are employed to provide realistic representations of physical items in the digital world. These modalities include Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) representations, which allow end-users not only to visualize physical world processes but also to interact with them. This fosters interactions and interplay between the physical and the digital world. It also extends physical processes with digital items and enables an extended reality (XR).

The Technology Building Blocks

Extended Reality (XR) is a relatively new term, which indicates the extension of human reality with related objects and processes that reside in the physical world. XR leverages well-established technologies like VR, AR and MR to improve user experience and provide new means for humans to interact with physical world processes and their cyber-representations. The proper deployment and use of XR applications hinge on a deep understanding of the following building blocks:

• Virtual Reality (VR) enables the development of realistic, yet simulated experiences. It looks like an interactive 3D video visualization of a virtual world. A VR cyber-representation simulates physical world processes and phenomena, which can be inspired or mapped to real-world situations. However, it is also possible to create VR simulations that are completely imaginary i.e., not connected to the real world. VR has been around for several decades with many applications in areas like gaming, training, and education.
• Augmented Reality (AR) facilitates the creation of interactive experiences that involve enhanced cyber-presentations of real-world objects. For instance, there are AR applications that superimpose instructions and information over cyber-representations of equipment in industrial settings. However, in many cases, these enhanced cyber-representations include perceptive information captured from sensors like microphones, cameras and IoT devices. This enables more dynamic and realistic AR applications, which display to end-users the status of physical world objects and processes.
• Mixed Realty (MR) produces new environments based on the merging of real and virtual worlds, in ways that enable and foster the interaction between simulated and real processes. As such MR combines elements from the real world with AR and VR cyber-representations. The main characteristics of MR are the mixing of real and simulated experiences, as well as the real-time interactions between actual and simulated objects. Contrary to AR, MR enables sophisticated interactions with simulated worlds. This is one or the main differences between AR and MR: In AR interactions with simulated objects are rare and quite limited.
• Tactile Internet refers to the evolution of the current networking infrastructures in directions that enable humans, machines and their cyber-representations to interact in real-time and from remote. The emerging tactile Internet infrastructures provide ultra-low latency functionalities, typically functionalities with latency less than 1ms. This is what enables real-time interactions through the internet. At the same time, tactile internet infrastructures are secure and resilient. Tactile Internet networks can be combined with AR and MR applications to enable remote real-time interactions between real and simulated processes. This enables XR applications that break time and space boundaries and deliver control functionalities regardless of time and the users’ location.

The above-listed technologies are already available and deployed in various applications. Their availability and number of applications are expected to increase in the years to come as a result of advances in user devices (e.g., AR headsets) and in the underlying networking infrastructures (e.g., 5G and 6G networks).

Tactile Contactless XR Intelligence in Real-Life Applications and Services

Extended reality applications and tactile networking infrastructures are already changing various sectors of the economy, including:

• Education: XR applications are increasingly used in educational contexts. Their added value lies in the delivery of courses that enable users to interact with realistic simulations and real-world contexts. This provides excellent opportunities for practical, on-the-job training, beyond conventional exercises.
• Industry: There are many XR-powered use cases in industrial contexts, such as remote maintenance of assets and remote control of production operations. For instance, there are AR applications that provide real time support to technicians in equipment maintenance and repair tasks. Using these applications maintenance engineers can interact with AR cyber representations of assets to obtain unique insights about how to best perform inspection, service, and repair activities. AR applications will gradually replace conventional paper and digital manuals.
• Transport: The advent of the tactile Internet facilitates the implementation of applications that control vehicles from remote. Based on AR, it is possible to control a fleet of semi-autonomous vehicles from a single control center, in order to boost the efficiency and safety or their operations.
• Healthcare: Remote care is considered a killer XR application in the years to come. It will enable healthcare professionals and medical practitioners to receive real-time support for their tasks, while at the same time simulating the outcomes of critical interventions prior to executing them on the patients. As a prominent example, telesurgery applications are expected to make surgeons’ expertise available to remote locations with a view to supporting remote care models.
• Serious games: In recent years gaming is used in various businesses beyond entertainment purposes. For instance, serious games are developed to train sales and marketing professionals in realistic contexts. Soon, serious games will increasingly incorporate MR and AR features such as business simulations and interactions of professionals with AR apps.

Overall, the era of tactile XR applications has already started. Virtual reality development and augmented reality development are no longer far-fetched tasks for modern enterprises. Rather they are powerful technologies that improve business competitiveness in a variety of industrial sectors. Therefore, business enterprises must seek the best possible ways to deploy and fully leverage XR. Likewise, augmented reality and virtual reality companies must develop growth strategies that will capitalize on technology and market trends. For all these companies the future of augmented reality and mixed reality technologies is now.