COVID-19 is Creating a Future That's Out of Touch

Living with COVID-19 has suddenly made all of us hyper-aware of how often we touch things throughout our day. You tend to realize this as you find yourself perpetually washing your hands. 

Think about your typical day pre-COVID-19. You might pick up a gas pump handle, hold multiple products in the grocery store, enter payment information on key pads and touchscreens, grasp a handgrip on public transportation, push elevator buttons at work, and open plenty of doors and turn on a number of lights.

Given this daily deluge of touch points, could we be at a tipping point for how we interact with the world around us? For those fortunate enough to still be able to work, the shift has been to nearly full-time remote. Once we emerge from our social slumber and adjust to a new normal, will our future models of engagement and system interfaces shift significantly?

While touchless interfaces and systems have been around for years, there are a number of ways these capabilities may be on the brink of increased adoption. 

What could our future look like if we were to try and limit our physical contact?

Personal Devices 

Perhaps the easiest way to limit the number of systems you touch is to only touch the systems which you control. Having the ability to use your mobile phone to conduct business and perform a variety of tasks is extremely valuable when dealing with environments impacted by pandemics such as COVID-19. 

Mobile phone food orders and payments already reduce a significant number of touch points and potential risks. Can these capabilities be extended even further? Can our phone, using Near Field Communication (NFC), be used to communicate with additional systems and objects? Phones are already being used for hotel keys, movie tickets, and a variety of other virtual capabilities. Will they evolve further to serve as our singular personal controller to interact with the world around us?  

Contactless Biometrics 

Nearly 3 years ago on this blog I heralded the benefits of contactless biometric technology. This was in the midst of the market seeing increased presence of contactless fingerprint systems. It was also a period of significant growth for the face recognition industry. 

In today’s COVID-19 environment, contactless biometrics are even more important. The market leading modality of face recognition suddenly faces questions though. What happens when your users are wearing masks? Biometric vendors are now scrambling to answer this question. Will iris recognition have a resurgence? Will contactless fingerprint become more viable? 

Additional modalities may also gain momentum. Hitachi’s palm and finger vein technology has been used for years in Asia at ATMs and is now being positioned for touchless user authentication. Amazon has also been reportedly looking at their own hand recognition capability for point of sale payment systems.

Identity authentication’s use of contactless technology will only become more ubiquitous in the future.  

Natural Language Processing

We have seen a significant uptick in voice as an input mechanism for a number of years. Market adoption has been spurred by the growth of smart speakers and personal voice assistants such as Amazon Echo, Google Home, and Apple’s Siri. 

Moving away from traditional input devices driven by touch, such as the keyboard and mouse, opens the door for more voice-based systems. Beyond asking Alexa to play a song or start a timer, maybe we start telling our computers what to do. 

With custom wake words and voice commands we can suddenly start to converse with our machines to initiate more complex tasks. Combine natural language processing with robotic process automation and suddenly you have the ability to call out to a bot to perform a series of tasks. 

Gesture Recognition

Remember playing with the Microsoft Kinect? It was one of the first forays into gaming without a controller and experienced initial success by providing the ability to interact with the system simply using gestures. How will COVID-19 open up opportunities for extended uses of gesture technology? Instead of having to touch a screen or mouse, why not just wave your hand? 

A variety of sensors, such as Elliptic Labs' ultrasound products and Google’s Project Soli radar technology on the Pixel, combined with traditional computer vision capabilities leveraging existing laptop cameras, demonstrate the range of options for gesture-based solutions in the future. 

Eyesight’s Device Sense product also offers a number of ways to use gestures to interact with systems including hand and finger tracking, universal hand signs, user presence, and facial analysis. 

Touchless Displays

While at the Consumer Electronics Show (CES) this year I saw a technology from VTouch that allowed a user to perform point and click operations without touching the screen. A small device was mounted to the top of the display and its sensor could distinguish when and where a user interacted with the display. Here is a video of an attendee using their system.

Given the growth of self service kiosks why not deploy one that allows the user to navigate the system without physically touching the screen? 

Wearables

Wearable technology has traditionally been used to perform actions local to the user, but could it also be extended to serve as an interface device to other systems? 

There are a number of products in the market that use a wearable ring to capture user input. Apple has filed a patent and is pursuing the development of a smart ring that also provides the ability to point at and interact with other devices. 

Wearables stretch beyond simple jewelry. Companies such as Tap provide multi-finger input mechanisms that allow users to perform complex tasks including typing. If you really want to go to the extreme, TESLASUIT provides full body suits filled with sensors intended for VR immersion that could be adapted to other use cases in the future. 

Will we reach a day where everyone wears a device or series of sensors and is able to walk up to any interactive system, navigate the user interface, and perform a particular task? 

Spatial Computing and Mixed Reality

Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technology continues to improve and gain momentum. Given the growth of these technologies, will COVID-19 present new opportunities for us to change how we bridge the virtual world with the physical world? 

Will we be able to interact with a physical object without touching it, while also gaining additional information presented via our glasses or headset? Using a Microsoft HoloLens, could I pull up the menu of a local restaurant and have it rendered in space, swipe through the offered items, select what I want, and pay to have it delivered without having to touch anything? 

While AR and MR have struggled to establish themselves in the broader consumer market, COVID-19 may nudge them one step closer to increased adoption. 

Human-Robot Interaction

Do you remember Rosie the Robot from the Jetsons? She was a robot who was also a personal maid and assistant for the family. Since the Jetsons are often seen as the standard bearer of technology to come, this is another opportunity for touchless interfaces. There are a number of personal robot assistants in the market, however they remain more of a niche technology. Will COVID-19 change the use case potential for these devices?

How many systems, such as those in the service industry, will be replaced by robots. Instead of using touchscreens or interacting with other humans directly, could simple tasks be conducted via direct verbal communication with a robot instead? 

While conversational interfaces with robots seems far off, advances in robotics and artificial intelligence are growing exponentially. 

Assistive Technologies

COVID-19 requires us to think outside of the box when considering how we might interact with the world around us in the future. Could technologies that have historically been targeted for users with disabilities provide a new set of capabilities to serve a larger user base? 

Eye-tracking technology has been utilized for years to help users interact with systems without their hands. Studies have shown that using your eyes to control input mechanisms is not only effective but can actually make you more productive. Companies like tobii and actigaze offer the opportunity for users to interact with systems in new ways. Could we walk up to an interactive public transportation map in the future, navigate the interface, and gain personal travel information simply by using our eyes?   

Autonomous Vehicles and Ride Sharing

You may not think about autonomous vehicles as an option to address the risks of COVID-19 but there are opportunities to reduce your touch points by leveraging this technology. 

When owning and driving a personal vehicle you touch the door handles, steering wheel, and your keys on multiple occasions throughout the day. You also use the gas pump and need to deal with maintenance and service providers.

In the future, ride hailing services can provide autonomous vehicles delivered to your door for you to simply sit in on your way to your destination. Of course you are sharing a vehicle used by others, but the number of touch points and risks may be greatly reduced. 

Smart Homes and the Internet of Things (IoT)

Another technology that has been gaining momentum over the last several years is the smart home and the Internet of Things. The benefit of this technology in relation to COVID-19 is the ability to have your environment react automatically to your presence and to have smart devices performing tasks instead of humans. This includes automatic lighting, climate control, and other home automation technology. 

Beyond your own home, IoT promises a future where smart cities allow citizens to benefit from services provided by their surrounding infrastructure, sensors, and devices. Most, if not all, of these capabilities are implemented without the need to directly interact with, or touch, them. 

Interactive Holograms

As we continue to examine various input technologies, we’d be remiss to not talk about implementations of some of the more immature technologies such as holograms. For example, a Chinese company recently posted a video online of a holographic elevator button technology that allows a user to click a button that is projected in the air.

While this technology has a ways to go before being commercially viable, this particular use case was targeted at pandemic response and provides a glimpse into future system interfaces. 

Brain-Machine Interfaces

At the furthest end of the spectrum of future touchless technologies is the study of brain-machine interfaces. Depending on your view, this technology can be exhilarating, downright frightening, or cautiously somewhere in between.

Elon Musk has thrown his innovative hat into the ring with this technology and is implementing the Neuralink project. Perhaps this technology won’t even require us to leave the house and we’ll all just plug into the Matrix from our couch. 

Challenges and Opportunities Ahead

As you can see, there are a number of emerging technologies that may contribute to a future we can only imagine. One that allows us to interact with minimal touch. However, implementation and utility of these technologies is not foolproof. Beyond the residual touch points that could still present exposure to a virus lay significant technical hurdles. 

Just as we’ve seen with the uneven growth of the IoT market, the definition of standards and levels of interoperability are prerequisites for mass adoption. Too often vendors implement their solution with no vision for the larger ecosystem and as a result stunt the growth of their very industry. Cross fertilization and integration amongst these industries amplifies this problem. 

Touchless technology also presents unique challenges such as the absence of tactile stimuli. Haptic feedback of some sort is needed to provide users with appropriate and constructive feedback while moving their hands through the air. How we interact with these systems, and more importantly how we receive feedback, are tantamount to success. 

Even with the challenges, the opportunities are glorious. These technologies serve as a baseline toolkit for the creation and implementation of fantastic systems, both virtual and physical, that will allow society to interact and engage in ways previously unimaginable.