AR-enabled devices can also transmit what an on-site user is seeing to a remote expert, who can respond with immediate guidance. In effect, this instantly puts the expert at the user’s side, regardless of location. This capability not only improves worker performance but substantially reduces costs—as Lee Company, which sells and services building systems, has discovered. It uses AR to help its field technicians with installations and repairs. A remote expert can see what the tech is viewing through his or her AR device, guide the tech through the work to be done, and even annotate the tech’s view with instructions. Getting expert support from a central location in real time has increased Lee’s tech utilization dramatically. And, by reducing the number of repeat visits, Lee saves more than $500 per technician per month in labor and travel costs. The company calculates a return of $20 on every dollar invested in AR.

Interact.

Traditionally, people have used physical controls such as buttons, knobs, and, more recently, built-in touchscreens to interact with products. With the rise of SCPs, apps on mobile devices have increasingly replaced physical controls and allowed users to operate products remotely.

AR takes the user interface to a whole new level. A virtual control panel can be superimposed directly on the product and operated using an AR headset, hand gestures, and voice commands. Soon, users wearing smart glasses will be able to simply gaze at or point to a product to activate a virtual user interface and operate it. A worker wearing smart glasses, for instance, will be able to walk a line of factory machines, see their performance parameters, and adjust each machine without physically touching it.

The interact capability of AR is still nascent in commercial products but is revolutionary. Reality Editor, an AR app developed by the Fluid Interfaces group at MIT’s Media Lab, provides a glimpse of how it is rapidly evolving. Reality Editor makes it easy to add an interactive AR experience to any SCP. With it, people can point a smartphone or a tablet at an SCP (or, eventually, look at it through smart glasses), “see” its digital interfaces and the capabilities that can be programmed, and link those capabilities to hand gestures or voice commands or even to another smart product. For example, Reality Editor can allow a user to see a smart light bulb’s controls for color and intensity and set up voice commands like “bright” and “mood” to activate them. Or different settings of the bulb can be linked to buttons on a smart light switch the user can place anywhere that’s convenient.

The technologies underpinning these capabilities are still emerging, but the accuracy of voice commands in noisy environments is improving, and advances in gesture and gaze tracking have been rapid. GE has already tested the use of voice commands in AR experiences that enable factory workers to perform complex wiring processes in wind turbines—and has achieved a 34% increase in productivity.

Combining AR and Virtual Reality

AR’s well-known cousin, virtual reality, is a complementary but distinct technology. While AR superimposes digital information on the physical world, VR replaces physical reality with a computer-generated environment. Though VR is used mostly for entertainment applications, it can also replicate physical settings for training purposes. It is especially useful when the settings involved are hazardous or remote. Or, if the machinery required for training is not available, VR can immerse technicians in a virtual environment using holograms of the equipment. So when needed, VR adds a fourth capability—simulate—to AR’s core capabilities of visualize, instruct, and interact.

AR will be far more widely applied in business than VR will. But in some circumstances, combining AR and VR will allow users to transcend distance (by simulating faraway locations), transcend time (by reproducing historical contexts or simulating possible future situations), and transcend scale (by allowing users to engage with environments that are either too small or too big to experience directly). What’s more, bringing people together in shared virtual environments can enhance comprehension, teamwork, communication, and decision making.

Ford, for example, is using VR to create a virtual workshop where geographically dispersed engineers can collaborate in real time on holograms of vehicle prototypes. Participants can walk around and go inside these life-size 3-D holograms, working out how to refine design details such as the position of the steering wheel, the angle of the dashboard, and the location of instruments and controls without having to build an expensive physical prototype and get everyone to one location to examine it.

The U.S. Department of Homeland Security is going a step further by combining AR instructions with VR simulations to train personnel in responding to emergency situations such as explosions. This reduces costs and—in cases in which training in real environments would be dangerous—risk. The energy multinational BP overlays AR training procedures on VR simulations that replicate specific drilling conditions, like temperature, pressure, topography, and ocean currents, and that instruct teams on operations and help them practice coordinated emergency responses to disasters without high costs or risk.

How AR Creates Value

AR creates business value in two broad ways: first, by becoming part of products themselves, and second, by improving performance across the value chain—in product development, manufacturing, marketing, service, and numerous other areas.

AR as a product feature.

The capabilities of AR play into the growing design focus on creating better user interfaces and ergonomics. The way products convey important operational and safety information to users has increasingly become a point of differentiation (consider how mobile apps have supplemented or replaced embedded screens in products like Sonos audio players). AR is poised to rapidly improve such interfaces.

Dedicated AR heads-up displays, which have only recently been incorporated into automobiles, have been a key feature in elite military products, such as fighter jets, for years and have been adopted in commercial aircraft as well. These types of displays are too expensive and bulky to integrate into most products, but wearables such as smart glasses are a breakthrough interface with wide-ranging implications for all manufacturers. With smart glasses, a user can see an AR display on any product enabled to communicate with them.

If you view a kitchen oven through smart glasses, for example, you might see a virtual display that shows the baking temperature, the minutes remaining on the timer, and the recipe you are following. If you approach your car, an AR display might show you that it is locked, that the fuel tank is nearly full, and that the left-rear tire’s pressure is low.

Because an AR user interface is purely software based and delivered via the cloud, it can be personalized and can continually evolve. The incremental cost of providing such an interface is low, and manufacturers also stand to save considerable amounts when traditional buttons, switches, and dials are removed. Every product manufacturer needs to carefully consider the disruptive impact that this next-generation interface may have on its offering and competitive positioning.

AR and the value chain.

The effects of AR can already be seen across the value chain, but they are more advanced in some areas than in others. In general, visualize and instruct/guide applications are now having the greatest impact on companies’ operations, while the interact capability is still emerging and in pilot testing.

Product development.

Though engineers have been using computer-aided design (CAD) capabilities to create 3-D models for 30 years, they have been limited to interacting with those models through 2-D windows on their computer screens, which makes it harder for them to fully conceptualize designs. AR allows 3-D models to be superimposed on the physical world as holograms, enhancing engineers’ ability to evaluate and improve designs. For example, a life-size 3-D hologram of a construction machine can be positioned on the ground, and engineers can walk around it, peer under and over it, and even go inside it to fully appreciate the sight lines and ergonomics of its design at full scale in its intended setting.

AR also lets engineers superimpose CAD models on physical prototypes to compare how well they match. Volkswagen is using this technique—which makes any difference between the latest design and the prototype visually obvious—to check alignment in digital design reviews. This improves the accuracy of the quality assurance process, in which engineers previously had to painstakingly compare 2-D drawings with prototypes, and makes it five to 10 times faster.

We expect that in the near future AR-enabled devices such as phones and smart glasses, with their embedded cameras, accelerometers, GPS, and other sensors, will increasingly inform product design by exposing when, where, and how users actually interact with the product—how often a certain repair sequence is initiated, for example. In this way the AR interface will become an important source of data.

Manufacturing.

In manufacturing, processes are often complex, requiring hundreds or even thousands of steps, and mistakes are costly. As we’ve learned, AR can deliver just the right information the moment it’s needed to factory workers on assembly lines, reducing errors, enhancing efficiency, and improving productivity.

In factories, AR can also capture information from automation and control systems, secondary sensors, and asset management systems and make visible important monitoring and diagnostic data about each machine or process. Seeing information such as efficiency and defect rates in context helps maintenance technicians understand problems and prompts factory workers to do proactive maintenance that may prevent costly downtime.

Iconics, which specializes in automation software for factories and buildings, has begun to integrate AR into its products’ user interfaces. By attaching relevant information to the physical location where it will be best observed and understood, the AR interfaces enable more-efficient monitoring of machines and processes.

Logistics.

Warehouse operations are estimated to account for about 20% of all logistics costs, while picking items from shelves represents up to 65% of warehouse costs. In most warehouses, workers still perform this task by consulting a paper list of things to collect and then searching for them. This method is slow and error-prone.

The logistics giant DHL and a growing number of other companies are using AR to enhance the efficiency and accuracy of the picking process. AR instructions direct workers to the location of each product to be pulled and then suggest the best route to the next product. At DHL this approach has led to fewer errors, more-engaged workers, and productivity gains of 25%. The company is now rolling out AR-guided picking globally and testing how AR can enhance other types of warehouse operations, such as optimizing the position of goods and machines in layouts. Intel is also using AR in warehouses and has achieved a 29% reduction in picking time, with error rates falling to near zero. And the AR application is allowing new Intel workers to immediately achieve picking speeds 15% faster than those of workers who’ve had only traditional training.

Marketing and sales.

AR is redefining the concept of showrooms and product demonstrations and transforming the customer experience. When customers can see virtually how products will look or function in a real setting before buying them, they have more-accurate expectations, more confidence about their purchase decisions, and greater product satisfaction. Down the road, AR may even reduce the need for brick-and-mortar stores and showrooms altogether.

When products can be configured with different features and options—which can make them difficult and costly to stock—AR is a particularly valuable marketing tool. The construction products company AZEK, for instance, uses AR to show contractors and consumers how its decking and paver products look in various colors and arrangements. Customers can also see the simulations in context: If you look at a house through a phone or a tablet, the AR app can add a deck onto it. The experience reduces any uncertainty customers might feel about their choices and shortens the sales cycle.

In e-commerce, AR applications are allowing online shoppers to download holograms of products. Wayfair and IKEA both offer libraries with thousands of 3-D product images and apps that integrate them into a view of an actual room, enabling customers to see how furniture and decor will look in their homes. IKEA also uses its app to collect important data about product preferences in different regions.

After-sales service.

This is a function where AR shows huge potential to unlock the value-creating capabilities of SCPs. AR assists technicians serving customers in the field in much the same way it helps workers in factories: by showing predictive analytics data generated by the product, visually guiding them through repairs in real time, and connecting them with remote experts who can help optimize procedures. For example, an AR dashboard might reveal to a field technician that a specific machine part will most likely fail within a month, allowing the tech to preempt a problem for the customer by replacing it now.

At KPN, a European telecommunications service provider, field engineers conducting remote or on-site repairs use AR smart glasses to see a product’s service-history data, diagnostics, and location-based information dashboards. These AR displays help them make better decisions about how to resolve issues, producing an 11% reduction in overall costs for service teams, a 17% decrease in work-error rates, and higher repair quality.

Xerox used AR to connect field engineers with experts instead of providing service manuals and telephone support. First-time fix rates increased by 67%, and the engineers’ efficiency jumped by 20%. Meanwhile, the average time it took to resolve problems dropped by two hours, so staffing needs fell. Now Xerox is using AR to connect remote technical experts directly with customers. This has increased by 76% the rate at which technical problems are resolved by customers without any on-site help, cutting travel costs for Xerox and minimizing downtime for customers. Perhaps not surprisingly, Xerox has seen its customer satisfaction rates rise to 95%.

Human resources.

Early AR adopters like DHL, the U.S. Navy, and Boeing have already discovered the power of delivering step-by-step visual worker training on demand through AR. AR allows instruction to be tailored to a particular worker’s experience or to reflect the prevalence of particular errors. For example, if someone repeatedly makes the same kind of mistake, he can be required to use AR support until his work quality improves. At some companies, AR has reduced the training time for new employees in certain kinds of work to nearly zero and lowered the skill requirements for new hires.

This is especially advantageous for the package delivery company DHL, which faces surges in demand during peak seasons and is heavily dependent on the effective hiring and training of temporary workers. By providing real-time training and hands-on guidance on navigating warehouses and properly packing and sorting materials, AR has reduced DHL’s need for traditional instructors and increased the onboarding speed for new employees.

AR and Strategy

AR will have a widespread impact on how companies compete. As we’ve explained in our previous HBR articles, SCPs are changing the structure of almost all industries as well as the nature of competition within them—often expanding industry boundaries in the process. SCPs give rise to new strategic choices for manufacturers, ranging from what functionality to pursue and how to manage data rights and security, to whether to expand a company’s scope of products and compete in smart systems.

The increasing penetration of AR, along with its power as the human interface with SCP technologies, raises some new strategic questions. While the answers will reflect each company’s business and unique circumstances, AR will become more and more integral to every firm’s strategy.

Here are the essential questions companies face:

1. What is the range of AR opportunities in the industry, and in what sequence should they be pursued?

Companies must weigh AR’s potential impact on customers, product capabilities, and the value chain.

2. How will AR reinforce a company’s product differentiation?

AR opens up multiple differentiation paths. It can create companion experiences that expand the capabilities of products, give customers more information, and increase product loyalty. AR interfaces that enhance products’ functionality or ease of use can be big differentiators, as can those that substantially improve product support, service, and uptime. And AR’s capacity to provide new kinds of feedback on how customers use products can help companies uncover further opportunities for product differentiation.

The right differentiation path will depend on a company’s existing strategy; what competitors are doing; and the pace of technology advances, especially in hardware.

3. Where will AR have the greatest impact on cost reduction?

AR enables new efficiencies that every firm must explore. As we’ve noted, it can significantly lower the cost of training, service, assembly, design, and other parts of the value chain. It can also substantially cut manufacturing costs by reducing the need for physical interfaces.

Each company will need to prioritize AR-driven cost-reduction efforts in a way that’s consistent with its strategic positioning. Firms with sophisticated products will need to capitalize on AR’s superior and low-cost interface, while many commodity producers will focus on operational efficiencies across the value chain. In consumer industries and retail, marketing-related visualize applications are the most likely starting point. In manufacturing, instruct applications are achieving the most immediate payoff by addressing inefficiencies in engineering, production, and service. And AR’s interact capability, though still emerging, will be important across all industries with products that have customization and complex control capabilities.

4. Should the company make AR design and deployment a core strength, or will outsourcing or partnering be sufficient?

Many firms are scrambling to access the digital talent needed for AR development, which is in short supply. One skill in great demand is user experience or user interface (UX/UI) design. It’s critical to present 3-D digital information in ways that make it easy to absorb and act on; companies want to avoid making a stunning but unhelpful AR experience that defeats its core purpose. Effective AR experiences also require the right content, so people who know how to create and manage it—another novel skill—are crucial too. Digital modeling capabilities and knowledge of how to apply them in AR applications are key as well.

Over time we expect companies to create teams dedicated to AR, just as they set up such teams to build and run websites in the 1990s and 2000s. Dedicated teams will be needed to establish the infrastructure that will allow this new medium to flourish and to develop and maintain the AR content. Many firms have started to build AR skills in-house, but few have mastered them yet.

Whether to hire and train AR employees or partner with specialty software and services companies is an open question for many. Some companies have no choice but to treat AR talent as a strategic asset and invest in acquiring and developing it, given AR’s potentially large impact on competition in their business. However, if AR is important but not essential to competitive advantage, firms can partner with specialty software and services companies to leverage outside talent and technology.

The challenges, time, and cost involved in building the full set of AR technologies we have described are significant, and specialization always emerges in each component. In the early stages of AR, the number of technology and service suppliers has been limited, and companies have built internal capabilities. However, best-of-breed AR vendors with turnkey solutions are starting to appear, and it will become increasingly difficult for in-house efforts to keep up with them.

5. How will AR change communications with stakeholders?

AR complements existing print and 2-D digital communication approaches and in some cases can replace them altogether. Yet we see AR as much more than just another communication channel. It is a fundamentally new means of engaging with people. Just consider the novel way it helps people absorb and act on information and instructions.

The web, which began as a way to share technical reports, ultimately transformed business, education, and social interaction. We expect that AR will do the same thing for communication—changing it in ways far beyond what we can envision today. Companies will need to think creatively about how they can use this nascent channel.

Deploying AR

AR applications are already being piloted and deployed in products and across the value chain, and their number and breadth will only grow.

Every company needs an implementation road map that lays out how the organization will start to capture the benefits of AR in its business while building the capabilities needed to expand its use. When determining the sequence and pace of adoption, companies must consider both the technical challenges and the organizational skills involved, which vary from context to context. Specifically, organizations need to address five key questions:

1. Which development capabilities will be required?

Some AR experiences involve more complexity than others. Experiences that allow people to visualize products in different configurations or settings—like those created by IKEA, Wayfair, and AZEK—are a relatively easy place for companies to start. Consumers just need to be encouraged to download and launch AR apps, and only a mobile device is needed to use them.

Instruction applications, like the ones Boeing and GE employ in manufacturing, are more difficult to build and use. They require the capacity to develop and maintain dynamic 3-D digital content and often benefit greatly from the use of head-mounted displays or smart glasses, which are still in the early stages of development.

Apps that produce interactive experiences, which create significant value for both consumers and businesses, are the most challenging to develop. They also involve less-mature technology, such as voice or gesture recognition, and the need to integrate with software that controls SCPs. Most companies will start with static visualizations of 3-D models, but they should build the capability to move quickly into dynamic instructional experiences that have greater strategic impact.

2. How should organizations create digital content?

Every AR experience, from the least to the most sophisticated, requires content. In some cases it’s possible to repurpose existing digital content, such as product designs. Over time, however, more-complex, dynamic contextual experiences must be built from scratch, which requires specialized expertise.

Simple applications, such as an AR-enhanced furniture catalog, may need only basic product representations. More-sophisticated business instruction applications, however, such as those used for machine repair, will require accurate and highly detailed digital product representations. Companies can create these by adapting CAD models used in product development or by using digitization techniques such as 3-D scanning. The most sophisticated AR experiences also need to tap real-time data streams from enterprise business systems, SCPs, or external data sources and integrate them into the content. To prepare for broadening the AR portfolio, companies should take an inventory of existing 3-D digital assets in CAD and elsewhere and invest in digital modeling capabilities.

3. How will AR applications recognize the physical environment?

To accurately superimpose digital information on the physical world, AR technologies must recognize what they’re looking at. The simplest approach is to determine the location of the AR device using, say, GPS and show relevant information for that location without anchoring it to a specific object. This is known as an “unregistered” AR experience. Vehicle heads-up navigation displays typically work this way.

Higher-value “registered” experiences anchor information to specific objects. They can do this through markers, such as bar codes, logos, or labels, which are placed on the objects and scanned by the user with an AR device. A more powerful approach, however, uses technology that recognizes objects by comparing their shape to a catalog of 3-D models. This allows a maintenance technician, for example, to instantly recognize and interact with any type of equipment he or she is responsible for maintaining and to do so from any angle. While markers are a good starting point, shape-recognition technologies are advancing quickly, and organizations will need the capability to use them to tap into many of the highest-value AR applications.

4. What AR hardware is required?

AR experiences aimed at broad consumer audiences have typically been designed for smartphones, taking advantage of their simplicity and ubiquity. For more-sophisticated experiences, companies use tablets, which offer larger screens, better graphics, and greater processing power. Since tablet penetration is lower, companies will often provide them to users. For certain high-value applications—notably those in aircraft and automobiles—manufacturers are building dedicated AR heads-up displays into their products—a costly approach.

Eventually, however, most AR applications for service, manufacturing, and even product interfaces will require head-mounted displays that free users’ hands. This technology is currently both immature and expensive, but we expect that affordable smart glasses will become widely available in the next few years and will play a major part in releasing AR’s full power. Microsoft, Google, and Apple now offer AR technologies optimized for their own devices. However, most organizations should take a cross-platform approach that allows AR experiences to be deployed across multiple brands of phones and tablets and should make sure they’re ready for smart glasses when they arrive. (See “The Battle of the Smart Glasses.”)

5. Should you use a software-development or a content-publishing model?

Many early AR experiences have been delivered through stand-alone software applications that are downloaded, complete with digital content, to a phone or a tablet. This approach creates reliable, high-resolution experiences and allows organizations to make apps that don’t require internet connectivity. The problem with this model is that any change to the AR experience requires software developers to rewrite the app, which can create expensive bottlenecks.

An emerging alternative uses commercial AR-publishing software to create AR content and host it in the cloud. The AR experience can then be downloaded on demand using a general-purpose app running on an AR device. Like website content, the AR content can be updated or supplemented without changing the software itself—an important benefit when large amounts of information and frequent content changes are involved. The content-publishing model will become common as more and more machines and products include real-time AR interaction and control. A content-publishing capability is essential to scaling AR up across the organization.

The Broader Impact

The digital revolution, with its SCPs and explosion of data, is unleashing productivity and unlocking value across the economy. Increasingly, the constraint is not a lack of data and knowledge but how to assimilate and act on them—in other words, the interface with humans. AR is emerging as a leading solution to this challenge.

At the same time, the rapid evolution of machine learning and automation is raising serious concerns about human opportunity. Will there be enough jobs for everyone, especially for people without advanced education and knowledge? In a world of artificial intelligence and robots, will humans become obsolete?

It is easy to conclude that new technology diminishes human opportunity. Yet new inventions have been replacing human labor for centuries, and they have led to growth in employment, not a decline. Technology has dramatically increased our productivity and our standard of living. It has given rise to new kinds of offerings that meet new needs and require new types of workers. Many of today’s jobs involve products and services that did not even exist a hundred years ago. A lesson of history is that today’s digital revolution will generate new waves of innovation and new kinds of work that we cannot yet imagine.

The role of humans in this future is misunderstood. People have unique strengths that machines and algorithms will not replicate anytime soon. We have sophisticated motor skills—well beyond what robots are capable of today—that allow us to do the subtle manipulation that’s needed in, say, replacing a machine part or wiring a turbine. Even relatively less skilled work, such as drawing blood, pruning a garden, or repairing a flat tire, requires human dexterity and defies automation. Human cognition adapts instantaneously to novel situations; people easily adjust the way they interpret information, solve problems, exercise judgment, and take action to suit their circumstances. Humans have flexibility, imagination, intuition, and creative ability that for the foreseeable future are beyond the reach of any machine.

While the advances in artificial intelligence and robotics are impressive, we believe that combining the capabilities of machines with humans’ distinctive strengths will lead to far greater productivity and more value creation than either could generate alone. What’s needed to realize this opportunity is a powerful human interface that bridges the gap between the digital and physical worlds. We see AR as a historic innovation that provides this. It helps humans enhance their own capabilities by taking full advantage of new digital knowledge and machine capabilities. It will profoundly change training and skill development, allowing people to perform sophisticated work without protracted and expensive conventional instruction—a model that is inaccessible to so many today. AR, then, enables people to better tap into the digital revolution and all it has to offer.