Whether it’s trying on lipstick or clothing online, using floor plan software to find out how furniture will fit in a new home or ordering a contactless Coke using a cell phone, businesses are continually finding new ways to promote their products and services using extended reality (XR) technology.
Investment in augmented and virtual reality—together known as mixed or extended reality—is expected to mushroom in the coming years: from $10 billion in 2019 to $62 billion in 2027, and from $8 billion in 2020 to $100 billion in 2024, respectively, according to Grand View Research.
So are the opportunities for those advertisers who would use the technology to manipulate, deceive and even cause harm to consumers, say University of Michigan School of Information researchers.
In a study highlighted at the recent Conference on Human Factors in Computing Systems, the researchers studied the possible ways advertisers could manipulate the public, using scenario construction to investigate potential future manipulative XR advertisements and their harms. They came up with seven scenarios: military games, ugly furniture, fake relatives, political alternate reality, pervasive t-shirts, deodorant crush and hunger pains.
The team identified five key mechanisms of manipulative XR advertising: misleading experience marketing; inducing artificial emotions; sensing and targeting people when they are vulnerable; emotional manipulation through hyper-personalization (ads made just for the individual); and distortion of reality.
“There are many ways this can manifest,” said lead author Abraham Mhaidli, a doctoral student at the U-M School of Information. “One way could be distorting a consumer’s sense of reality by overlaying graphics on someone’s AR glasses to change what they are seeing. For example, a political ad may try to paint a picture of a booming economy and release ads on AR glasses that subtly overlay graphics which hide or erase evidence of poverty.
“Another possible danger is misleading experience marketing, which occurs when companies present ‘previews’ of products through XR technologies that are seemingly realistic and with users being unable to tell that the virtual product has been doctored.”
XR advertising is not inherently detrimental to people, but there remains a need to be vigilant for bad actors seeking to use XR technologies to harm consumers, the researchers say.
“A key aspect here is that augmented and extended reality technologies are inherently designed to alter reality, and this potential could be exploited to deceive and manipulate consumers,” said co-author Florian Schaub, assistant professor of information. “In contrast, advertising in other media might also be tailored to you or masquerade as content—think infomercials, sponsored articles in news media/magazines, product placement in movies—but they do not inherently change what you perceive as reality.”
The scenario construction was a two-step process with the researchers first becoming well versed in XR devices, XR advertising and manipulative advertising, so that they could create realistic exercises.
From this, they produced a series of prompts to guide creation of the scenarios, asking questions like: Could a certain manipulative advertising technique be replicated in XR? How could an already-existing XR advertising technique be used by bad actors?
They then created scenarios and use-cases that demonstrate how future XR manipulative techniques might manifest.
Mhaidli and Schaub say most of the harms they examined are not yet reality because XR technology is still limited: XR devices aren’t sophisticated enough yet to create political ads that give a different photorealistic perspective to individuals based on preconceived beliefs, or that can reliably sense a person’s hunger pangs or a young man’s physiological response to his crush, in order to target fast food and deodorant their direction.
“When manipulative XR ads do come, I would say that being aware of the risks is a good step towards resisting possible manipulation,” Mhaidli said.
They say more research is needed to better understand the impact that immersive XR advertising has on consumers. Time also needs to be taken to develop and evaluate a privacy framework, understand acceptable data practices for XR advertising, and develop interventions to increase XR ad literacy among consumers.
“I think we need a broader public debate and policy discussion about what kinds of manipulation of people’s reality are acceptable and whether and how XR experiences should come with disclosures about how they affect reality (similar to food nutrition labels or warnings on cigarette packs),” Schaub said.
