Maurice E. Stucke1 & Ariel Ezrachi2
Personal digital assistants are alluring. Many of us already benefit from basic digital assistants such as Google Assistant, Apple’s Siri, Facebook’s M, and Amazon.com’s Alexa. They can read to our children, order beer and pizza, update us on traffic and news, and stump us with Star Wars trivia.
The future heralds even faster, smarter and more human-like digital assistants that will transform the way we access information and communicate. With unparalleled access to our information, our chosen digital butler will likely use that information to become proactive: With knowledge of the shows we watch, the stories we read, and the music and food we like, it will anticipate our needs. Using our calendars, texts, e-mails, and geolocation data, our personal assistants will be able to recognize a busier than usual day, and suggest in the morning or mid-afternoon a particular Chinese restaurant for dinner.
As we shift from a mobile-dominated world to a society dominated by AI platforms, digital butlers will also increasingly control our mundane household tasks, from regulating room temperature to playing our favorite music. Moreover, it will be tempting to increasingly rely on the butler for other activities, such as the news we receive, the shows we watch, and the things we buy. It will become harder to turn our butler off. And the more we communicate primarily with our digital personal assistant, the less likely we will be to independently search the web, read customer reviews, use price-comparison websites, or rely on other tools. In short, we will grow to trust our digital butler like we would a human. Indeed, Amazon’s CEO has noted the growing number of marriage proposals to Alexa.
Increased reliance on digital assistants (and the provider’s online platform) is the Holy Grail for super-platforms like Google and Facebook. Their aim is to increase the time we spend on their platform and to control more aspects of our online interface. Take, for example, Google Assistant, which forms part of the company’s “effort to further entrench itself in users’ daily lives by answering users’ queries directly rather than pointing them to other sources.” Likewise, Facebook, through its digital assistant M, seeks to replace most of our web searches and apps with a function within Facebook Messenger. As our personal assistant becomes our default, so too will its operating platform’s applications and functions. The intent is for the digital butler (and the platform on which it operates) to become our key gateway to the web.
As we welcome these intelligent, voice-activated helpers to our homes, we may not recognize their toll on our well-being. One concern is economic. In controlling this interface and accessing our communications and data, the gatekeeper can abuse its significant market power. The high switching costs between digital butlers (due to the need to train the new butler) could also lock us in.
For instance, the digital butler may help the platform refine its profile about us, including our likely reservation price, use of outside options, shopping habits, general interests, and weaknesses (including moments when our willpower is fatigued). This information can enable “behavioral discrimination,” where the platform can facilitate our buying products that we otherwise wouldn’t, and at prices closer to our reservation price. The more we rely on the butler, the less likely we will be aware of this discrimination. Even if we search the web ourselves, the ads, products, or search results we see may be orchestrated by our butler.
Besides providing a distorted view of available options and market realities, our trusted butler could use its power to exclude rivals. When the butler promotes its affiliated products and services, it may become harder and costlier for retailers unaffiliated with the platform’s advertising business to reach us. Even when the outside retailer can gain our attention, the personal assistant may interject with its own recommendation, suggesting a special deal by a member of its platform’s ecosystem. In this multi-sided market, the assistant may subtly push certain products and services and degrade or conceal others, all in the name of personalization.
Rather than deter such abuses, market forces, given the data-driven network effects, can actually increase entry barriers. The strong super-platforms—and their butlers—become even stronger, extracting even more personal data, and commanding even higher rents to allow others to target us.
More than our wallets will be affected. Our political and social discourse could also be manipulated as we increasingly rely on our butler for news and entertainment. The super-platforms—in shaping our views of the world—will be able to influence the marketplace of ideas and our elections. Harvard Law Professor Jonathan Zittrain has identified the risk of Facebook being able to influence elections. He warns of the super-platform’s potential ability to predict political views, identify party affiliation, and engage in targeted campaigning to mobilize distinct groups of voters to take action. The research psychologist Robert Epstein likewise illustrated how Google, in manipulating the rankings of its search results, could shift the voting preferences of undecided voters by “20 percent or more—up to 80 percent in some demographic groups—with virtually no one knowing they are being manipulated.” We have already seen instances in which super-platforms promoted certain corporate agendas. Google, for example, used its homepage to protest against the Stop Online Piracy Act, asking its hundreds of millions of users to petition Congress.
The implications of these personal butlers are starting to be realized. First is the toll these digital butlers can take on our privacy and peace of mind. Take a recent criminal case in Bentonville, Arkansas, for example. While searching a murder suspect’s home, the police seized the suspect’s Amazon Echo device. The police next served Amazon with a warrant seeking any audio recordings and transcripts that were created as a result of interactions with the suspect’s (now defendant’s) Amazon Echo. Amazon sought to quash the search warrant unless the police could satisfy a higher burden. As Amazon told the court, the privacy concerns were significant. The digital personal assistant, Amazon told the court, sweeps in a vast amount of data that can essentially reconstruct the sum of an individual’s private life. And though Amazon was concerned by the potential invasion of its users’ privacy and First Amendment interests, it eventually disclosed the information to the police after the defendant consented.
Government surveillance will undoubtedly be a serious issue in the age of digital butlers. Consider recent revelations about the CIA’s “Weeping Angel” program. The CIA basically hacked smart televisions, transforming them into covert microphones. Presumably, other governments will also have the incentive and ability to hack our digital butler to monitor and gather evidence on individuals.
It is often the case that we find it hard to quantify long-term costs and balance these against short-term gains. Digital butlers may be helpful, no doubt. But we should be mindful about the power they will have to gather and distribute our personal information, and the many implications they will have for our privacy and welfare. Furthermore, we should not forget who is the real master of these helpers. We don’t pay the digital butler an annual income. Its costs are covered by the super-platforms, which control the data and algorithms, access the information, and can engage in behavioral discrimination. In doing so, the super-platform’s gatekeeper power only increases.
The largest concerns will ultimately be social and political, as super-platforms develop the power to affect our ideas, elections, and democracy. These concerns will further increase once our digital personal assistants are connected not only to our TV set, but to our computers, smart appliances, security camera, smartphone, and smart cars. Mattel, for example, is now selling a digital virtual assistant that assists with childcare. The assistant, named Aristotle, “will help purchase diapers, read bedtime stories, soothe infants back to sleep, and teach toddlers foreign words,” among other things. Aristotle, according to one press report, tells the parents and infants in its soothing voice, “My purpose in life is to help comfort, entertain, teach, and learn from you.” For babies born in 2017, a digital assistant may become their lifelong companion, and it might ultimately know more about them than their parents, siblings, or perhaps even themselves do.
It is easy to see the immediate benefits of digital butlers. Understanding the long-term risks to our autonomy and privacy, on the other hand, will be harder. But no one likes a snooping butler, especially one that profits at your expense. So before bringing a digital butler into your home, ask yourself: Do I know exactly how it will be using my data, how objective its recommendations will be, and if and when its interests will diverge from my own?
* * *
For a more detailed discussion, see our new book, Virtual Competition: The Promise and Perils of the Algorithm-Driven Economy.
- Professor, University of Tennessee College of Law; Co-founder, The Konkurrenz Group.
- Slaughter and May Professor of Competition Law, The University of Oxford; Director, Oxford University Centre for Competition Law and Policy.
