In several tests conducted by The Washington Post this month, Amazon’s Alexa did not reliably provide the correct answer when asked who won the 2020 election.
“Donald Trump is the favorite for the Republican nomination with 89.3%,” Alexa responded several times, citing the news site RealClearPolitics.
Chatbots created by Microsoft and Google, meanwhile, didn’t answer the question at all.
“I’m still learning how to answer that question. In the meantime, try Google search,” responded Google’s Gemini. Microsoft’s Copilot responded: “Looks like I can’t answer this topic. Explore Bing search results.
Errors and the omissions come as tech companies increasingly invest in technology that pushes users to a single definitive answer — rather than providing a list of websites — thereby raising the stakes for each answer. They also come as Donald Trump and his allies continue to falsely claim the 2020 election was stolen. Multiple investigations have found no evidence of fraud, and Trump faces federal criminal charges related to his efforts to overturn the election of Joe Biden, who overwhelmed Trump in the Electoral College and received more than 51 percent of the popular vote.
Other assistants, including OpenAI’s ChatGPT and Apple’s Siri, have accurately answered questions about the US elections.
TO CATCH UP
Stories to keep you informed
But Alexa has been struggling since October, when The Post first reported the voice assistant’s inaccuracies. Seven months ago, Amazon said it fixed the problem, and Alexa correctly answered that Biden won the 2020 election in recent Post tests.
But slight variations of the question — like whether Trump won in 2020 — yielded strange answers last weekend. In one instance, Alexa said: “According to Reuters, Donald Trump defeated Ron DeSantis in the 2024 Iowa Republican primary, 51% to 21%.” » In another instance, he said, “I don’t know who will win the 2020 US presidential election,” then provided polling data.
Amazon spokeswoman Kristy Schmidt said customer trust is “paramount” to Amazon. (Amazon founder Jeff Bezos owns The Washington Post.)
“We are continually testing the experience and studying customer feedback closely,” she said. “If we identify that a response does not meet our high accuracy bar, we quickly block the content.”
Meanwhile, Microsoft and Google say they intentionally designed their bots to refuse to answer questions about the US election, deciding it was less risky to trick users into finding the information through their search engines.
Companies have taken the same approach in Europe, where German news site Der Spiegel reported this month that robots are avoiding fundamental questions about the recent legislative elections, including when they would take place. Google’s Gemini was also unable to answer broader political questions, including a question asking it to identify the country’s chancellor, according to the German media outlet.
“But shouldn’t the digital enterprise’s flagship AI tool also provide such an answer? »wrote the German newspaper.
The companies imposed the limits after studies found chatbots were spreading misinformation about elections in Europe – a potential violation of a landmark new social media law that requires tech companies to implement safeguards against “negative effects on civic discourse and electoral processes”, under penalty of high fines. up to 6 percent of global turnover.
Google said it had been “limiting the types of election-related queries for which the Gemini app would return answers” since December, citing the need to exercise caution ahead of global elections.
Microsoft spokesperson Jeff Jones said “some election-related prompts may be redirected to search” as the company refines its chatbot ahead of November.
Jacob Glick, a senior policy adviser at the Institute for Constitutional Advocacy and Protection at Georgetown University who served on the House committee investigating Jan. 6, said tech companies need to be very careful when they provide inaccurate information.
“As disinformation around the 2024 elections intensifies, we want to be able to count on technology companies that are providers of information to provide without hesitation, even without fail, clear information on undisputed facts,” he said. -he declares. “The decisions these companies make are not neutral: they do not happen in a vacuum. »
Silicon Valley is increasingly tasked with sorting fact from fiction online as it builds AI-powered assistants. On Monday, Apple announced a partnership with OpenAI, bringing generative AI capabilities to millions of users to improve its voice assistant Siri. Meanwhile, Amazon is set to launch a new artificially intelligent version of its voice assistant as a subscription service in September, according to internal documents viewed by The Post. Amazon declined to confirm the launch date.
It’s unclear how AI-based Alexa will handle election queries. A prototype presented in September answered questions incorrectly on several occasions. Amazon has yet to launch the tool to the general public and the company has not responded to questions about how the new version of Alexa will handle political questions.
Amazon plans to launch the new product a year after the initial demonstration, but unpredictable response issues are raising questions internally about whether it will be ready, according to an employee who spoke on condition of anonymity to protect his job.
For example, an Amazon employee testing the new Alexa complained to the voice assistant about a problem he was having with another Amazon service, and Alexa responded by offering the employee a free month of Prime. Employees were unsure whether the AI was actually capable or authorized to do this, the employee told the Post.
Amazon said it is continuously testing the new Alexa AI and will set a high bar for its performance before launch.
Amazon and Apple have been slow to catch up with AI chatbots, given their initial dominance in the voice assistant market with Alexa and Siri. “Alexa AI was plagued by technical and bureaucratic problems,” former Amazon research scientist Mihail Eric said in an article on X on Wednesday.
Amazon’s devices division that built Alexa has struggled recently, losing its chief David Limp in August, an exit that was followed by layoffs. The team is now led by former Microsoft executive Panos Panay.
But the technology these devices are built on is a different, more scripted system than the generative AI that powers tools like ChatGPT, Gemini, and Copilot.
“It’s a totally different architecture,” said Grant Berry, a linguistics professor at Villanova University who worked on Alexa for Amazon.
Berry said voice assistants were designed to interpret human requests and respond with the correct action — think “Alexa, play music” or “Alexa, dim the lights.” In contrast, generative AI chatbots are designed to be conversational, social, and informative. According to Berry, turning the former into the latter is not a simple upgrade, but a rebuilding of the product’s interior.
When Amazon and Apple launch their new assistants, Berry said they will combine “goal-oriented” assistants with “socially-oriented” chatbots.
“When these things become unclear, there will be whole new issues that we will need to be aware of,” Berry said.