- Written by Marianna Spring
- BBC disinformation and social media correspondent
London Mayor Sadiq Khan said deepfake audio of him allegedly making inflammatory remarks ahead of Armistice Day came close to causing “serious disruption”.
Mr Khan said the law was “not fit for purpose” in tackling AI fakery because audio creators “got away with it”.
“We all know that’s what Sadiq is thinking,” said the man who first posted the video, tracked by the BBC.
However, another social media user who helped spread the audio apologized, saying he had “made a big mistake.”
The video uses artificial intelligence (AI) to create a replica of Khan’s voice saying the words scripted by the hoaxer, insulting Remembrance Weekend with expletives and insulting the remembrance weekend scheduled for the same day last November. He called for prioritizing pro-Palestinian marches.
Intended to sound like a secret recording, it read: “The important and most important thing is that one million Palestinians will march on Saturday.”
The video imitates comments made by Khan, the first Muslim mayor of London, who said: “I lead the Metropolitan Police. They will do as the Mayor of London tells them to do.” We need to figure it out.” The newspaper said the Prime Minister’s meeting with the Metropolitan Police Commissioner Sir Mark Rowley was “a waste of time” as it was “not his responsibility”.
The video went viral, including in far-right groups, and prompted a surge of hateful comments against the mayor on social media.
“When my friends and family see something like this, it’s very upsetting. I mean, I have two daughters, I have a wife, I have a brother. I have a mother.” said Mr. Mr Khan told the BBC.
The AI fake emerged amid an already tense political debate, with Chancellor Rishi Sunak describing a pro-Palestinian march held elsewhere in central London on Armistice Day as “disrespectful”. Then-Interior Secretary Suela Braverman called for those plans to be halted.
Listen to fake recording of London Mayor Sadiq Khan
Armistice Day commemorates the moment World War I ended on November 11, 1918, with commemorations taking place at Cenotaphs and across the country on the nearest Sunday, known as Remembrance Sunday.
“If you’re looking to sow disharmony and cause trouble, the timing couldn’t be better,” the Mayor of London told the BBC.
“What was being said was, for the far right and others, to put a red cloth on the bull. But my biggest concern was, are you an innocent listener of this? “Because this is a secret undercover recording, written in inverted commas, and it sounds like it.” Just like me, because of timing and context. ”
Both events took place on Saturday, November 11, with the pro-Palestinian march beginning several hours after a two-minute silence. However, counter-protesters, including some affiliated with far-right groups, were later denounced by police after the clashes and arrested on charges including inciting racial hatred.
Marianna Spring, one of the BBC’s most harassed journalists, dives into your inbox to investigate an extraordinary case of online hate. She meets the people at the center of these conflicts and in some cases unites them to see if understanding and even forgiveness is possible.
Khan said AI fakery further fueled the situation. “We almost had serious disruption,” he said, adding that deepfake audio could have alarming implications for other situations, such as close elections, referendums, and community unrest.
“People should be able to criticize me. But I think what they shouldn’t do is use AI to fabricate lies. And the person who created this, I don’t know what the consequences will be that Saturday. I think we need to recognize that.”
One of the larger accounts that spread the fake audio is Little Boats on X, which shares anti-Muslim and anti-immigrant content and has more than 58,000 followers.
The account’s director, Jeremy Davis, said he still suspected Khan held similar beliefs to those in the audio, but apologized for sharing the fake one.
“This was a smart AI job,” he said. “Mayor Khan, I’m sorry, I made a big mistake.”
But Davis didn’t create the audio.
Through screen captures, I traced the recording back to TikTok and the person who appears to have originated the clip, an account called HJB News with the ironic tagline “Keep it real.” This account shares anti-immigrant and racist content.
Counter-protesters clash with police on Armistice Day
HJB News accounts on other social media show they shared the audio on November 9, hours before it went viral. The fact that there is no trace of the audio elsewhere online before HJB News shared it makes it likely that HJB News also created this audio.
After exchanging a few messages, I received a call from the account representative. He only gave his name as Henry and did not allow me to record the conversation, but I took detailed notes of the conversation.
He claimed it was a “TikTok clip that I shared,” and when I pointed out that he had shared it first, he denied creating it, saying, “I don’t want to comment.”
He said his account was not just publishing “fake clips.” “We will publish news that may be true with humor.”
On November 11, the Metropolitan Police Department announced that it had investigated the audio and found that it did not constitute a crime. HJB News told X, “Well, I’m relieved,” with an emoji of a smiling face with beads of sweat rolling down.
Khan said the fake audio was not a “bit of fun” or “satire” and that its creators were not “naive” about its potential consequences.
But the Mayor of London said he accepted the apologies of those who shared the article, citing an “innocent mistake”.
Mr Khan said bodies such as the Electoral Commission, which is responsible for keeping UK elections “free and fair”, also needed more powers to deal with fabricated information.
There is currently no criminal law in the UK that specifically covers this type of scenario.
The mayor said he was also “very concerned” that the social media company did not contact the mayor or authorities when the fake audio went viral.
I contacted X, TikTok and Instagram and the clip was shared. X and Instagram did not respond. A TikTok spokesperson said the social media site “does not allow synthetic media containing the likeness of real individuals” and removes this type of content.
TikTok said it spoke with both the Mayor’s Office and the Metropolitan Police in November 2023 about the platform’s approach to this content, and warned that similar issues could be raised directly in the future.
