The Scoop: Strange AI spam is being weaponized on Facebook

If you’ve spent any amount of time on Facebook recently, you’ve surely seen the bizarre AI-generated images popping up in your feed, likely from algorithmically suggested pages you’ve never opted into liking.

Some of the images are obviously fake, such as “Shrimp Jesus.” Yet others are more subtle, such as an AI-generated image that purports to be a historical image – perhaps even using a real caption as its prompt in a bid to avoid copyright flags.

But what is the end game of these images and the pages that spam them, racking up millions of views and confusing your mom?

According to CNN, there are varying motives. Some are just looking for Facebook bonus payments, raking in thousands of dollars each month by goosing engagement from the easily fooled or easily outraged. But others might be more nefarious, seeking to gather user data or even slip in mis- and disinformation amid the bad AI fakes. Bad actors can avoid deeper scrutiny by peppering in the occasional more politically motivated meme or deepfake.

Some pages might even amass huge followings by posting innocuous content, only to later change the name and posting style to something politically motivated – thus using their huge fanbase to push a political agenda to an audience who never saw it coming.

Why it matters: In addition to the obvious destabilizing effects on democracy caused by courting audiences with AI slop, this raises several concerns for good-faith social media managers.

First, this is your competition. Bizarre and salacious images that are presented as real are capturing attention while authentically crafted content that’s honest about what it is and how it’s made struggles to gain traction. It’s an uphill climb. It also might mean audiences are more skeptical of your own content, even when it’s real and fully vetted. Credulity and suspicion are at war, and both can hurt your brand.

Meta says it’s attempting to police this content, including adding “AI Info” that identifies artificial content – but it’s proving easy for bad actors to evade, leaving users to count fingers and look for blurring around the edges to identify the real from the fake.

The best thing you can do is maintain scrupulous honesty and transparency about your own page, its purpose and your use of AI. It’s old-fashioned and may not get you millions of views right off the bat, but it’s the only way for ethical marketers to move forward.

Editor’s Top Reads:

• Over the holiday weekend, TikTok users claimed they’d discovered an “infinite money” glitch from Chase Bank, allowing them to withdraw money from their accounts they didn’t actually have. Yeah, it turns out they were engaging in a digital version of check kiting. Which is a crime. “We are aware of this incident, and it has been addressed,” Chase wrote in a statement to The Guardian. “Regardless of what you see online, depositing a fraudulent check and withdrawing the funds from your account is fraud, plain and simple.” This is yet another example of how misinformation can spread online – no AI required. Whether the first “discoverers” of this were maliciously trying to trick others into committing a crime or simply idiots, we don’t know. But Chase responded clearly and with no room for ambiguity – on a holiday weekend, no less. Kudos on strong social listening and a decisive response to a ridiculous situation.
• The Honey Deuce is taking over the U.S. Open. The drink, which combines vodka, raspberry liqueur and lemonade, topped with three tennis ball-esque melon balls, has become a viral sensation. It’s expected to earn more than $10 million in sales this year, retailing at $23 a pop. It’s even earned the TikTok approval of Serena Williams, who was able to try the drink for the first time since she wasn’t competing this year. The drink’s quirky presentation and connection with the event allows it to break through even to those who aren’t (yet) interested in tennis and drum up even more positive PR for the event, gaining headlines in news sources across the country. It’s a clever example of a side door into an event, boosting interest among new audiences – and potentially making them raving fans.
• Raygun, real name Rachel Gunn, shot to infamy during the Paris Olympics for her … unique breakdancing performance The Australian earned zero points across her rounds of competition, coming in dead last. But she did become a viral meme for her moves – and gained widespread condemnation for making a mockery of breakdancing. Gunn is now on an apology tour, speaking on an Australian television program about the experience. “It is really sad to hear those criticisms, and I am very sorry for the backlash that the community has experienced, but I can’t control how people react,” she said. Paris marked the first – and perhaps only – showing of breakdancing as an Olympics sport. Gunn’s performance overshadowed all others and she became the face of the sport, for better and for worse. Apologizing is a good step, but how can Gunn boost other breakers and use her new fame – and her role as a lecturer at Macquarie University — to draw attention to the sport in a positive way?

Allison Carter is editor-in-chief of PR Daily. Follow her on Twitter or LinkedIn.