AI slop is everywhere. The real risk is what it does to user trust

There is a particular kind of post that feels inescapable online right now.

You’ve undoubtedly seen a version of it. A celebrity appears on the Met Gala red carpet in a gown they never wore. A baby is shown seeing their mother clearly for the first time after being given glasses. A strange product photo appears in a marketplace listing. A dramatic video spreads across a social feed. At first glance, it all looks plausible. Then something feels off. It might be something small, like a misshapen hand, a reflection, a texture, or gibberish text.

This is the internet’s new authenticity problem. AI-generated content is improving at speed, and at the same time it is multiplying, spreading across feeds, comment sections, marketplaces, creator platforms, dating apps and community spaces, just to name a few. A lot of it is harmless. Some of it is even funny and creative. Some of it is clearly low effort. But some of it is designed to mislead.

And increasingly, all of it is forcing platforms to answer a harder trust and safety question: how do you protect user trust when people can no longer tell what is real?

What is AI slop?

Merriam-Webster’s Word of the Year in 2025, “AI slop” has become a shorthand for the flood of low-quality, mass-produced AI-generated content filling the internet. It can be anything from an obviously fake image to a recycled motivational post, a spammy comment, a fabricated product review, a synthetic video, a bogus profile, or a piece of engagement bait engineered to provoke a reaction.

But the term can be too neat. The challenge for platforms isn’t simply that AI-generated content exists, but that it exists at such huge volume, across myriad formats, with different levels of quality and very different levels of risk.

“AI-generated content is pervasive and sometimes it’s very hard to tell what’s real,” says Alexandra Popken, VP of Trust and Safety and AI Services at WebPurify, an IntouchCX company. “Recently, I saw an AI-generated video of a baby being given glasses and seeing his mother for the first time. It was extremely realistic, but the giveaway was the hospital bracelet – when you zoomed in, the text was gibberish.”

That sophistication is what makes it so dangerous. The early internet trained people to spot obvious spam, but the AI era asks users to make much finer judgments. Is this image synthetic? Is this review real? Is this video misleading, or just edited? Is this post a joke, a scam, a political manipulation, or a piece of harmless AI creativity?

For trust and safety teams, those distinctions raise a conundrum. Not all AI-generated content should be removed. But not all of it can be ignored.

The engagement trap

One reason AI slop is so difficult to tackle is that it can perform well.

Confusing content is engaging content. When users suspect something might be AI-generated, they often do exactly what platforms are designed to encourage: they comment, argue, share, zoom in, debunk, defend and debate. “I see comment sections on AI-generated content filled with users asking, ‘Is this AI?’” says Popken. “That alone is engagement, and it ultimately boosts viewership. So in some cases, platforms may be incentivised to keep the content up.”

This creates an uncomfortable incentive problem. If a synthetic image or video drives comments, watch time and shares, the platform may benefit from the very ambiguity that weakens user trust. A post doesn’t need to be true to be sticky. It only needs to make people react.

This is where AI slop becomes more than a content quality issue and starts creeping into discussions about product and platform design.

A platform that optimizes heavily for engagement may end up rewarding content that confuses users. A recommender system may distribute synthetic media because people interact with it, not because it is trustworthy. Moderation teams may then be left managing the downstream effects of incentives they did not create.

In the short term, this can look like growth. In the long term, it can corrode the user experience. People may keep scrolling, but they trust less of what they see.

Why AI slop is hard to moderate

The challenge for content moderators is that AI slop often sits in the gray zone.

Some synthetic content is misleading but not directly harmful. Some is low quality but not policy-violating. Some is high quality and benign. Some becomes harmful only because of context, such as a fake image attached to a breaking news event, an AI-generated product photo in a marketplace listing, a synthetic profile on a dating app, or a manipulated audio clip of a public figure.

As Popken explains, “Trust and safety teams are grappling with what to keep up versus take down. Much of this content, while misleading, isn’t necessarily harmful. Some of it is low quality, but some of it is really high quality. So where do you draw the line?”

That line cannot be drawn by technology alone.

Synthetic media detection is improving, and many platforms are investing in models like WebPurify’s that can identify manipulated or AI-generated content. But detection still has limits. AI systems can miss content, misclassify authentic media, or struggle with edge cases where context matters more than pixels or metadata.

Popken explains that platforms need layered systems. Automated tools can help detect and classify content at scale, while human moderators can assess context, intent, likelihood of harm and policy fit. At the same time, quality assurance teams can test whether enforcement is consistent and policy specialists can decide when AI-generated content should be removed, labeled, downranked or allowed.

The goal shouldn’t be to remove every piece of AI-generated content, but to manage the risks it creates.

The bigger risk: users stop trusting everything

The obvious fear is that users will believe fake content. But there is another risk that may be just as damaging: users may stop believing real content.

“If people stop trusting everything online, whether that’s images, audio, comments or video, even ordinary interactions start to create a different kind of societal problem,” says Ailís Daly, Head of Trust and Safety, EMEA, at WebPurify.

Daly also warns that platforms don’t want to create a world where “everything inconvenient gets dismissed as AI-generated or fake.”

That is the deeper trust problem. When synthetic content becomes common enough, users become suspicious of everything. Real creators are accused of being fake. Authentic footage is dismissed as AI. Genuine users are labeled bots. Communities become more cynical, more hostile and less willing to believe what they see.

And this hits platforms both commercially and socially.

On marketplaces, authenticity affects purchasing decisions. On dating apps, it shapes whether users feel safe engaging with other people. On creator platforms, it affects whether audiences trust the people they follow. On social platforms, it influences whether users believe images, videos, comments or accounts. On review sites, it can undermine the integrity of the entire product experience.

Once users feel that a platform is full of synthetic, deceptive or low-quality content, the damage is difficult to reverse. The issue is no longer one fake image or misleading post, but a broader feeling that the platform cannot be trusted to manage its environment.

Labels may be enough, but only if they work

A realistic response to AI slop can’t be “take everything down.” AI-generated content is now part of online life, and much of it will be allowed under platform policies.

In many cases, labeling may be the right intervention. “When the content doesn’t violate policy, a label is often enough,” says Popken. “It signals that what you’re viewing isn’t necessarily harmful or bad, but it isn’t human-created.”

That is a sensible middle ground. Platforms can remove content that violates policy, such as scams, impersonation, harassment, NCII, fraud or dangerous misinformation. They can label synthetic or manipulated media that may otherwise mislead users. They can downrank repetitive, low-quality or engagement-bait AI content. They can escalate high-risk or ambiguous cases for human review. And they can educate users so labels are meaningful rather than cosmetic.

But labels only protect trust if they are accurate, visible and consistent. A label that appears on some synthetic content but not on similar posts elsewhere may create more confusion. A vague label that users do not understand may do little to change behavior. A detection system that misses too much content may give users false confidence.

For labels to work, they need to be part of a wider authenticity strategy.

Authenticity is now a trust and safety workflow

Platforms need to stop treating AI slop as a weird content trend and start treating authenticity as an operational trust and safety challenge.

That means building clear policies for AI-generated and manipulated media and deciding when disclosure is required, when content should be labeled, when it should be removed, and when it should be limited in distribution. It also means investing in detection systems across images, video, audio and text, and using human review for context-heavy decisions. And moderation teams must create escalation paths for impersonation, fraud, harassment, synthetic intimacy abuse, and real-world harm.

Lastly, it means recognizing that AI slop will look different depending on the platform.

There is no one-size-fits-all answer, but there is a clear direction of travel: platforms need moderation systems that can understand not just whether content is allowed, but whether users are being misled about what they are seeing.

AI can help with that work at scale, but human expertise remains essential for the decisions that require context, judgment and an understanding of how harm actually unfolds inside online communities.

AI-generated content isn’t going away. So the question is whether platforms can give users enough confidence to navigate it.

The future of online trust may depend less on whether users can spot every fake, and more on whether platforms can show they are taking authenticity seriously.