Why content authenticity is becoming a frontline Trust & Safety problem

For years, questions of content authenticity, for the most part, sat at the edges of Trust & Safety. Deepfakes were a novelty. Synthetic images were rare enough to treat as curiosities rather than core risks. But as that technology has accelerated in recent years, that era is over.

Today, Trust & Safety teams are dealing with a very different reality: an internet saturated with AI‑generated content, much of it low quality and forgettable, some of it highly convincing, and an increasing share of it designed to deceive. The challenge now is no longer simply identifying what content is fake. It’s deciding what actually causes harm and what to do about it.

As Alexandra Popken, SVP of Trust & Safety and AI Services at WebPurify, an IntouchCX company, explains, “We’re already drowning in low‑quality, mass‑produced AI content – what people are starting to call ‘AI slop’ – and it’s everywhere.” Open TikTok, Instagram, or Facebook, and it’s hard to miss. Some forecasts suggest that close to 90% of online content could soon be AI‑generated. On its own, that volume is staggering.

But volume isn’t the real problem. “The harder part,” Popken says, “is that alongside all that noise, there’s a growing wave of high‑quality, highly convincing AI content. In some cases, it’s genuinely difficult to tell what’s real and what isn’t.” That shift from obvious fakery to plausible deception is what has pulled authenticity squarely into the frontline of Trust & Safety work.

Fake doesn’t automatically mean harmful

One of the most persistent misconceptions about AI‑generated content is that being fake automatically makes it dangerous. In practice, the opposite is often true.

Popken points to viral moments like the now‑infamous image of Pope Francis in a puffer jacket. “It was completely fabricated and extremely believable,” she says. “But the real question isn’t just whether something is fake. It’s whether it is dangerous. Should it be removed? Does it cause real‑world harm?”

Those questions sit at the heart of modern Trust & Safety. “These are the gray zones Trust & Safety teams are going to be navigating for a long time,” Popken adds. The work isn’t binary, and it isn’t solved by detection alone. It requires judgement, context, and a nuanced understanding of how content lands with different audiences.

That’s why content authenticity has become such a difficult operational problem. Identifying synthetic media is one thing, but deciding whether it crosses a harm threshold – and justifying that decision internally and externally – is another entirely.

Why today’s tools only go so far

Platforms aren’t ignoring the problem. Labels, watermarks, and provenance tools are already being tested and deployed. But Popken is clear‑eyed about their limitations.

“We’re already seeing platforms introduce AI labels,” she says. “And on the technical side, there’s experimentation with watermarking and cryptographic metadata to signal authenticity and provenance.” In controlled environments, those measures can be helpful.

But the problem is scale. Much of this tooling relies on self‑reporting or only works within a single ecosystem. “We know from age verification that self‑reporting isn’t an effective mechanism,” Popken says. And tools that can identify whether content was generated by one specific model quickly fall apart once content moves across platforms.

In other words, these systems often scratch the surface without addressing the full risk. As Popken explains, they “only work in very narrow cases.”

Authenticity, fraud, and the erosion of trust

For Daly, the consequences of synthetic content aren’t abstract. “We’re already seeing AI‑generated voices and images being used to trick people,” she says. “That’s not theoretical. That’s happening now.”

Fraud and impersonation are where authenticity failures quickly become safety failures. Synthetic media makes it easier to exploit trust at scale, whether that’s through scam calls, fake endorsements, or impersonation of public figures and loved ones.

Daly also worries about what happens when trust erodes in both directions. “If people believe everything they see online, they’re vulnerable,” she says. “But if they start believing nothing, that’s a problem too.” Platforms that can’t clearly communicate what they stand behind will ultimately risk leaving their users in a permanent state of uncertainty.

That tension between over‑trust and total distrust is becoming one of the defining challenges for Trust & Safety leaders.

Why this is a leadership problem, not just a tooling one

As authenticity decisions become more consequential, they’re also becoming more visible. Popken notes that platforms are increasingly expected to explain not just what they enforce, but why. “If you can’t explain why a decision was made,” she argues, “you can’t defend it to regulators, advertisers, or users.”

That expectation pushes content authenticity out of the realm of niche moderation issues and into core platform governance. Decisions about synthetic content affect brand trust, regulatory exposure, and user confidence all at once.

Daly sees this as part of a broader shift in how Trust & Safety is understood. When authenticity failures intersect with fraud, abuse, and misinformation, the cost of getting it wrong is no longer contained. It spills into reputational damage and legal scrutiny, not to mention real‑world harm.

Judgement is the new differentiator

The hardest part of grappling with content authenticity isn’t building better detectors. It’s deciding what matters.

As Popken has warned, “Mark my words: this is going to be a major focus for Trust & Safety teams.” Not because platforms lack tools, but because authenticity forces so many uncomfortable questions about responsibility and accountability.

In an internet now shaped by AI, Trust & Safety teams won’t be judged on whether they can spot what’s synthetic. They’ll be judged on whether they can make principled decisions about what comes down and how those choices are explained to the people most affected by them.

Content authenticity isn’t a future problem. It’s already here, and it’s redefining what Trust & Safety work looks like at the front lines.