8 questions every platform should be asking about Trust & Safety right now

Trust & Safety teams are under pressure from every direction. Synthetic content is getting better. Harmful behavior is getting harder to spot early. Youth safety is climbing the policy agenda. Regulators want more accountability. Users want clearer decisions. And platforms are being asked to make all of this work at speed, at scale, and often with limited room for error.

That means the next phase of Trust & Safety is unlikely to be defined by a single new threat. It will be defined by a harder operating environment, where online harms are more ambiguous, more scalable, and more deeply connected to product design, policy, and business risk.

Across WebPurify’s two latest ebooks, one theme that stands out is that the platforms best positioned for what comes next will be the ones asking sharper questions now. Not necessarily just what content to remove, but questions about content authenticity, prevention, explainability, resilience, and what safe growth actually looks like in practice.

Here are 8 questions every platform should be asking right now.

1. Can we still tell what’s real online – and does our enforcement model reflect that?

For years, moderation teams could rely on at least some visible signals to help distinguish credible content from misleading or manipulated material, but that line is getting harder to hold.

As Alexandra Popken, SVP of Trust & Safety and AI Services, explains, “While a lot of AI content is noisy and low effort, there’s also a growing wave of high-quality, convincing AI output. In some cases, it’s genuinely hard to tell what’s real versus fake.”

That challenge goes well beyond a handful of obvious deepfakes. Platforms are now dealing with synthetic reviews, AI-generated personas, mass-produced misinformation, and low-value “AI slop” that can still shape user perception at scale. The question is no longer just whether a piece of content is technically false, but whether existing moderation systems are equipped for an environment in which authenticity itself is under pressure.

And the damage does not stop with the people who believe the misleading content. Popken also warns that there is “a pervasive harm if people start distrusting everything they see online.” That is the deeper risk: not only that bad actors can manufacture credibility, but that the wider information environment becomes less believable for everyone.

2. Are we prepared for harmful content that scales faster than human teams can respond?

The next challenge is volume. Even if platforms can identify emerging forms of abuse, they still have to deal with the fact that harmful content now appears faster, in more formats, and at a scale human moderation teams cannot manage on their own.

Pratyoosh Ranjan Maharana, Senior Risk Manager at Amazon, argues that the biggest shift this year is “how organizations are moving from reactive content moderation to AI-powered systems that can actually anticipate and prevent problems before they escalate.” He points to “the sheer volume of AI-generated content flooding platforms” and “the reality that human moderators simply can’t keep pace with the scale of today’s digital interactions.”

That does not mean human expertise is becoming less important. It means that the old model of relying primarily on people to catch problems after they surface is becoming less viable by the month. Platforms need to ask whether their detection, triage, and escalation workflows are built for the speed of the current threat environment, not the one they were managing two years ago.

3. Are we spotting weak signals early enough, or only reacting once a problem becomes visible?

Many of the biggest Trust & Safety failures do not begin as obvious crises. They start as edge cases, unusual patterns, workarounds, fringe communities, or small behavioral shifts that are easy to dismiss until they are not.

That is why Emer Cassidy, Head of Trust & Safety at Zalando, argues that teams need to “move beyond ‘searching’ to ‘scanning’ for weak signals.” In her view, that means identifying emerging AI-generated harms and regulatory shifts before they reach an inflection point.

This is a useful way to think about preparedness. Too often, platforms wait until a problem is visible enough to trigger widespread complaints and media coverage, or even executive attention. By then, the costs are higher and the room for thoughtful intervention is smaller. A more mature Trust & Safety function asks what is changing beneath the surface and what small signals might point to a bigger threat taking shape.

4. Are our policies built to evolve, or are they too static for the threats ahead?

Even the best moderation teams will struggle if their policies are too rigid to keep up with fast-moving harms. New scam formats, new styles of manipulated content, and new patterns of abuse can spread far faster than traditional policy review cycles.

Cassidy sees a different model emerging. “We’ll be moving away from static, 50-page community guidelines to ‘Living Policies’ that can be updated in real-time via AI prompts to address sudden viral threats like novel scam formats,” she says.

That idea matters because online harms do not stand still. A policy framework that is reviewed occasionally and interpreted unevenly across teams is unlikely to be enough moving forward. Platforms should be asking whether their policy model is adaptive, whether updates can happen quickly, and whether moderation, product, and policy teams are working from the same current understanding of risk.

Looking for practical next steps? 12 Actions to Stay Ahead of Emerging Trust & Safety Risks in 2026 brings together expert recommendations on how platforms can respond to fast-changing harms, strengthen moderation strategies, and build safer systems.

5. Do users understand why decisions are being made on the platform?

We often focus on how fast a platform removes the right content, but trust in enforcement also depends on whether users can challenge and make sense of those decisions.

As Cassidy explains, platforms should be investing in “Explainable AI” so that when an automated system removes content, the reason is transparent. That expectation is likely to grow as automation becomes more deeply embedded in moderation workflows.

Opaque enforcement creates its own problems. Users may feel they have been treated unfairly. Regulators may see weak governance. Internal teams may struggle to defend decisions consistently. Platforms should be asking whether their moderation systems are merely operational, or whether they are also legible. Clearer decision-making, clearer appeals, and clearer communication are becoming part of the safety model itself.

6. Are we over-relying on AI, or under-using it?

The AI debate in Trust & Safety is often framed too simply. One side treats automation as the answer to scale. The other treats it with so much caution that opportunities for smarter detection and escalation are missed.

Laura Higgins, Senior Director of Community Safety & Civility at Roblox, offers a more balanced view. “While AI is critical for speed and scale of detection, we’ve seen a trend of over-relying on the tech as the sole answer to safety issues,” she says. “These challenges are far too complex for it to ever be a matter of ‘either, or.’” More importantly, she adds, “There is an increasing realization that human expertise is vital to training the tech, overseeing its implementation, and feeding back insights to help us stay ahead of fast-changing online trends.”

That is probably the more useful question for platforms now. Not whether to use AI, but where it works best, where it needs oversight, and how to design workflows that combine automation with context, judgment, and continual learning.

7. Are Trust & Safety, product, policy, and operations actually working from the same playbook?

Many online safety failures are not caused by one bad decision. They happen because the relevant teams are working from different assumptions, different timelines, or different definitions of acceptable risk.

Sophie Walsh, Director of Trust & Safety at Depop, argues that Trust & Safety teams will increasingly be expected to “anticipate harm before it materializes.” That means “embedding safety considerations earlier in product design, model development, and policy decisions.”

This is not just a question of better intentions. It is a question of organizational readiness. If policy teams are updating rules in one lane, product teams are shipping in another, and operations teams are left to absorb the consequences later, platforms will stay trapped in reactive mode. The stronger model is one in which safety signals move across functions early enough to influence design, rollout, and governance before a problem becomes expensive.

8. Are we measuring the right things or just the easiest ones?

Trust & Safety teams have always had access to metrics like volume reviewed, actions taken, queues cleared, turnaround times met. The harder question teams now face is whether those are the metrics that matter most.

Walsh makes the challenge clear when she says platforms should be “measuring success by harm reduction, not just enforcement volume.” That shift sounds simple, but it asks a lot more of organizations. Harm prevented is harder to quantify than content removed. Trust preserved is harder to chart than moderation throughput.

Still, this is where the field is heading. Platforms should be asking whether they are measuring quality of intervention, repeat harm, appeals outcomes, user confidence, and long-term resilience, rather than relying too heavily on outputs that are convenient to count but incomplete as indicators of success.