Unsung heroes: Moderators on the front lines of internet safety

What, one might ask, does a content moderator do, exactly? To answer that question, let’s start at the beginning.

What is content moderation?

Although the term moderation is often misconstrued, its central goal is clear—to evaluate user-generated content for its potential to harm others. When it comes to content, moderation is the act of preventing extreme or malicious behaviors, such as offensive language, exposure to graphic images or videos, and user fraud or exploitation.

There are six types of content moderation:

1. No moderation: No content oversight or intervention, where bad actors may inflict harm on others
2. Pre-moderation: Content is screened before it goes live based on predetermined guidelines
3. Post-moderation: Content is screened after it goes live and removed if deemed inappropriate
4. Reactive moderation: Content is only screened if other users report it
5. Automated moderation: Content is proactively filtered and removed using AI-powered automation
6. Distributed moderation: Inappropriate content is removed based on votes from multiple community members

Why is content moderation important to companies?

Malicious and illegal behaviors, perpetrated by bad actors, put companies at significant risk in the following ways:

• Losing credibility and brand reputation
• Exposing vulnerable audiences, like children, to harmful content
• Failing to protect customers from fraudulent activity
• Losing customers to competitors who can offer safer experiences
• Allowing fake or imposter account

The critical importance of content moderation, though, goes well beyond safeguarding businesses. Managing and removing sensitive and egregious content is important for every age group.

As many third-party trust and safety service experts can attest, it takes a multi-pronged approach to mitigate the broadest range of risks. Content moderators must use both preventative and proactive measures to maximize user safety and protect brand trust. In today’s highly politically and socially charged online environment, taking a wait-and-watch “no moderation” approach is no longer an option.

“The virtue of justice consists in moderation, as regulated by wisdom.” — Aristotle

Why are human content moderators so critical?

Many types of content moderation involve human intervention at some point.  However, reactive moderation and distributed moderation are not ideal approaches, because the harmful content is not addressed until after it has been exposed to users. Post-moderation offers an alternative approach, where AI-powered algorithms monitor content for specific risk factors and then alert a human moderator to verify whether certain posts, images, or videos are in fact harmful and should be removed. With machine learning, the accuracy of these algorithms does improve over time.

Although it would be ideal to eliminate the need for human content moderators, given the nature of content they’re exposed to (including child sexual abuse material, graphic violence, and other harmful online behavior), it’s unlikely that this will ever be possible. Human understanding, comprehension, interpretation, and empathy simply can’t be replicated through artificial means. These human qualities are essential for maintaining integrity and authenticity in communication. In fact, 90% of consumers say authenticity is important when deciding which brands they like and support (up from 86% in 2017).