Online Harassment: Who Does it Affect, and How?

November 7, 2018 | Image Moderation, Profanity Filter, Video Moderation, UGC

In our September newsletter, we discussed the Momo Challenge, a disturbing “game” circulating on social media sites, threatening members with the release of personal information unless the member completes a series of tasks, which often encourage self-harm, and even suicide.

Today, we’re taking a closer look at online harassment and cyberbullying.

A recent Pew Research Center survey of 743 teens and 1,058 parents in the U.S. found that 59% of U.S. teens have personally been subjected to “abusive online behavior.” They categorize the types of harassment as:

  1. Offensive name-calling
  2. Spreading of false rumors
  3. Receiving explicit images they didn’t ask for
  4. Constant asking of where they are, what they’re doing, who they’re with, by someone other than a parent
  5. Physical threats
  6. Having explicit images of them shared without their consent

An earlier study of adult internet users found that 73% have witnessed harassment online and 40% have personally experienced it. 66% of those users said the harassment occurred on a social networking site or app, while 22% mentioned the comments section of a website.

In addition, a UCLA psychology study found bystanders are less likely to intervene with online bullying than they are in real life situations. The authors note it’s likely because “bystanders do not see the anguish of victims of online bullying.”

Consider two online phenomena targeting teens within the last few years:

 
Blue Whale Challenge
Beginning in mid-2016, a “game” reportedly began circulating on social media sites. It consisted of a series of tasks gradually requiring self-harm and finally telling the player to commit suicide. While no suicides have been definitively tied to the game, experts believe that it has likely led to instances of self-harming and copycat groups.

Momo Challenge
Similar to the Blue Whale Challenge, this latest “game” challenges participants to communicate with unknown numbers, often via WhatsApp. Members are sent disturbing images and threats if they refuse to comply with tasks that require increasing self-harm and encourage suicide. Law enforcement officials in Argentina are investigating whether the Momo Challenge motivated a 12-year-old girl and a 16-year-old boy to take their own lives.

(Note: WebPurify is already rejecting a specific image often used as an avatar for accounts associated with Momo. Administrators of sites that cater to children and teens should be aware, and encourage young users to avoid responding to and block accounts related to the challenge).

Cyberbulling and online harassment don’t only affect teens, and their consequences don’t always stay in the online realm:

 
Murder of Robert Godwin, Sr.
Robert Godwin Sr. was shot and killed in April of 2017 by a stranger who recorded the shooting and posted the video to Facebook. The video was publicly available for more than two hours before it was removed by Facebook. Early the following month, Mark Zuckerburg said the company was hiring additional staff for its “global community operations” team to screen Facebook Live content.

Kurt Eichenwald and the strobing GIF
In 2017, a Dallas grand jury indicted John Rayne Rivello, who was charged with aggravated assault with a deadly weapon after tweeting a GIF animation designed to provoke an epileptic seizure to journalist Kurt Eichenwald. Eichenwald, who has epilepsy, said the tweet Rivello sent him in December caused him to have a seizure. Rivello faces 10 years in prison. Eichenwald said that dozens of copycat images where later sent to him.

How does online harassment affect users? How much responsibility do sites and apps hold for harassment that happens on their platforms?

 
According to NCBI, a division of the National Institutes of Health, most of the research on cyberbullying assesses how much and how badly victims are harassed, rather than the potential long term psychological consequences. It’s a relatively new topic, as far as research methods go.

However, The American Academy of Pediatrics says cyberbullying causes psychological and physical effects. Bullies and victims alike experience anxiety, depression, increased insomnia, abdominal pain, substance abuse, and suicidal thoughts at higher rates than those who perpetrate or experience real-life bullying. The AAP recommends that health care providers specifically ask patients about their online life and if they are being harassed.

Unfortunately, all sites and apps with a social component, or any user-generated content, should expect their platform to be used for harassment. Rather than waiting for it to happen and then reacting, they should put safeguards and policies in place as soon as possible. Implementing solutions before there’s a problem not only protects users, but also helps companies avoid a reputation as an unsafe site and/or a public relations crisis.

Turnkey Image Moderation Criteria
Reject images that:
Turnkey Video Moderation Criteria
Reject videos that:
Other Language Label: If a video complies with our moderation criteria but contains languages other than English, we will label it as “Other Language.”