Last Updated on 13th October 2021

We’ve all had that wow moment, when something we thought possible only in the imagination of sci-fi writers suddenly exists at our fingertips! Just when we think we have seen everything technology has to offer, something new comes along.

These moments can stop us in our tracks and make us wonder whatever is next. At INEQE Safeguarding Group, we have seen a lot of new apps and platforms utilising deepfakes. The big question is: should you be worried?

You may have seen (and laughed) at some funny videos that use deepfake technology. For example, videos in which celebrities, friends, or relatives have had their faces manipulated to sing, dance, or even tell jokes. When deepfakes are used in this way, they are unlikely to cause serious harm.

However, there is a much more worrying and disturbing side to how deepfakes are being used online. This article brings you the latest research on deepfakes and explains how they can be misused. Read on to find out how you can protect the children in your care as this technology continues to evolve.

What are Deepfakes?

A deepfake is an image, video, sound, voice, or GIF which has been manipulated by a computer to superimpose someone’s face, body, or voice onto something else.

Deepfakes have become popular due to the accessibility of mobile and computer-based apps. This means users without sophisticated technological skills can easily access, create, and distribute deepfakes.

Deepfakes can be produced using computer and mobile apps to upload images from a user’s camera roll. The quality of deepfakes can vary depending on the sophistication of the tech used and the skills of the creator. However, the standard of the deepfake produced will rarely matter compared to the potential harm it may cause when used to harass, bully, or abuse a victim.

A Focus on the Facts

  • In 2019, AI company Sensity warned that 96% of deepfakes they surveyed online were non-consensual ‘pornographic’ materials.

  • Of the above deepfakes, 90% were targeted at women, making women overwhelming majority in victims of sexualised deepfake tech.

  • In the past, deepfakes were often used to create pornographic materials of famous women. With the tech becoming more accessible to everyday users, non-famous people are increasingly becoming victims too.

  • Sensity registered that 124 million deepfake ‘pornographic’ videos were available on the top four pornography websites.

  • Last year, Sensity also warned that automated bots were being used to ‘strip’ the clothes from publicly available photos found on social media. 104,852 women were targeted.  70% of targets were private individuals whose photos had been harvested from social media. 

  • The Law Commission is currently reviewing legislation into image-based abuse, which includes the sharing of ‘alerted images’ including deepfakes.

©commons.wikimedia.org/wiki/File:Sw-face-swap

 

The Harmful Side of Deepfakes

There are already several ways deepfakes are being misused to cause distress and harm. As this technology evolves and becomes more accessible to users, it is important to understand how it can be misused.

Bullying

Deepfakes have been used in cases of cyberbullying to deliberately mock, taunt, or inflict public embarrassment on victims. The novel appearance of these images may distract from the real issue that they can be used to bully or harass children and young people.

Extortion and Exploitation

Deepfakes can be used to create incriminating, embarrassing, or suggestive material. Some deepfakes are so good that it becomes difficult to distinguish between them and the real thing. Convincing other people that an embarrassing or abusive image is fake can create additional layers of vulnerability and distress. These images can then be used to extort money or additional ‘real’ images.

Additionally, deepfakes can be used to create so-called ‘revenge porn’, which is a form of image-based sexual abuse as retaliation or vengeance typically associated with the end of a relationship or not agreeing to a sexual relationship with the perpetrator.

There is also the potential for deepfakes to be used as a form of homophobic abuse, in which a person is depicted in gay pornography. This could then be used to ‘out’ the person or as an attempt to ‘destroy their reputation.’ For young people struggling with their sexual orientation, being depicted in any sexualised deepfakes may be particularly distressing.

Image-Based Sexual Abuse

There have been cases where images of children have been harvested and used to generate sexualised deepfakes. The realistic depiction of a victim engaging in a sex act can damage a child’s wellbeing and mental health. We know that deepfake software can be used to remove clothing from victims digitally. In some cases, there are commercial services where users can pay to have images professionally manipulated.

It is important that parents, carers, and safeguarding professionals are aware of the risks of this form of (non-contact) sexual abuse. In some cases, victims themselves may be unaware that their images have been harvested and misused to create deepfakes.

While many young people may be aware of and understand how images can be manipulated in this way, others may not. It is important to speak to them about the issue of deepfakes and how they can be misused.

How to spot a Deepfake  

Deepfakes can vary in their quality and professionalism. Some will be quite obviously fake. For others, it can be tricky to spot whether they are real or not. Here are some tell-tale signs to look out for:

  • Glitches – there are typically signs if you look closely at the video itself. Is there rippling or blurring around key facial features, like the neck, eyes, or mouth? This may become more obvious when a person moves, blinks, or turns their head or body.

  • Audio – there may be an indication that lip movements do not match what you are hearing. Look closely for natural mouth movements.

  • Blurring – are there any key parts of the video or image that are blurred or missing definition? You can usually detect this in features like teeth, hair, or skin tone.

Our Advice

  • Learn – The best way to help protect children from deepfakes is to educate yourself. Share this article with other parents and safeguarding professionals to help spread the word!

  • Talk – Discuss deepfakes and the importance of image consent with the children in your care. Ensure they know why they should ask someone before using an image of them to create a deepfake or manipulated picture.

  • Check – Make sure all the devices your children own or have access to have the best safety settings enabled. Speak to the children in your care about their safety and privacy settings online. You should also check that they limit public access to their social media images.

Join our Safeguarding Hub Newsletter Network

Members of our network receive weekly updates on the trends, risks and threats to children and young people online.

Sign Up

What is Coronavirus? An explainer for Children

Our animated video has reached over 350,000 views on YouTube.

Watch Video on youtube

Constant Change is Here to Stay

The right information in your pocket, when you need it.

Read More

Safeguarding Update: Squid Game

read

Fortnite: Here’s What You Need to Know

Read

Let’s Talk About
Furries

read

Roblox: What You Need to Know

Read

National Guidance for Child Protection

read