As someone who grew up on the internet, I credit it as one of the most important influences on who I am today. I had a computer with internet access in my bedroom from the age of It gave me access to a lot of things which were totally inappropriate for a young teenager, but it was OK.
The culture, politics, and interpersonal relationships which I consider to be central to my identity were shaped by the internet, in ways that I have always considered to be beneficial to me personally. I have always been a critical proponent of the internet and everything it has brought, and broadly considered it to be emancipatory and beneficial.
I state this at the outset because thinking through the implications of the problem I am going to describe troubles my own assumptions and prejudices in significant ways. One of the thus-far hypothetical questions I ask myself frequently is how I would feel about my own children having the same kind of access to the internet today. And I find the question increasingly difficult to answer.
I understand that this is a natural evolution of attitudes which happens with age, and at some point this question might be a lot less hypothetical. I would want my kids to have the same opportunities to explore and grow and express themselves as I did. I would like them to have that choice. And this belief broadens into attitudes about the role of the internet in public life as whole.
Someone or something or some combination of people and things is using YouTube to systematically frighten, traumatise, and abuse children, automatically and at scale, and it forces me to question my own beliefs about the internet, at every level. Last year, there were a number of articles posted about the Surprise Egg craze. Surprise Eggs videos depict, often at excruciating length, the process of unwrapping Kinder and other egg toys.
There are thousands and thousands of these videos and thousands and thousands, if not millions, of children watching them. The video titles are a continuous pattern of obscure branded lines and tie-ins: Surprise-egg videos are all accompanied by pre-roll, and sometimes mid-video and ads.
That should give you some idea of just how odd the world of kids online video is, and that list of video titles hints at the extraordinary range and complexity of this situation. Another huge trope, especially the youngest children, is nursery rhyme videos.
Little Baby Bum , which made the above video, is the 7th most popular channel on YouTube. With just videos, they have accrued On-demand video is catnip to both parents and to children, and thus to content creators and advertisers. The first of these tactics is simply to copy and pirate other content. I am not alleging anything bad about Play Go Toys; I am simply illustrating how the structure of YouTube facilitates the delamination of content and author, and how this impacts on our awareness and trust of its source.
As another blogger notes , one of the traditional roles of branded content is that it is a trusted source. This no longer applies when brand and content are disassociated by the platform, and so known and trusted content provides a seamless gateway to unverified and potentially harmful content. Yes, this is the exact same process as the delamination of trusted news media on Facebook feeds and in Google results that is currently wreaking such havoc on our cognitive and political systems and I am not going to explicitly explore that relationship further here, but it is obviously deeply significant.
When some trend, such as Surprise Egg videos, reaches critical mass, content producers pile onto it, creating thousands and thousands more of these videos in every possible iteration. This is the origin of all the weird names in the list above: A striking example of the weirdness is the Finger Family videos harmless example embedded above.
Once again, the view numbers of these videos must be taken under serious advisement. A huge number of these videos are essentially created by bots and viewed by bots, and even commented on by bots.
That is a whole strange world in and of itself. What I find somewhat disturbing about the proliferation of even relatively normal kids videos is the impossibility of determining the degree of automation which is at work here; how to parse out the gap between human and machine. The example above, from a channel called Bounce Patrol Kids , with almost two million subscribers, show this effect in action.
It posts professionally produced videos, with dedicated human actors, at the rate of about one per week. Once again, I am not alleging anything untoward about Bounce Patrol, which clearly follows in the footsteps of pre-digital kid sensations like their fellow Australians The Wiggles. And yet, there is something weird about a group of people endlessly acting out the implications of a combination of algorithmically generated keywords: Other channels do away with the human actors to create infinite reconfigurable versions of the same videos over and over again.
What is occurring here is clearly automated. Stock animations, audio tracks, and lists of keywords being assembled in their thousands to produce an endless stream of videos. This very indeterminacy and reach is key to its existence, and its implications. Its dimensionality makes it difficult to grasp, or even to really think about. Much has been made of the algorithmic interbreeding of stock photo libraries and on-demand production of everything from tshirts to coffee mugs to infant onesies and cell phone covers.
The above example, available until recently on Amazon, is one such case, and the story of how it came to occur is fascinating and weird but essentially comprehensible.
The fact that it took a while to notice might ring some alarm bells however. Nobody set out to create these shirts: Once again though, the people creating this content failed to notice, and neither did the distributor.
They literally had no idea what they were doing. Once again, a content warning: This warning will recur. The title alone confirms its automated provenance. Again, this is weird but frankly no more than the Surprise Egg videos or anything else kids watch.
I get how innocent it is. The offness creeps in with the appearance of a non-Aladdin character —Agnes, the little girl from Despicable Me. Agnes is the arbiter of the scene: As many of the Wrong Heads videos as I could bear to watch all work in exactly the same way. It goes on and on. I get the game, but the constant overlaying and intermixing of different tropes starts to get inside you. Following the publication of this article, the Toy Freaks channel was removed by YouTube as part of a widespread removal of contentious content.
As well as nursery rhymes and learning colours, Toy Freaks specialises in gross-out situations, as well as activities which many, many viewers feel border on abuse and exploitation, if not cross the line entirely, including videos of the children vomiting and in pain. Toy Freaks is a YouTube verified channel, whatever that means. I think we know by now it means nothing useful. As with Bounce Patrol Kids, however you feel about the content of these videos, it feels impossible to know where the automation starts and ends, who is coming up with the ideas and who is roleplaying them.
In turn, the amplification of tropes in popular, human-led channels such as Toy Freaks leads to them being endlessly repeated across the network in increasingly outlandish and distorted recombinations. Here is a relatively mild, but still upsetting example: A step beyond the simply pirated Peppa Pig videos mentioned previously are the knock-offs. These too seem to teem with violence.
In the official Peppa Pig videos, Peppa does indeed go to the dentist, and the episode in which she does so seems to be popular — although, confusingly, what appears to be the real episode is only available on an unofficial channel. In the official timeline, Peppa is appropriately reassured by a kindly dentist. In the version above, she is basically tortured, before turning into a series of Iron Man robots and performing the Learn Colours dance.
They make up an entire YouTube subculture. All the 4chan tropes are there, the trolls are out, we know this. In the example above, the agency is less clear: I kind of hope it is. I understand that most of them are not trying to mess kids up, not really, even though they are. Obviously this content is inappropriate, obviously there are bad actors out there, obviously some of these videos should be removed.
Obviously too this raises questions of fair use, appropriation, free speech and so on. But reports which simply understand the problem through this lens fail to fully grasp the mechanisms being deployed, and thus are incapable of thinking its implications in totality, and responding accordingly.
YouTube Kids, an official app which claims to be kid-safe but is quite obviously not, is the problem identified, because it wrongly engenders trust in users. But as with Toy Freaks, what is concerning to me about the Peppa videos is how the obvious parodies and even the shadier knock-offs interact with the legions of algorithmic content producers until it is completely impossible to know what is going on. There is a lot of effort going into making these. More than spam revenue can generate — can it?
Once again, I want to stress: Here are a few things which are disturbing me: The first is the level of horror and violence on display. I spend a lot of time arguing for this tendency, with regards to human sexual freedom, individual identity, and other issues. Here, and overwhelmingly it sometimes feels, that tendency is itself a violent and destructive one. The second is the levels of exploitation, not of children because they are children but of children because they are powerless.
Exploitation is encoded into the systems we are building, making it harder to see, harder to think and explain, harder to counter and defend against. Not in a future of AI overlords and robots in the factories, but right here, now, on your screen, in your living room and in your pocket. Many of these latest examples confound any attempt to argue that nobody is actually watching these videos, that these are all bots. The video used animations from the Grand Theft Auto game series overlaid with cartoon characters assaulting, killing, and burying one another.
And of course there are vast, vast numbers of these videos. Channel after channel after channel of similar content, churned out at the rate of hundreds of new videos every week. For the final time: There is more violent and more sexual content like this available.
A friend who works in digital video described to me what it would take to make something like this: