Fight the New Drug (00:00)
Can you speak at all to the ways that AI have has changed this or cases of sex tortion, other ways kind of digital sexual exploitation is happening now?

Kellye Netz (00:11)
Yeah, that that’s been a huge thing recent over the last couple of years is the whole sextortion issue. Though all s it’s really similar to a lot of the other online stuff that we see. Kids are online, unmonitored, they don’t know who they’re talking to. Predators are so good at making kids think that they know something about them.

All it takes is for me to say, Hey, I know your friend Ava. It’s like, okay, well, Ava’s a super common name. Of course, we all know someone named Ava. So for a kid to be like, yeah, I know Ava. And now this relationship has started with a complete stranger. And then as the relationship progresses, they just become so comfortable behind that anonymity that the screen provides that they are willing to send photos of themselves.

And you know, Ava’s friend has now convinced them, like, like send me a picture with your shirt off. Send me a picture of your butt. And pretty soon it’s the whole thing.

The frustrating part about that is a lot of our sextortion cases are actually originating outside the United States, at least at our department. The vast majority of them are coming, the IP addresses, other things that we use to determine where people are, they’re coming back to just like random countries in Africa. And so there’s not really anything that we can do with that. But the really heartbreaking thing about it is it’s devastating for these kids and the adults it happens to, but the kids especially, I think they don’t have the brain development to realize that this is a temporary problem.

It’s not permanent; it’s not gonna last forever. And with that like kind of short-sightedness, they are just devastated by it and a lot of them will become suicidal or they’ll isolate from their family. They don’t want anyone to know, and that’s obviously the wrong thing, the wrong response, and it just leads to further harm.

I’ve talked to a lot of kids who had to, then, you know, they they sent a they sent a picture of–they shouldn’t have sent. Now they’re being extorted for it. So they’ve had to steal money from a parent or from someone else to get a gift card to send the payment. And they’re just getting sucked in further and further. And meanwhile, their mental health is rapidly declining. They feel like this is the end of the world. Might as well just kill myself; you know it’s never gonna be okay again. Everyone’s gonna find out, and they don’t have the skills to know, like no there’s a way out of this, but you’re gonna have to tell somebody.

Fight the New Drug (04:04)
Yeah. Which is why it’s so important, again, we’re aware of these issues, and that parents are talking to their kids, and making sure kids know that they can always come to their parents, they always have a safe, trusted adult, and to know, you know, what these sextortion schemes are, really are at the end of the day.

Can you speak to some of the platforms that predators are on online that parents may not be aware of? Areas where children are speaking to these predators and where these things can happen?

Kellye Netz (04:33)
Yeah, I would say the most common is gaming. So on any gaming site, a lot of the time it’s Roblox. Roblox does a great job of reporting to law enforcement, but they’re still just getting used.

A lot of kids will use Roblox; that’s where chats will start, that’s where a lot of predators will get access to kids, and then they’ll have them move over to a different platform.

We see a lot of Discord. Discord does a great job of reporting as well. However, their platform is just really commonly used for predators to gain access to children. A lot of what goes on on those sites can feel very anonymous because there’s so many users that I think kids feel kind of comfortable saying more than they normally would because they’re hidden behind a username, and they’re in a an essentially a group chat, to use an old-school term at this point, but they’re in this group, and all these people are saying all these things, and they feel more comfortable to, like, go along with it because everyone else is doing it.

So those those are probably the primary two where we see it kind of begin, but a lot of the time then the the child will be asked to like move over to a different platform where the predators know that they’re more protected, like Telegram. I would–no one should let their child have that. They’re not a US-based company; they don’t report, they have no safety features in place. CSAM abounds, as do all other things, because they’re not being governed by the United States, so we really can’t do anything with it. I can’t get a search warrant for Telegram, they just won’t respond.

And so any like, non-US-based social media provider should be a hard no for any parent, at least with the others, even though I don’t think any kids should have social media because of what I see, but I will concede that that’s just a cop thing, and that some kids are maybe better able to manage it than others. But at least you have the kind of safety net with the US-based providers that they’re doing their reporting to NCMEC, the national center.

Fight the New Drug (06:06)
And even with the reporting, I mean there’s so much content. It’s likely not every single thing is being reported that should be, I assume.

Kellye Netz (06:15)
Yeah. No, one hundred percent. They’re…it’s just what’s getting stuck in the net, really. And they’re I’m sure they’re doing their best. Like I said, Discord and Roblox, we get reports from them all the time. I’ve been to conferences where they’ve spoken about the volume, but they’re not only dealing with CSAM; they’re also needing to address all kinds of other issues, like kids that are expressing suicidal thoughts online, or kids that are talking about harming others, or making threats against their school or like terror-type stuff. So they’ve definitely got their hands full.

Fight the New Drug (06:55)
Can you speak a little bit to what we’re seeing happen with AI and deepfakes and nudifying apps? You know, we talked a little bit about sextortion when a child sends photos of themselves, but what about in instances where, you know, deepfakes are made of people, of young people, of adults that look real but are not, but then still are having some detrimental impacts on people’s lives? Where does that fall in the work that you you navigate?

Kellye Netz (07:24)
It’s a hard area. I actually had one of those cases earlier this week. But that’s hard because there’s a couple issues there. One is good, at least here in California, it doesn’t matter if it’s AI, it’s still considered CSAM. So that’s great for us as investigators to not be having to prove that something is a real person, because let’s face it, AI has gotten real good.

So that is a positive towards AI, like that this the law doesn’t care. If it’s CSAM, it’s CSAM, we don’t care if they’re real or fake. As far as the sextortion, that’s a really strange gray area where if the person only shows the image to the juvenile, you know, it if the…then we don’t necessarily have like a distribution of CSAM because they’re only showing it to the person that it’s re supposed to be of, even if it’s not real. So I don’t…there’s not a good law for that right now.

If they were to distribute it, we would then have a crime, but that’s not really what we’re seeing. A lot of the time, they’re just using it to extort money from the kids. So that again is really just creating the same issue, whether or not the kid actually sent the picture or not, if it’s just AI. They’re still suffering the consequences of that, feeling trapped, feeling like they need to lie and steal, and might as well just kill themselves because again, they’re just trapped in this situation that they cannot see a way out of because their poor little brains just haven’t figured it out yet.

Fight the New Drug (09:10)
Yeah. And for anyone who thinks maybe that’s a, you know, a flippant trajectory to say we’re going from this sextortion to, you know, this act of suicide ideation, there have been far too many cases where that has been the end result, and it’s devastating to see. But truly these young people don’t think that there’s another way out sometimes. So this is a very, very serious issue.

What are you seeing in terms of the way that young people are using AI and nudifying apps and what that’s doing?

Kellye Netz (09:44)
Definitely creating a lot of issues for schools. Kids think it’s funny, you know, to to make pictures of one another, pictures of teachers, that could cause and has caused huge issues for staff members who have then been found out that there’s this nude image of them. It’s like it’s not real.

Yeah, that’s…it’s devastating. It’s the same…it’s devastating in the same way. It’s devastating if it’s real. It really doesn’t make any difference.

It hurts the same way no matter what. And trying to tell someone like that’s not a real picture of me, it really just feeds the fire. And so I think that might actually be worse, because then when you have someone like so adamantly denying something, especially a kid, I think it just causes the other kids to feed into it even more.

Fight the New Drug (10:43)
Right. And it it’s so tricky. I mean, we’re hearing all too often the way that these issues are impacting schools. And again, for parents, even if they think, you know, not my kid, these are things that can affect your kid. Even if your child doesn’t have social media, doesn’t have access to gaming systems, doesn’t have all of these things, if you have all of the filters in the world, these are still very pr prevalent issues in the world that they’re growing up in. So it’s so important that we’re aware of all of these issues.

Kellye Netz (11:15)
Yeah, my my kids don’t have social media, but they’ve still gotten the pictures sent to them. You know, they’ve still they still know about it. They’ve seen videos of other, you know, AI videos of other kids. It doesn’t matter. The kids are all showing each other and sharing it anyways.

Fight the New Drug (11:35)
Yeah. That’s what I was just gonna ask, you know, you’re not only approaching this as an investigator, but also as a parent. And how has your work changed the way that you parent? You mentioned your kids don’t have social media. Are there other ways that it’s changed the way you parent?

Kellye Netz (11:49)
I’m sure my kids would tell you that. Mom, she’s so annoying. She doesn’t let us do anything. I try not to be like overly paranoid. I try to not be a helicopter parent and just fretting about everything. I don’t think that’s a good way to live. I want my kids to grow up to be confident, contributing members of society. And so you can’t hover.

But there’s definitely things that I just am gonna choose. You know, my husband and I are just gonna choose not to put them in a situation where they’re exposed to something or where they have to make a choice that’s beyond their ability. You know, if I have a teen son, should I put him in a position where he has to choose not to look at all the girls in bikinis on Instagram? I just feel like that’s setting him up for failure. Because of course he wants to see that.

You know, and then my kids will definitely say that I talk to them about weird things and that none of their friends’ parents talk to them about sex, and why do we have to? And I just, I feel like you have to talk about it. And I just tell them like their friends’ parents should be talking to them about it. And if their friends want to talk about it, but their parents won’t, send them over. I’ll talk to them about. I’ll answer their questions. Like I’m not afraid to say penis. And it drives them my kids crazy. But it’s so important to talk to them about it, even when we have an issue, even when a has been crossed. At least then we can have a conversation about it and figure out what to do about it.

My son has a phone. He’s had the internet at times, and then he’s had the internet taken away. And that’s just something that I think you have to keep evaluating as a parent is like, what is my kid doing? What are they looking at? You know, the poor kid had YouTube and was just watching fishing videos. But YouTube knows, that that algorithm knows that this is a teenage boy who is watching videos of fishing and planes, and so all their ads started becoming sexual. And I just don’t want him exposed to that. So YouTube went away.

But as a parent, it’s just really important that you know what your kids are doing. I think that’s the biggest issue that I see regularly is, I’ll go and I’ll talk to parents about some image of their child that I have been sent, and they’ll just not even know what that app was or what’s that social media platform. A lot of parents will just say, oh yeah, like I don’t even understand what that is. Like, then your child shouldn’t have it. If you don’t know what it is, then it should be a no. If you want your child to have a Snapchat, you should also have a Snapchat and know how to work it, so that you can help your child use it safely. And that goes for that goes for everything.

So the amount of parents that I’ve talked to who I’ll say, your child was on Discord, they were on this channel having this conversation. What’s Discord? What’s a channel? How did they get to that? That’s probably the biggest issue that leads to all of this just unfettered internet activity.

Fight the New Drug (15:37)
And I think that’s such an important point. Because technology is always evolving, and there are new apps coming out all of the time, and you know, social media is changing all the time, algorithms are changing, and these are predatory industries that, as you mentioned, are targeting young people, especially young young boys, in these ways. You know, parents can’t just say, Okay, I’ve learned everything, I feel good about it, here’s a phone, and then that’s it. It it’s an ongoing continually learning process because things are evolving and changing so much.

You know, we present in schools, and we’re constantly having to update our presentation so that we can inform parents and young people about the way that these things are changing. And and there is a responsibility on parents to maintain an understanding. And I think so many parents want to, they just don’t know where to begin. You know, so many parents are worried they’ll either say too much or not enough. And what advice would you give parents who are just trying to start these important conversations with their their young children?

Kellye Netz (16:36)
I would say just start with the tools available to them. I mean, Fight the New Drug has great resources, as does NCMEC, the National Center for Missing Exploited Children. They have videos called, I think, Into the Cloud, that are designed for elementary school-age, middle school-age, high school-age kids. Like, start with what’s already available. You do not have to recreate the wheel. kids are all going through pretty much the same thing, so it’s gonna be applicable.

So I would just encourage parents to use those things to start the conversation. That’s obviously not where it ends. You know, that’s kind of would be, I guess, the equivalent of like my mom like awkwardly handing me like a book about puberty and then being like, good luck. You have to continue the conversation,

And knowing that they do come to me then and say, Mom, I I’m struggling with this, or Mom, this ad popped up, and I don’t think it’s appropriate. It’s like, thank you. Yeah, use the tools that are out there so you don’t have to come up with, you know, what to say on your own. And that is just at least the beginning of the conversation.

Fight the New Drug (17:51)
Yeah, at Fight the New Drug, we have a conversation blueprint discussion guide. We have created our own AI on our website where parents can go ask any question. And instead of getting all of the answers from any AI, it will give you very specific answers to research on pornography and all of the resources we have available. And we also have a free documentary series called Brainheart World that’s made to be appropriate for ages middle school and up that parents can watch with young people to help start these conversations.

And you know, as you mentioned, maybe your kids are a little embarrassed, or maybe they’re saying, Mom, we talk about this all the time, we know, but all the evidence you need of it being effective is that they come to you when there is something concerning and and that’s ultimately the goal for parents is you know, you don’t want to just have the conversation once or twice and then have your kids go to Google when they need answers to questions. You wanna be the person they come to. And if you don’t have the answer, it’s okay to say, I’m not sure. Let’s look into that and I’ll get back to you.