A 22-year-old student teacher in Illinois sent a private Snapchat message to her boyfriend and two roommates.
A student had walked up to her laptop and deleted her lesson plan mid-class. frustrated, she typed something like "should I shoot him" with a gun emoji. venting. four people. private group chat.
An hour later, police walked into her school and arrested her.
here's what happened in between.
Snapchat's AI scanned the private message. flagged it as a potential threat. automatically reported it to the FBI. the FBI forwarded it to local law enforcement. deputies arrived at the school within the hour.
she cooperated immediately. handed over her phone. when shown the message she said: "oh yes. okay. yeah. i'm realizing that was a bad joke. i did not mean it at all serious at all."
police determined she was not a threat. school officials determined she was not a threat. prosecutors reviewed and issued a disorderly conduct charge. she was released the next morning. she lost her student teaching placement.
This means:
Snapchat scans private messages.
Not just public posts or stories. private group chats between you and your closest contacts. automated AI. no human reviewed it first, got flagged and reported.
Snapchat AI → FBI → local police → school → arrest.
in under an hour.
Snapchat's privacy policy says it may share your information with law enforcement when it believes there is a risk of harm. the definition of "risk of harm" is determined by an algorithm.
you are not told when a message is flagged.
you are not told when a report is filed.
you find out when the police arrive.
the message was private.
it wasn't.
A woman made a joke in a private Snapchat message. Snapchat’s AI reported it to the FBI. She was arrested before any human she sent it to ever reported her.
