These apps — many of which have been adopted by Gen Z as teens and young people seek out more private corners of the Internet — are ill-equipped to police such content. They are fundamentally designed to keep communications private, presenting different challenges than Facebook, YouTube and Twitter, where violent screeds and videos have been algorithmically amplified to millions of viewers.
The way that the generation uses social media more generally could render years of work to spot and identify public signs of upcoming violence obsolete, social media experts warn.
“There is this shift toward more private spaces, more ephemeral content,” said Evelyn Douek, a senior research fellow at Knight First Amendment Institute at Columbia University. “The content moderation tools that platforms have been building and that we’ve been arguing about are kind of dated or talking about the last war.”
Gov. Greg Abbott said Wednesday that the Texas gunman, who authorities have identified as Salvador Rolando Ramos, 18, wrote on social media that “I’m going to shoot my grandmother” and “I’m going to shoot an elementary school” shortly before the attack.” Facebook confirmed that the messages were sent privately but declined to say which of its social networks were used.
Stephen Garcia, who considered himself Ramos’s best friend in eighth grade, previously told The Washington Post that Ramos used the Yubo app, a platform where users can swipe on each other’s profile, Tinder-style, or hang out in live-streaming rooms and virtually “meet” other users by playing games and chatting.
Yubo spokeswoman Amy Williams said in an email that the company is not able to release information outside of direct requests from law enforcement, but that the company is investigating an account that has been banned from its platform.
“We are deeply saddened by this unspeakable loss and are fully cooperating with law enforcement on their investigation,” she said.
In the case of Buffalo shooting, the alleged gunman Payton Gendron sent an invitation to an online chatroom on the instant messaging platform Discord that was accepted by 15 users, which allowed them to scroll back through months of Gendron’s voluminous writings and racist screeds, The Post has reported. Users who clicked through to the room also could view an online video stream, where footage of the Buffalo attack was broadcast. That attack was also broadcast on Twitch, a live-streaming service popular among video game users.
Discord and Twitch did not immediately respond to requests for comment.
Twitch was able to remove the stream within two minutes after the gunman began shooting, Angela Hession, the company’s head of trust and safety, said previously. The site has an all-hours escalation system in place to address urgent reports, such as live-streamed violence.
Discord has since said the messages were visible only to the suspect until he shared them with others the day of the attack.
In the wake of high-profile mass shootings in recent years, communities, school districts and tech companies made major investments in safety systems aimed to root out violent screeds in the hopes of preventing an attack before it happens. The Uvalde Consolidated Independent School District has previously used an artificial intelligence-backed program to scan social media posts for potential threats years before the attack, although it’s unclear whether it was in use at the time of the shooting.
But these tools are ill-equipped to address the surging popularity of live video streaming and private or disappearing messaging, which are increasingly used by young adults and teens. Those messages are then closed off to outsiders, who might be able to spot the warning signs that a troubled individual might be about to inflict harm on themselves and others.
These newer social networks also have far less history dealing with violent content, and they’re less likely to have policies and personnel in place to respond to the incitement of violence on their services, experts said.
“For smaller sites or newer sites, they’re having the moments that bigger services like Facebook and YouTube were having in 2015 and 2016,” said Emma Llansó, the director of the free expression project at the Center for Democracy and Technology, a nonprofit backed by major tech companies.
The shooters’ adoption of these upstart apps reflects a larger generational shift among social media use. Gen Z, teens and young adults born after 1996, have been flocking to apps that emphasize private messaging, live-streaming or allow their users to post content that disappears from public profiles after a certain amount of time.
They have largely shunned legacy social media apps such as Facebook, Twitter and YouTube, that rose to popularity by providing public and open spaces to communicate with the world.
The new apps’ role in the shootings have caught the attention of the New York and New Jersey state attorneys general, who in the wake of the Buffalo shooting launched probes into Discord and Twitch.
“Time and time again, we have seen the real-world devastation that is borne of these dangerous and hateful platforms,” New York attorney general Letitia James (D) said in a statement announcing the probe after the Buffalo shooting. “We are doing everything in our power to shine a spotlight on this alarming behavior and take action to ensure it never happens again.”
Social media has played a prominent role in many mass shootings, and there have been high-profile instances where gunmen have posted about their plans online in plain sight and have not been caught.
Tech giants have also been caught up in a years-long power struggle as they seek to balance privacy with policing content on their sites and demands from law enforcement agencies.
Facebook and other companies have moved toward strong encryption, technology that scrambles the contents of a message so that only the sender and receiver can see it. WhatsApp and Apple iMessage use it, as well as messaging apps like Signal. And Facebook has said it wants to introduce encrypted messaging as a default setting to Instagram and Facebook Messenger, prompting backlash from politicians and officials in law enforcement have warned that the broad adoption of this technology can leave them in the dark and made it more difficult for them to investigate violence.
Some major tech companies do scan messages for harmful content, such as child sexual abuse or spam. But experts warn that monitoring more private communication spaces is a delicate balance.
“There are so many incredibly legitimate reasons people want to use private communications,” Llansó said. “That is not something that should be sacrificed for all people because some people want to use private communications for atrocious reasons.”
Social media users tend to skew younger, but the generational gaps among the user base among private messaging apps like Snapchat are larger than they are for more traditional public-facing sites like Facebook.
When Snapchat users send private messages to each other they disappear after the recipient has read them. The app also pioneered the concept of “stories” — public posts that last for just one day — which was later copied by Facebook.
Snap said Wednesday that it has suspended an account that may have been connected to Ramos and that it is also working with law enforcement.
Meanwhile, Facebook has struggled to keep pace with the rapidly evolving social habits of teen users.
Facebook’s own internal research reports that young adults are “less engaged” than older adults, posing a significant risk to the company’s business, according to a trove of internal company documents known as the Facebook Files. The company’s research found that young adults prefer sharing updates about their life over text messages, rather than broadcasting to a broad range of Facebook friends. The researchers suggested that the company respond by leaning into groups and more private forms of sharing.
“It’s always going to be a cat and mouse game,” Douek said. “These are just sort of intractable problems. But that doesn’t mean that we can’t improve or we should let platforms off the hook.”
Rachel Lerman contributed to this report.
