Families can sue app developer for breaking its anti-bullying pledge

An appeals court revived a lawsuit against the anonymous messaging service Yolo, which allegedly broke a promise to unmask bullies on the app. In a ruling issued Thursday, the Ninth Circuit Court of Appeals said Section 230 of the Communications Decency Act shouldn’t block a claim that Yolo misrepresented its terms of service, overruling a lower court decision. But it determined the app can’t be held liable for alleged design defects that allowed harassment, letting a different part of that earlier ruling stand.

Yolo was a Snapchat-integrated app that let users send anonymous messages, but in 2021, it was hit with a lawsuit after a teenage user died by suicide. The boy, Carson Bride, had received harassing and sexually explicit messages from anonymized users that — he believed — he likely knew. Bride and his family attempted to contact Yolo for help, but Yolo allegedly never answered, and in some cases, emails to the company simply bounced. Snap banned Yolo and another app targeted in the lawsuit, and a year later, it banned all anonymous messaging integration.

Bride’s family and a collection of other aggrieved parents argued that Yolo broke a legally binding promise to its users. They pointed to a notification where Yolo claimed people would be banned for inappropriate use and deanonymized if they sent “harassing messages” to others. But as the ruling summarizes, the plaintiffs argued that “with a staff of no more than ten people, there was no way Yolo could monitor the traffic of ten million active daily users to make good on its promise, and it in fact never did.” Additionally, they claimed Yolo should have known its anonymous design facilitated harassment, making it defective and dangerous.

A lower court threw out both of these claims, saying that under Section 230, Yolo couldn’t be held responsible for its users’ posts. The appeals court was more sympathetic. It accepted the argument that families were instead holding Yolo responsible for promising users something it couldn’t deliver. “Yolo repeatedly informed users that it would unmask and ban users who violated the terms of service. Yet it never did so, and may have never intended to,” writes Judge Eugene Siler, Jr. “While yes, online content is involved in these facts, and content moderation is one possible solution for Yolo to fulfill its promise, the underlying duty … is the promise itself.”

The Yolo suit built on a previous Ninth Circuit ruling that let another Snap-related lawsuit circumvent Section 230’s shield. In 2021, it found Snap could be sued for a “speed filter” that could implicitly encourage users to drive recklessly, even if users were responsible for making posts with that filter. (The overall case is still ongoing.) On top of their misrepresentation claim, the plaintiffs argued Yolo’s anonymous messaging capability was similarly risky, an argument the Ninth Circuit didn’t buy — “we refuse to endorse a theory that would classify anonymity as a per se inherently unreasonable risk,” Siler wrote.

This recent ruling is part of an extended push-and-pull over Section 230’s scope. Several cases have sought to claim that apps are illegally defective if they lead to harassment or other harms, even if those harms were committed by users. Despite periodic victories, it’s still far from an established doctrine, and the Supreme Court declined to consider it for the Herrick v. Grindr case back in 2019. The Supreme Court also declined to pare down Section 230 in a case over whether YouTube and Twitter supported illegal terrorism. After this Ninth Circuit ruling, Yolo can still mount a defense that it reasonably attempted to enforce its user agreement, and the case isn’t over.

Even so, letting users sue a company for not upholding its content policy could theoretically allow lawsuits against nearly any service that doesn’t practice (often impossibly) perfect moderation. The Ninth Circuit insists that’s not what it’s doing. “Today’s decision does not expand liability for internet companies or make all violations of their own terms of service into actionable claims,” Siler writes. “In our caution to ensure [Section] 230 is given its fullest effect, we must resist the corollary urge to extend immunity beyond the parameters established by Congress and thereby create a free-wheeling immunity for tech companies.”