934 F.3d 53

Katzmann, Chief Judge, concurring in part and dissenting in part:
I agree with much of the reasoning in the excellent majority opinion, and I join that opinion except for Parts I and II of the Discussion. But I must respectfully part company with the majority on its treatment of Facebook's friend- and content-suggestion algorithms under the Communications Decency Act ("CDA").
As to the reasons for my disagreement, consider a hypothetical. Suppose that you are a published author. One day, an acquaintance calls. "I've been reading over everything you've ever published," he informs you. "I've also been looking at everything you've ever said on the Internet. I've done the same for this other author. You two have very similar interests; I think you'd get along." The acquaintance then gives you the other author's contact information and photo, along with a link to all her published works. He calls back three more times over the next week with more names of writers you should get to know.
Now, you might say your acquaintance fancies himself a matchmaker. But would you say he's acting as the publisher of the other authors' work?
Facebook and the majority would have us answer this question "yes." I, however, cannot do so. For the scenario I have just described is little different from how Facebook's algorithms allegedly work. And while those algorithms do end up showing users profile, group, or event pages written by other users, it strains the English language to say that in targeting and recommending these writings to users-and thereby forging connections, developing new social networks-Facebook is acting as "the publisher of ... information provided by another information content provider." 47 U.S.C. § 230(c)(1) (emphasis added).
It would be one thing if congressional intent compelled us to adopt the majority's reading. It does not. Instead, we today extend a provision that was designed to encourage computer service providers to shield minors from obscene material so that it now immunizes those same providers for allegedly connecting terrorists to one another. Neither the impetus for nor the text of § 230(c)(1) requires such a result. When a plaintiff brings a claim that is based not on the content of the information shown but rather on the connections Facebook's algorithms make between individuals, the CDA does not and should not bar relief.
The Anti-Terrorism Act ("ATA") claims in this case fit this bill. According to plaintiffs' Proposed Second Amended Complaint ("PSAC")-which we must take as true at this early stage-Facebook has developed "sophisticated algorithm[s]" for bringing its users together. App'x 347 ¶ 622. After collecting mountains of data about each user's activity on and off its platform, Facebook unleashes its algorithms to generate friend, group, and event suggestions based on what it perceives to be the user's interests. Id. at 345-46 ¶¶ 608-14. If a user posts about a Hamas attack or searches for information about a Hamas leader, Facebook may "suggest" that that user become friends with Hamas terrorists on Facebook or join Hamas-related Facebook groups. By "facilitat[ing] [Hamas's] ability to reach and engage an audience it could not otherwise reach as effectively," plaintiffs allege that Facebook's algorithms provide material support and personnel to terrorists. Id. at 347 ¶ 622; see id. at 352-58 ¶¶ 646-77. As applied to the algorithms, plaintiffs' claims do not seek to punish Facebook for the content others post, for deciding whether to publish third parties' content, or for editing (or failing to edit) others' content before publishing it. In short, they do not rely on treating Facebook as "the publisher" of others' information. Instead, they would hold Facebook liable for its affirmative role in bringing terrorists together.
When it comes to Facebook's algorithms, then, plaintiffs' causes of action do not run afoul of the CDA. Because the court below did not pass on the merits of the ATA claims pressed below, I would send this case back to the district court to decide the merits in the first instance. The majority, however, cuts off all possibility for relief based on algorithms like Facebook's, even if these or future plaintiffs could prove a sufficient nexus between those algorithms and their injuries. In light of today's decision and other judicial interpretations of the statute that have generally immunized social media companies-and especially in light of the new reality that has evolved since the CDA's passage-Congress may wish to revisit the CDA to better calibrate the circumstances where such immunization is appropriate and inappropriate in light of congressional purposes.
I.
To see how far we have strayed from the path on which Congress set us out, we must consider where that path began. What is now 47 U.S.C. § 230 was added as an amendment to the Telecommunications Act of 1996, a statute designed to deregulate and encourage innovation in the telecommunications industry. Pub. L. 104-104, § 509, 110 Stat. 56, 56, 137-39; see Reno , 521 U.S. at 857, 117 S.Ct. 2329. Congress devoted much committee attention to traditional telephone and broadcast media; by contrast, the Internet was an afterthought, addressed only through floor amendments or in conference. Reno , 521 U.S. at 857-58, 117 S.Ct. 2329. Of the myriad issues the emerging Internet implicated, Congress tackled only one: the ease with which the Internet delivers indecent or offensive material, especially to minors. See Telecommunications Act of 1996, tit. V, subtit. A, 110 Stat. at 133-39. And § 230 provided one of two alternative ways of handling this problem.
The action began in the Senate. Senator James J. Exon introduced the CDA on February 1, 1995. See 141 Cong. Rec. 3,203. He presented a revised bill on June 9, 1995, "[t]he heart and the soul" of which was "its protection for families and children." Id. at 15,503 (statement of Sen. Exon). The Exon Amendment sought to reduce the proliferation of pornography and other obscene material online by subjecting to civil and criminal penalties those who use interactive computer services to make, solicit, or transmit offensive material. Id. at 15,505.
The House of Representatives had the same goal-to protect children from inappropriate online material-but a very different sense of how to achieve it. Congressmen Christopher Cox (R-California) and Ron Wyden (D-Oregon) introduced an amendment to the Telecommunications Act, entitled "Online Family Empowerment," about two months after the revised CDA appeared in the Senate. See id. at 22,044. Making the argument for their amendment during the House floor debate, Congressman Cox stated:
We want to make sure that everyone in America has an open invitation and feels welcome to participate in the Internet. But as you know, there is some reason for people to be wary because, as a Time Magazine cover story recently highlighted, there is in this vast world of computer information, a literal computer library, some offensive material, some things in the bookstore, if you will, that our children ought not to see.
As the parent of two, I want to make sure that my children have access to this future and that I do not have to worry about what they might be running into on line. I would like to keep that out of my house and off my computer.
Id. at 22,044-45. Likewise, Congressman Wyden said: "We are all against smut and pornography, and, as the parents of two small computer-literate children, my wife and I have seen our kids find their way into these chat rooms that make their middle-aged parents cringe." Id. at 22,045.
As both sponsors noted, the debate between the House and the Senate was not over the CDA's primary purpose but rather over the best means to that shared end. See id. (statement of Rep. Cox) ("How should we do this? ... Mr. Chairman, what we want are results. We want to make sure we do something that actually works."); id. (statement of Rep. Wyden) ("So let us all stipulate right at the outset the importance of protecting our kids and going to the issue of the best way to do it."). While the Exon Amendment would have the FCC regulate online obscene materials, the sponsors of the House proposal "believe[d] that parents and families are better suited to guard the portals of cyberspace and protect our children than our Government bureaucrats." Id. at 22,045 (statement of Rep. Wyden). They also feared the effects the Senate's approach might have on the Internet itself. See id. (statement of Rep. Cox) ("[The amendment] will establish as the policy of the United States that we do not wish to have content regulation by the Federal Government of what is on the Internet, that we do not wish to have a Federal Computer Commission with an army of bureaucrats regulating the Internet ...."). The Cox-Wyden Amendment therefore sought to empower interactive computer service providers to self-regulate, and to provide tools for parents to regulate, children's access to inappropriate material. See S. Rep. No. 104-230, at 194 (1996) (Conf. Rep.); 141 Cong. Rec. 22,045 (statement of Rep. Cox).
There was only one problem with this approach, as the House sponsors saw it. A New York State trial court had recently ruled that the online service Prodigy, by deciding to remove certain indecent material from its site, had become a "publisher" and thus was liable for defamation when it failed to remove other objectionable content. Stratton Oakmont, Inc. v. Prodigy Servs. Co. , 1995 WL 323710, at *4 (N.Y. Sup. Ct. May 24, 1995) (unpublished). The authors of § 230 saw the Stratton-Oakmont decision as indicative of a "legal system [that] provides a massive disincentive for the people who might best help us control the Internet to do so." 141 Cong. Rec. 22,045 (statement of Rep. Cox). Cox-Wyden was designed, in large part, to remove that disincentive. See S. Rep. No. 104-230, at 194.
The House having passed the Cox-Wyden Amendment and the Senate the Exon Amendment, the conference committee had before it two alternative visions for countering the spread of indecent online material to minors. The committee chose not to choose. Congress instead adopted both amendments as part of a final Communications Decency Act. See Telecommunications Act of 1996, §§ 502, 509, 110 Stat. at 133-39; Reno , 521 U.S. at 858 n.24, 117 S.Ct. 2329. The Supreme Court promptly struck down two major provisions of the Exon Amendment as unconstitutionally overbroad under the First Amendment, leaving the new § 230 as the dominant force for securing decency on the Internet. See Reno , 521 U.S. at 849, 117 S.Ct. 2329.
Section 230 overruled Stratton-Oakmont through two interlocking provisions, both of which survived the legislative process unscathed. The first, which is at issue in this case, states that "[n]o provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider." 47 U.S.C. § 230(c)(1). The second provision eliminates liability for interactive computer service providers and users for "any action voluntarily taken in good faith to restrict access to or availability of material that the provider or user considers to be ... objectionable," or "any action taken to enable or make available to ... others the technical means to restrict access to [objectionable] material." Id. § 230(c)(2). These two subsections tackle, in overlapping fashion, the two jurisprudential moves of the Stratton-Oakmont court: first, that Prodigy's decision to screen posts for offensiveness rendered it "a publisher rather than a distributor," 1995 WL 323710, at *4 ; and second, that by making good-faith efforts to remove offensive material Prodigy became liable for any actionable material it did not remove.
The legislative history illustrates that in passing § 230 Congress was focused squarely on protecting minors from offensive online material, and that it sought to do so by "empowering parents to determine the content of communications their children receive through interactive computer services." S. Rep. No. 104-230, at 194. The "policy" section of § 230's text reflects this goal. See 47 U.S.C. § 230(b)(3)-(4). It is not surprising, then, that Congress emphasized the narrow civil liability shield that became § 230(c)(2), rather than the broad rule of construction laid out in § 230(c)(1). Indeed, the conference committee summarized § 230 by stating that it "provides 'Good Samaritan' protections from civil liability for providers or users of an interactive computer service for actions to restrict or to enable restriction of access to objectionable online material"-a description that could just as easily have applied to § 230(c)(2) alone. S. Rep. No. 104-230, at 194. Congress also titled the entirety of § 230(c) "Protection for 'Good Samaritan' blocking and screening of offensive material," suggesting that the definitional rule outlined in § 230(c)(1) may have been envisioned as supporting or working in tandem with the civil liability shield in § 230(c)(2).
None of this is to say that § 230(c)(1) exempts interactive computer service providers from publisher treatment only when they remove indecent content. Statutory text cannot be ignored, and Congress grabbed a bazooka to swat the Stratton-Oakmont fly. Whatever prototypical situation its drafters may have had in mind, § 230(c)(1) does not limit its protection to situations involving "obscene material" provided by others, instead using the expansive word "information." Illuminating Congress's original intent does, however, underscore the extent of § 230(c)(1)'s subsequent mission creep. Given how far both Facebook's suggestion algorithms and plaintiffs' terrorism claims swim from the shore of congressional purpose, caution is warranted before courts extend the CDA's reach any further.
II.
With the CDA's background in mind, I turn to the text. By its plain terms, § 230 does not apply whenever a claim would treat the defendant as "a publisher" in the abstract, immunizing defendants from liability stemming from any activity in which one thinks publishing companies commonly engage. Contra ante , at 65, 66, 70. It states, more specifically, that "[n]o provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider." 47 U.S.C. § 230(c)(1) (emphases added). "Here grammar and usage establish that 'the' is a function word indicating that a following noun or noun equivalent is definite ...." Nielsen v. Preap , --- U.S. ----, 139 S. Ct. 954, 965, 203 L.Ed.2d 333 (2019) (citation and internal quotation marks omitted). The word "publisher" in this statute is thus inextricably linked to the "information provided by another." The question is whether a plaintiff's claim arises from a third party's information, and-crucially-whether to establish the claim the court must necessarily view the defendant, not as a publisher in the abstract, but rather as the publisher of that third-party information. See FTC v. LeadClick Media, LLC , 838 F.3d 158, 175 (2d Cir. 2016) (stating inquiry as "whether the cause of action inherently requires the court to treat the defendant as the 'publisher or speaker' of content provided by another").
For this reason, § 230(c)(1) does not necessarily immunize defendants from claims based on promoting content or selling advertising, even if those activities might be common among publishing companies nowadays. A publisher might write an email promoting a third-party event to its readers, for example, but the publisher would be the author of the underlying content and therefore not immune from suit based on that promotion. See 47 U.S.C. § 230(c)(1), (f)(3). Similarly, the fact that publishers may sell advertising based on user data does not immunize the publisher if someone brings a claim based on the publisher's selling of the data, because the claim would not treat the defendant as the publisher of a third party's content. Cf. Oberdorf v. Amazon.com Inc. , No. 18-1041, 930 F.3d 136, 153-54, 2019 WL 2849153, at *12 (3d Cir. July 3, 2019) (holding that the CDA does not bar claims against Amazon.com "to the extent that" they "rely on Amazon's role as an actor in the sales process," including both "selling" and "marketing"). Section 230(c)(1) limits liability based on the function the defendant performs, not its identity.
Accordingly, our precedent does not grant publishers CDA immunity for the full range of activities in which they might engage. Rather, it "bars lawsuits seeking to hold a service provider liable for its exercise of a publisher's traditional editorial functions-such as deciding whether to publish, withdraw, postpone or alter content" provided by another for publication. LeadClick , 838 F.3d at 174 (citation and internal quotation marks omitted); accord Oberdorf , 930 F.3d at 151, 2019 WL 2849153, at *10 ; Jane Doe No. 1 v. Backpage.com, LLC , 817 F.3d 12, 19 (1st Cir. 2016) ; Jones v. Dirty World Entm't Recordings LLC , 755 F.3d 398, 407 (6th Cir. 2014) ; Barnes v. Yahoo!, Inc. , 570 F.3d 1096, 1102 (9th Cir. 2009) ; Zeran , 129 F.3d at 330 ; see Klayman v. Zuckerberg , 753 F.3d 1354, 1359 (D.C. Cir. 2014) ; Ben Ezra, Weinstein, & Co., Inc. v. Am. Online Inc. , 206 F.3d 980, 986 (10th Cir. 2000). For instance, a claim against a newspaper based on the content of a classified ad (or the decision to publish or withdraw that ad) would fail under the CDA not because newspapers traditionally publish classified ads, but rather because such a claim would necessarily treat the newspaper as the publisher of the ad-maker's content. Similarly, the newspaper does not act as an "information content provider"-and thus maintains its CDA protection-when it decides to run a classified ad because it neither "creates" nor "develops" the information in the ad. 47 U.S.C. § 230(f)(3).
This case is different. Looking beyond Facebook's "broad statements of immunity" and relying "rather on a careful exegesis of the statutory language," Barnes , 570 F.3d at 1100, the CDA does not protect Facebook's friend- and content-suggestion algorithms. A combination of two factors, in my view, confirms that claims based on these algorithms do not inherently treat Facebook as the publisher of third-party content. First, Facebook uses the algorithms to create and communicate its own message: that it thinks you, the reader-you, specifically-will like this content. And second, Facebook's suggestions contribute to the creation of real-world social networks. The result of at least some suggestions is not just that the user consumes a third party's content. Sometimes, Facebook's suggestions allegedly lead the user to become part of a unique global community, the creation and maintenance of which goes far beyond and differs in kind from traditional editorial functions.
It is true, as the majority notes, see ante , at 70, that Facebook's algorithms rely on and display users' content. However, this is not enough to trigger the protections of § 230(c)(1). The CDA does not mandate "a 'but-for' test that would provide immunity ... solely because a cause of action would not otherwise have accrued but for the third-party content." HomeAway.com, Inc. v. City of Santa Monica , 918 F.3d 676, 682 (9th Cir. 2019). Rather, to fall within § 230(c)(1)'s radius, the claim at issue must inherently fault the defendant's activity as the publisher of specific third-party content. Plaintiffs' claims about Facebook's suggestion algorithms do not do this. The complaint alleges that "Facebook collects detailed information about its users, including, inter alia, the content they post, type of content they view or engage with, people they communicate with, groups they belong to and how they interact with such groups, visits to third party websites, apps and Facebook partners." App'x 345 ¶ 608. Then the algorithms "utilize the collected data to suggest friends, groups, products, services and local events, and target ads" based on each user's input. Id. at 346 ¶ 610.
If a third party got access to Facebook users' data, analyzed it using a proprietary algorithm, and sent its own messages to Facebook users suggesting that people become friends or attend one another's events, the third party would not be protected as "the publisher" of the users' information. Similarly, if Facebook were to use the algorithms to target its own material to particular users, such that the resulting posts consisted of "information provided by" Facebook rather than by "another information content provider," § 230(c)(1), Facebook clearly would not be immune for that independent message.
Yet that is ultimately what plaintiffs allege Facebook is doing. The PSAC alleges that Facebook "actively provides 'friend suggestions' between users who have expressed similar interests," and that it "actively suggests groups and events to users." App'x 346 ¶¶ 612-13. Facebook's algorithms thus allegedly provide the user with a message from Facebook. Facebook is telling users-perhaps implicitly, but clearly-that they would like these people, groups, or events. In this respect, Facebook "does not merely provide a framework that could be utilized for proper or improper purposes; rather, [Facebook's] work in developing" the algorithm and suggesting connections to users based on their prior activity on Facebook, including their shared interest in terrorism, "is directly related to the alleged illegality of the site." Fair Housing Council of San Fernando Valley v. Roommates.Com, LLC , 521 F.3d 1157, 1171 (9th Cir. 2008) (en banc). The fact that Facebook also publishes third-party content should not cause us to conflate its two separate roles with respect to its users and their information. Facebook may be immune under the CDA from plaintiffs' challenge to its allowance of Hamas accounts, since Facebook acts solely as the publisher of the Hamas users' content. That does not mean, though, that it is also immune when it conducts statistical analyses of that information and delivers a message based on those analyses.
Moreover, in part through its use of friend, group, and event suggestions, Facebook is doing more than just publishing content: it is proactively creating networks of people. Its algorithms forge real-world (if digital) connections through friend and group suggestions, and they attempt to create similar connections in the physical world through event suggestions. The cumulative effect of recommending several friends, or several groups or events, has an impact greater than the sum of each suggestion. It envelops the user, immersing her in an entire universe filled with people, ideas, and events she may never have discovered on her own. According to the allegations in the complaint, Facebook designed its website for this very purpose. "Facebook has described itself as a provider of products and services that enable users ... to find and connect with other users ...." App'x 250 ¶ 129. CEO Mark Zuckerberg has similarly described Facebook as "build[ing] tools to help people connect with the people they want," thereby "extending people's capacity to build and maintain relationships." Id. at 251 ¶ 132. Of course, Facebook is not the only company that tries to bring people together this way, and perhaps other publishers try to introduce their readers to one another. Yet the creation of social networks goes far beyond the traditional editorial functions that the CDA immunizes.
Another way to consider the CDA immunity question is to "look ... to what the duty at issue actually requires: specifically, whether the duty would necessarily require an internet company to monitor[, alter, or remove] third-party content." HomeAway.com , 918 F.3d at 682. Here, too, the claims regarding the algorithms are a poor fit for statutory immunity. The duty not to provide material support to terrorism, as applied to Facebook's use of the algorithms, simply requires that Facebook not actively use that material to determine which of its users to connect to each other. It could stop using the algorithms altogether, for instance. Or, short of that, Facebook could modify its algorithms to stop them introducing terrorists to one another. None of this would change any underlying content, nor would it necessarily require courts to assess further the difficult question of whether there is an affirmative obligation to monitor that content.
In reaching this conclusion, I note that ATA torts are atypical. Most of the common torts that might be pleaded in relation to Facebook's algorithms "derive liability from behavior that is identical to publishing or speaking"-for instance, "publishing defamatory material; publishing material that inflicts emotional distress; or ... attempting to de-publish hurtful material but doing it badly." Barnes , 570 F.3d at 1107.
The fact that Facebook has figured out how to target material to people more likely to read it does not matter to a defamation claim, for instance, because the mere act of publishing in the first place creates liability.
The ATA works differently. Plaintiffs' material support and aiding and abetting claims premise liability, not on publishing qua publishing, but rather on Facebook's provision of services and personnel to Hamas. It happens that the way in which Facebook provides these benefits includes republishing content, but Facebook's duties under the ATA arise separately from the republication of content. Cf. id. (determining that liability on a promissory estoppel theory for promising to remove content "would come not from Yahoo's publishing conduct, but from Yahoo's manifest intention to be legally obligated to do something, which happens to be removal of material from publication"). For instance, the operation of the algorithms is allegedly provision of "expert advice or assistance," and the message implied by Facebook's prodding is allegedly a "service" or an attempt to provide "personnel." 18 U.S.C. § 2339A(b).
For these reasons, § 230(c)(1) does not bar plaintiffs' claims.
III.
Even if we sent this case back to the district court, as I believe to be the right course, these plaintiffs might have proven unable to allege that Facebook's matchmaking algorithms played a role in the attacks that harmed them. However, assuming arguendo that such might have been the situation here, I do not think we should foreclose the possibility of relief in future cases if victims can plausibly allege that a website knowingly brought terrorists together and that an attack occurred as a direct result of the site's actions. Though the majority shuts the door on such claims, today's decision also illustrates the extensive immunity that the current formulation of the CDA already extends to social media companies for activities that were undreamt of in 1996. It therefore may be time for Congress to reconsider the scope of § 230.
As is so often the case with new technologies, the very qualities that drive social media's success-its ease of use, open access, and ability to connect the world-have also spawned its demons. Plaintiffs' complaint illustrates how pervasive and blatant a presence Hamas and its leaders have maintained on Facebook. Hamas is far from alone-Hezbollah, Boko Haram, the Revolutionary Armed Forces of Colombia, and many other designated terrorist organizations use Facebook to recruit and rouse supporters. Vernon Silver & Sarah Frier, Terrorists Are Still Recruiting on Facebook, Despite Zuckerberg's Reassurances , Bloomberg Businessweek (May 10, 2018), http://www.bloomberg.com/news/articles/2018-05-10/terrorists-creep-onto-facebook-as-fast-as-it-can-shut-them-down. Recent news reports suggest that many social media sites have been slow to remove the plethora of terrorist and extremist accounts populating their platforms, and that such efforts, when they occur, are often underinclusive. Twitter, for instance, banned the Ku Klux Klan in 2018 but allowed David Duke to maintain his account, see Roose & Conger, supra , while researchers found that Facebook removed fewer than half the terrorist accounts and posts those researchers identified, see Waters & Postings, supra , at 8; Desmond Butler & Barbara Ortulay, Facebook Auto-Generates Videos Celebrating Extremist Images , Assoc. Press (May 9, 2019), http://apnews.com/f97c24dab4f34bd0b48b36f2988952a4. Those whose accounts are removed often pop up again under different names or with slightly different language in their profiles, playing a perverse and deadly game of Whack-a-Mole with Silicon Valley. See Isaac, supra ; Silver & Frier, supra .
Of course, the failure to remove terrorist content, while an important policy concern, is immunized under § 230 as currently written. Until today, the same could not have been said for social media's unsolicited, algorithmic spreading of terrorism. Shielding internet companies that bring terrorists together using algorithms could leave dangerous activity unchecked.
Take Facebook. As plaintiffs allege, its friend-suggestion algorithm appears to connect terrorist sympathizers with pinpoint precision. For instance, while two researchers were studying Islamic State ("IS") activity on Facebook, one "received dozens of pro-IS accounts as recommended friends after friending just one pro-IS account." Waters & Postings, supra , at 78. More disturbingly, the other "received an influx of Philippines-based IS supporters and fighters as recommended friends after liking several non-extremist news pages about Marawi and the Philippines during IS's capture of the city." Id. News reports indicate that the friend-suggestion feature has introduced thousands of IS sympathizers to one another. See Martin Evans, Facebook Accused of Introducing Extremists to One Another Through 'Suggested Friends' Feature , The Telegraph (May 5, 2018), http://www.telegraph.co.uk/news/2018/05/05/facebook-accused-introducing-extremists-one-another-suggested.
And this is far from the only Facebook algorithm that may steer people toward terrorism. Another turns users' declared interests into audience categories to enable microtargeted advertising. In 2017, acting on a tip, ProPublica sought to direct an ad at the algorithmically-created category "Jew hater"-which turned out to be real, as were "German Schutzstaffel," "Nazi Party," and "Hitler did nothing wrong." Julia Angwin et al., Facebook Enabled Advertises to Reach 'Jew Haters ,' ProPublica (Sept. 14, 2017), https://www.propublica.org/article/facebookenabled-advertisers-to-reach-jew-haters. As the "Jew hater" category was too small for Facebook to run an ad campaign, "Facebook's automated system suggested 'Second Amendment' as an additional category ... presumably because its system had correlated gun enthusiasts with anti-Semites." Id.
That's not all. Another Facebook algorithm auto-generates business pages by scraping employment information from users' profiles; other users can then "like" these pages, follow their posts, and see who else has liked them. Butler & Ortutay, supra . ProPublica reports that extremist organizations including al-Qaida, al-Shabab, and IS have such auto-created pages, allowing them to recruit the pages' followers. Id. The page for al-Qaida in the Arabian Peninsula included the group's Wikipedia entry and a propaganda photo of the damaged USS Cole, which the group had bombed in 2000. Id. Meanwhile, a fourth algorithm integrates users' photos and other media to generate videos commemorating their previous year. Id. Militants get a ready-made propaganda clip, complete with a thank-you message from Facebook. Id.
This case, and our CDA analysis, has centered on the use of algorithms to foment terrorism. Yet the consequences of a CDA-driven, hands-off approach to social media extend much further. Social media can be used by foreign governments to interfere in American elections. For example, Justice Department prosecutors recently concluded that Russian intelligence agents created false Facebook groups and accounts in the years leading up to the 2016 election campaign, bootstrapping Facebook's algorithm to spew propaganda that reached between 29 million and 126 million Americans. See 1 Robert S. Mueller III, Special Counsel, Report on the Investigation Into Russian Interference in the 2016 Presidential Election 24-26, U.S. Dep't of Justice (March 2019), http://www.justice.gov/storage/report.pdf. Russia also purchased over 3,500 advertisements on Facebook to publicize their fake Facebook groups, several of which grew to have hundreds of thousands of followers. Id. at 25-26. On Twitter, Russia developed false accounts that impersonated American people or groups and issued content designed to influence the election; it then created thousands of automated "bot" accounts to amplify the sham Americans' messages. Id. at 26-28. One fake account received over six million retweets, the vast majority of which appear to have come from real Twitter users. See Gillian Cleary, Twitterbots: Anatomy of a Propaganda Campaign , Symantec (June 5, 2019), http://www.symantec.com/blogs/threat-intelligence/twitterbots-propaganda-disinformation. Russian intelligence also harnessed the reach that social media gave its false identities to organize "dozens of U.S. rallies," some of which "drew hundreds" of real-world Americans. Mueller, Report , supra , at 29. Russia could do all this only because social media is designed to target messages like Russia's to the users most susceptible to them.
While Russia's interference in the 2016 election is the best-documented example of foreign meddling through social media, it is not the only one. Federal intelligence agencies expressed concern in the weeks before the 2018 midterm election "about ongoing campaigns by Russia, China and other foreign actors, including Iran," to "influence public sentiment" through means "including using social media to amplify divisive issues." Press Release, Office of Dir. of Nat'l Intelligence, Joint Statement from the ODNI, DOJ, FBI, and DHS: Combatting Foreign Influence in U.S. Elections, (Oct. 19, 2018), https://www.dni.gov/index. php/newsroom/press-releases/item/1915-joint-statement-from-the-odni-doj-fbi-and-dhs-combating-foreign-influence-in-u-s-elections. News reports also suggest that China targets state-sponsored propaganda to Americans on Facebook and purchases Facebook ads to amplify its communications. See Paul Mozur, China Spreads Propaganda to U.S. on Facebook, a Platform It Bans at Home , N.Y. Times (Nov. 8, 2017), https://www.nytimes.com/2017/11/08/technology/china-facebook.html.
Widening the aperture further, malefactors at home and abroad can manipulate social media to promote extremism. "Behind every Facebook ad, Twitter feed, and YouTube recommendation is an algorithm that's designed to keep users using: It tracks preferences through clicks and hovers, then spits out a steady stream of content that's in line with your tastes." Katherine J. Wu, Radical Ideas Spread Through Social Media. Are the Algorithms to Blame? , PBS (Mar. 28, 2019), https://www.pbs.org/wgbh/nova/article/radical-ideas-social-media-algorithms. All too often, however, the code itself turns those tastes sour. For example, one study suggests that manipulation of Facebook's news feed influences the mood of its users: place more positive posts on the feed and users get happier; focus on negative information instead and users get angrier. Adam D. I. Kramer et al., Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks , 111 PNAS 8788, 8789 (2014). This can become a problem, as Facebook's algorithm "tends to promote the most provocative content" on the site. Max Fisher, Inside Facebook's Secret Rulebook for Global Political Speech , N.Y. Times (Dec. 27, 2018), http://www.nytimes.com/2018/12/27/world/facebook-moderators.html. Indeed, "[t]he Facebook News Feed environment brings together, in one place, many of the influences that have been shown to drive psychological aspects of polarization." Jaime E. Settle, Frenemies: How Social Media Polarizes America (2018). Likewise, YouTube's video recommendation algorithm-which leads to more than 70 percent of time people spend on the platform-has been criticized for shunting visitors toward ever more extreme and divisive videos. Roose & Conger, supra ; see Jack Nicas, How YouTube Drives People to the Internet's Darkest Corners , Wall St. J. (Feb. 7, 2018), https://www.wsj.com/articles/how-youtube-drives-viewers-to-the-internets-darkest-corners-1518020478. YouTube has fine-tuned its algorithm to recommend videos that recalibrate users' existing areas of interest and steadily steer them toward new ones-a modus operandi that has reportedly proven a real boon for far-right extremist content. See Kevin Roose, The Making of a YouTube Radical , N.Y. Times (June 8, 2019), http://www.nytimes.com/interactive/2019/06/08/technology/youtube-radical.html.
There is also growing attention to whether social media has played a significant role in increasing nationwide political polarization. See Andrew Soergel, Is Social Media to Blame for Political Polarization in America? , U.S. News & World Rep. (Mar. 20, 2017), https://www.usnews.com/news/articles/2017-03-20/is-social-media-to-blame-for-political-polarization-in-america. The concern is that "web surfers are being nudged in the direction of political or unscientific propaganda, abusive content, and conspiracy theories." Wu, Radical Ideas , supra . By surfacing ideas that were previously deemed too radical to take seriously, social media mainstreams them, which studies show makes people "much more open" to those concepts. Max Fisher & Amanda Taub, How Everyday Social Media Users Become Real-World Extremists , N.Y. Times (Apr. 25, 2018), http://www.nytimes.com/2018/04/25/world/asia/facebook-extremism.html. At its worst, there is evidence that social media may even be used to push people toward violence. The sites are not entirely to blame, of course-they would not have such success without humans willing to generate and to view extreme content. Providers are also tweaking the algorithms to reduce their pull toward hate speech and other inflammatory material. See Isaac, supra ; Roose & Conger, supra . Yet the dangers of social media, in its current form, are palpable.
While the majority and I disagree about whether § 230 immunizes interactive computer services from liability for all these activities or only some, it is pellucid that Congress did not have any of them in mind when it enacted the CDA. The text and legislative history of the statute shout to the rafters Congress's focus on reducing children's access to adult material. Congress could not have anticipated the pernicious spread of hate and violence that the rise of social media likely has since fomented. Nor could Congress have divined the role that social media providers themselves would play in this tale. Mounting evidence suggests that providers designed their algorithms to drive users toward content and people the users agreed with-and that they have done it too well, nudging susceptible souls ever further down dark paths. By contrast, when the CDA became law, the closest extant ancestor to Facebook (and it was still several branches lower on the evolutionary tree) was the chatroom or message forum, which acted as a digital bulletin board and did nothing proactive to forge off-site connections.
Whether, and to what extent, Congress should allow liability for tech companies that encourage terrorism, propaganda, and extremism is a question for legislators, not judges. Over the past two decades "the Internet has outgrown its swaddling clothes," Roommates.Com , 521 F.3d at 1175 n.39, and it is fair to ask whether the rules that governed its infancy should still oversee its adulthood. It is undeniable that the Internet and social media have had many positive effects worth preserving and promoting, such as facilitating open communication, dialogue, and education. At the same time, as outlined above, social media can be manipulated by evildoers who pose real threats to our democratic society. A healthy debate has begun both in the legal academy and in the policy community about changing the scope of § 230. Perhaps Congress will clarify what I believe the text of the provision already states: that the creation of social networks reaches beyond the publishing functions that § 230 protects. Perhaps Congress will engage in a broader rethinking of the scope of CDA immunity. Or perhaps Congress will decide that the current regime best balances the interests involved. In the meantime, however, I cannot join my colleagues' decision to immunize Facebook's friend- and content-suggestion algorithms from judicial scrutiny. I therefore must in part respectfully dissent, as I concur in part.
I agree with the majority that the CDA's exception for enforcement of criminal laws, 47 U.S.C. § 230(e)(1), does not apply to plaintiffs' claims, see ante , at 71-72. However, I find the question to be somewhat closer than the majority does, in part because some of the statutes enumerated in § 230(e)(1) themselves contain civil remedies. Section 230(e)(1) states that "[n]othing in [§ 230 ] shall be construed to impair the enforcement of section 223 or 231 of this title, chapter 71 (relating to obscenity) or 110 (relating to sexual exploitation of children) of title 18, or any other Federal criminal statute." One of those enumerated chapters-Chapter 110 of Title 18-includes a civil suit provision for victims of specific child sex crimes. See 18 U.S.C. § 2255. Meanwhile, 47 U.S.C. § 223 -which prohibits obscene or harassing phone calls-specifies that civil fines may be levied "pursuant to civil action by," or "after appropriate administrative proceedings" of, the Federal Communications Commission ("FCC"), and it authorizes the Attorney General to bring civil suits to enjoin practices that violate the statute. 47 U.S.C. § 223(b)(5)(B) -(b)(6). If § 230(e)(1) covers "enforcement" of the listed chapters in their entirety, it is difficult to see how it would not cover other provisions that authorize civil suits for violations of criminal laws, particularly given that the enumerated list is followed by "or any other criminal law."
However, as detailed post , § 230 was designed as a private-sector-driven alternative to a Senate plan that would allow the FCC "either civilly or criminally, to punish people" who put objectionable material on the Internet. 141 Cong. Rec. 22,045 (1995) (statement of Rep. Cox); accord id. at 22,045-46 (statement of Rep. Wyden); see Reno v. ACLU , 521 U.S. 844, 859 & n.24, 117 S.Ct. 2329, 138 L.Ed.2d 874 (1997). On the House floor, author Christopher Cox disparaged the idea of FCC enforcement and then stated: "Certainly, criminal enforcement of our obscenity laws as an adjunct is a useful way of punishing the truly guilty." 141 Cong. Rec. 22,045 (emphasis added). This history, along with the provision's title, strongly suggests that § 230(e)(1) was intended as a narrow criminal-law exception. It would be odd, then, to read § 230(e)(1) as allowing for civil enforcement by, among others, the FCC, even if only in aid of criminal law enforcement.
It helped that the Cox-Wyden Amendment exempted from its deregulatory regime the very provisions that the Exon Amendment strengthened, see Telecommunications Act of 1996, §§ 502, 507-508, 509(d)(1), 110 Stat. at 133-39, and that Congress stripped from the House bill a provision that would have denied jurisdiction to the FCC to regulate the Internet, compare id. § 509, 110 Stat. at 138 (eliminating original § 509(d)), with 141 Cong. Rec. 22,044 (including original § 509(d)).
The policy section of the statute also expresses Congress's desire "to preserve the vibrant and competitive free market that presently exists for the Internet and other interactive computer services, unfettered by Federal or State regulation." 47 U.S.C. § 230(b)(2). It is therefore true that "Section 230 was enacted, in part , to maintain the robust nature of Internet communication." Ricci v. Teamsters Union Local 456 , 781 F.3d 25, 28 (emphasis added) (quoting Zeran v. Am. Online, Inc. , 129 F.3d 327, 330 (4th Cir. 1997) ); see ante , at 63. As the legislative history laid out in this opinion shows, however, one cannot fully understand the purpose of § 230 without considering that it was one chamber's proposal in a disagreement between the two houses of Congress over how best to shield children from indecent material, and that in that contest the House was principally concerned with two things: (1) overruling Stratton-Oakmont and (2) preventing "a Federal Computer Commission with an army of bureaucrats regulating the Internet." 141 Cong. Rec. 22,045 (statement of Rep. Cox).
This point-that Congress chose broader language than may have been necessary to accomplish its primary goal-should not be confused with the Seventh Circuit's rationale for § 230(c)(1)'s general application: that "a law's scope often differs from its genesis." See Chi. Lawyers' Cmte. for Civil Rights Under Law, Inc. v. Craigslist, Inc. , 519 F.3d 666, 671 (7th Cir. 2008). True as this axiom might be, it does not apply here-the language of § 230(c)(1) remained untouched from introduction to passage. Nor is there any evidence from the legislative record that interest groups altered the statutory language. But cf. id. ("Once the legislative process gets rolling, interest groups seek (and often obtain) other provisions."). That § 230(c)(1)'s breadth flowed from Congress's desire to overrule Stratton-Oakmont , rather than from mere interest group protectionism, matters.
Many of Facebook's algorithms mentioned in the PSAC, such as its third-party advertising algorithm, its algorithm that places content in a user's newsfeed, and (based on the limited description in the PSAC) its video recommendation algorithm, remain immune under the analysis I set out here.
See, e.g. , Gregory Waters & Robert Postings, Spiders of the Caliphate: Mapping the Islamic State's Global Support Network on Facebook 8, Counter Extremism Project (May 2018), http://www.counterextremism.com/sites/default/files/Spiders%20of%20the%20Caliphate%20%28May%202018%29.pdf; Yaacov Benmeleh & Felice Maranz, Israel Warns Twitter of Legal Action Over Requests to Remove Content , Bloomberg (Mar. 20, 2018), http://www.bloomberg.com/news/articles/2018-03-20/israel-warns-twitter-of-legal-steps-over-incitement-to-terrorism; Mike Isaac, Twitter Steps Up Efforts to Thwart Terrorists' Tweets , N.Y. Times (Feb. 5, 2016), http://www.nytimes.com/2016/02/06/technology/twitter-account-suspensions-terrorism.html; Kevin Roose & Kate Conger, YouTube to Remove Thousands of Videos Pushing Extreme Views , N.Y. Times (June 5, 2019), http://www.nytimes.com/2019/06/05/business/youtube-remove-extremist-videos.html.
See, e.g. , Sarah Marsh, Social Media Related to Violence by Young People, Say Experts , The Guardian (Apr. 2, 2018), https://www.theguardian.com/media/2018/apr/02/social-media-violence-young-people-gangs-say-experts; Kevin Roose, A Mass Murder of, and for, the Internet , N.Y. Times (Mar. 15, 2019), https://www.nytimes.com/2019/03/15/technology/facebook-youtube-christchurch-shooting.html; Craig Timberg et al., The New Zealand Shooting Shows How TouTube and Facebook Spread Hate and Violent Images-Yet Again , Wash. Post (Mar. 15, 2019), https://www.washingtonpost.com/technology/2019/03/15/facebook-youtube-twitter-amplified-video-christchurch-mosque-shooting; Julie Turkewitz & Kevin Roose, Who Is Robert Bowers, the Suspect in the Pittsburgh Synagogue Shooting? , N.Y. Times (Oct. 27, 2018), https://www.nytimes.com/2018/10/27/us/robert-bowers-pittsburgh-synagogue-shooter.html.
See Caitlin Dewey, A Complete History of the Rise and Fall-and Reincarnation!-of the Beloved '90s Chatroom , Wash. Post (Oct. 30, 2014), http://www.washingtonpost.com/ news/the-intersect/wp/2014/10/30/a-complete-history-of-the-rise-and-fall-and-reincarnation-of-the-beloved-90s-chatroom; see also Then and Now: A History of Social Networking Sites , CBS News, http://www.cbsnews.com/pictures/then-and-now-a-history-of-social-networking-sites (last accessed July 9, 2019) (detailing the evolution of social media sites from Classmates, launched only "as a list of school affiliations" in December 1995; to "the very first social networking site" Six Degrees, which launched in May 1997 but whose networks were limited "due to the lack of people connected to the Internet"; to Friendster, launched in March 2002 and "credited as giving birth to the modern social media movement"; to Facebook, which was "rolled out to the public in September 2006").
See, e.g. , Danielle Keats Citron & Benjamin Wittes, The Problem Isn't Just Backpage: Revising Section 230 Immunity , 2 Geo. L. Tech. Rev. 453, 454-55 (2018) ; Jeff Kosseff, Defending Section 230: The Value of Intermediary Immunity , 15 J. Tech. L. & Pol'y 123, 124 (2010) ; Daniela C. Manzi, Managing the Misinformation Marketplace: The First Amendment and the Fight Against Fake News , 87 Fordham L. Rev. 2623, 2642-43 (2019). Much of the enterprising legal scholarship debating the intersection of social media, terrorism, and the CDA comes from student Notes. See, e.g. , Jaime E. Freilich, Note, Section 230's Liability Shield in the Age of Online Terrorist Recruitment , 83 Brook. L. Rev. 675, 690-91 (2018) ; Anna Elisabeth Jayne Goodman, Note and Comment, When You Give a Terrorist a Twitter: Holding Social Media Companies Liable for their Support of Terrorism , 46 Pepp. L. Rev. 147, 182-86 (2018) ; Nicole Phe, Note, Social Media Terror: Reevaluating Intermediary Liability Under the Communications Decency Act , 51 Suffolk U. L. Rev. 99, 126-30 (2018).
See, e.g. , Tarleton Gillespie, How Social Networks Set the Limits of What We Can Say Online , Wired (June 26, 2018), http://www.wired.com/story/how-social-networks-set-the-limits-of-what-we-can-say-online; Christiano Lima, How a Widening Political Rift Over Online Liability Is Splitting Washington , Politico (July 9, 2019), http://www.politico.com/story/2019/07/09/online-industry-immunity-section-230-1552241; Mark Sullivan, The 1996 Law That Made the Web Is in the Crosshairs , Fast Co. (Nov. 29, 2018), http://www.fastcompany.com/90273352/maybe-its-time-to-take-away-the-outdated-loophole-that-big-tech-exploits; cf. Darrell M. West & John R. Allen, How Artificial Intelligence Is Transforming the World , Brookings (Apr. 24, 2018), http://www.brookings.edu/research/how-artificial-intelligence-is-transforming-the-world ("The malevolent use of AI exposes individuals and organizations to unnecessary risks and undermines the virtues of the emerging technology.").

Passage view · GavelSight