Social Dilemma (film)

A Netflix documentary released in September 2020.

Overview

Jef Raskin, one of Apple’s early designers, argued that Isaac Asimov’s First Law of Robotics should also apply to computers.

“A computer shall not harm your work or, through inaction, allow your work to come to harm.” —Chapter 1, The Humane Interface

Originally a principle governing robots’ relationship with humans, Raskin adapted it to computers’ relationship with work. Unlike imaginary robots, computers then were not agents actively acting with some “intention.” Now, however, computers permeate everyday life so deeply that they cannot be separated from it, and many services and software products are actively planned to influence human thought and behavior. In modern society, perhaps we must seriously consider “harm to humans by computers,” as in the original principle.

“The Social Dilemma,” released by Netflix in September 2020, exposes the designed harmful effects of social media such as Facebook, Instagram, and Twitter on people and society. I think it offers much food for thought to UX designers, data analysts, performance marketers, and software developers, so I briefly summarized it and collected some related thoughts.

Interestingly, Jef Raskin’s son Aza Raskin appears in the documentary.

A brief summary

The film alternates between documentary interviews, mainly with former employees of major social media companies—Google, Meta, Instagram (acquired by Meta), Twitter, and YouTube (acquired by Google)—and a small family drama performed by actors. The drama seems intended to convey the documentary’s arguments persuasively.

Its central criticism can be summarized as follows.

To make money, social media companies squeeze users’ attention, inducing them to keep using their services and seeing ads. In the process, they collect and exploit usage patterns, including personal information. Repeating and refining this process causes various personal and social problems, for which the companies take no responsibility.

Some memorable points, summarized and excerpted:

“There has never been a system in history in which a small team of a dozen or so mostly white men in their twenties and thirties living in California makes decisions affecting the world’s population with almost no restrictions.” (Describing Google’s Gmail design team around 2013.)

“Ray Kurzweil’s singularity is the point when computers surpass human strengths. That time has not yet arrived, but the point when computers surpass human weaknesses has already come. Social media relentlessly targets weaknesses in human psychology to exploit users’ attention and extract advertising revenue.”

“If you’re not paying for the product, you are the product. Social media sells user data to advertisers. The user is the product.”

Let me elaborate on the documentary’s argument.

Most social networking services are free. They primarily make money by brokering advertising, and do three main things to increase return on advertising spend.

Understand users’ inclinations and behavior patterns: They analyze what kinds of posts users read, which they react to with likes or anger, whom they interact with frequently, and where and when they usually connect. To understand these patterns better, they conduct various experiments, sometimes explicitly and sometimes implicitly.
Generate revenue through targeted advertising: Based on this data, they develop and provide analytics and automation tools enabling advertisers to target users with particular inclinations. Advertisers use these services to show targeted ads, paying the social media company in return. Advertising fees generally constitute social media revenue.
Encourage more frequent, longer use: User data is used not only for advertising, but also to encourage people to use the platform as frequently and for as long as possible. Various forms of personalization determine when to send which push notifications, and which updates to show in what order when users connect. More frequent, longer use creates more opportunities to show ads and learn about users.

Described this way, it may seem harmless. But examining the negative implications of each point changes the impression substantially.

Understand users’ inclinations and behavior patterns: For example, Facebook and Google plant identifying codes to track people’s activities on other websites even when they are not using the companies’ services. Persistent privacy concerns have prompted stronger regulations in recent years (such as the EU’s GDPR and California’s CCPA) and stronger protections in some browsers (such as Brave and Safari), but these responses remain inadequate. For example, Google has already spent years developing a new tracking system that stitches together fragments of data gathered through various methods in preparation for losing the ability to track users through third-party cookies.
Generate revenue through targeted advertising: “Advertising” here includes not only product ads but also fake news created by one country to cause political turmoil in another, and conspiracy theories (the climate crisis is a lie, coronavirus is a hoax, and so on). Fake news and conspiracy theories spread faster than truth, making this worse. There is also a case in which a company collected user data “without consent” through Facebook’s API to influence a U.S. presidential election. Recent efforts to address these problems and strengthen privacy policies often seem like temporary measures rather than fundamental solutions. For an example, look at the length of Wikipedia’s article collecting criticisms of Facebook.
Encourage more frequent, longer use: Some data collection and experiments designed to capture more user attention would not be permitted at universities for ethical reasons. Features deliberately designed to addict people resemble—and in some respects surpass—techniques long studied and applied by the gambling industry, yet regulation and public awareness are minimal by comparison (see History of deceptive patterns). Moreover, nearly every advertising-funded service competes for user attention. Unless that structure changes, services cannot escape the arms race to design ever more persistent, powerful mechanisms, and the damage falls on individual users and society as a whole.

The documentary emphasizes that on the other side of the little phone screen we look at are algorithms designed to extract our attention more effectively, vast servers running those algorithms, and countless specialists (UX designers, data analysts, performance marketers, and software developers). It is an asymmetrical environment overwhelmingly stacked against users.

For example, the image below shows some of the hundreds of tags Google assigned to classify me (those beginning with A). You can visit Google’s ad settings site to see how Google perceives you. If you consented to Google’s ad targeting, this information is provided in anonymized form to advertisers and publishers through Google Analytics, Google AdWords, and similar services.

Most people probably consented to sharing information without realizing it. Obtaining consent in this way is itself a product of clever design.

In traditional human–computer interaction, an interface helped people do what they wanted to do better. Social media differs substantially from traditional tools by actively inducing and intervening in human behavior.

Facebook’s response to “The Social Dilemma”

Shortly after the documentary’s release, Facebook published <What ‘The Social Dilemma’ Gets Wrong>.

It argues that the film blames social media platforms for complex social problems and includes only the views of former employees, failing to reflect Facebook’s recent efforts. Here is a critical summary of its seven topics.

“First, Facebook builds products to create value, not addiction.” The document stresses that Facebook does not intend to induce addiction and is making various efforts to improve users’ mental health. That is welcome. But I think intentions and outcomes also need to be distinguished. Every design produces intended outcomes and unavoidable byproducts independent of intent. A light bulb is designed to emit light, for example, but also generates heat regardless of intent. Reducing heat requires deliberate additional effort. I wish the statement had acknowledged the problem and described efforts to solve it, rather than merely denying intent.

“Second, you are not the product. Facebook funds free access through advertising.” The document says Facebook sells ads, not users, and does not provide personal information in the process. But considering that “the user is the product” is a metaphorical expression, responding that it sells ads rather than users looks like a straw man. It also says that “unless users consent, we do not give advertisers information that identifies individual users.” This needs two qualifications.

In other words, if users consent, advertisers do receive personally identifiable information. One might think users can simply withhold consent, but we must also consider the many ways design can easily elicit it.
Whether information is PII (personally identifiable information) cannot be treated as a dichotomy, because techniques for identifying individuals by combining “non-identifying” information continue to improve.

“Third, Facebook’s algorithm isn’t ‘mad.’” According to the document, other services use similar algorithms to provide useful features, and Facebook is no different. For example, many people probably watched “The Social Dilemma” because Netflix’s algorithm recommended it. I think this is a fair point. But companies need to consider more seriously whether all these algorithms cause unintended problems. Many news-personalization algorithms recommend stories readers are likely to click. The design intent may be more clicks, longer visits, and higher ad revenue, but a byproduct can easily be stronger confirmation bias. As above, we need to examine not only intentions but outcomes, especially design byproducts.

“Fourth, Facebook works to protect your privacy.” The document emphasizes safeguards introduced following a Federal Trade Commission investigation. Although it does not specify this, the “investigation” concerns the Facebook–Cambridge Analytica data scandal. Facebook paid a fine of about $5 billion (roughly 5.5 trillion Korean won). Cambridge Analytica collected large amounts of personal data through Facebook using methods permitted then but prohibited now, and used it for various political purposes, including helping elect Trump in the 2016 U.S. presidential election. If this interests you, I recommend another Netflix documentary, <The Great Hack>.

“Fifth, Facebook works to address polarization.” The document says polarization predates Facebook and that Facebook is making various efforts to address it. Those efforts deserve praise, of course. But “it already existed” seems inappropriate. People’s concern is not that social media created a previously nonexistent problem, but that it amplified an existing one.

“Sixth, Facebook works to ensure fair elections.” It acknowledges mistakes in 2016 and describes subsequent improvements. The “mistakes” refer to the Facebook–Cambridge Analytica scandal. This is the first acknowledgment of a problem. I wish the other sections had been written similarly.

“Seventh, Facebook has built a global fact-checking network to combat false information and fake news.” Facebook says it does not ignore false information or hate speech and is making efforts including developing algorithms and building a fact-checking network. It acknowledges that an imperfect system misses things. I consider these efforts very important and warmly welcome them. However, as the Stanford Internet Observatory’s Renée DiResta points out, they are too heavily concentrated on English and a few Western languages. The reproduction and amplification of inequality through data and algorithms is probably a problem throughout IT, not just social media. A good topic for another article.

Expressions that minimize or deny problems or shift responsibility elsewhere trouble me. Still, the many efforts to address the issues raised are very welcome. I intend to watch supportively.

Keeping vigilance from becoming conspiracy thinking

The documentary sometimes portrays social media companies as having malicious intentions. I think these elements need to be received cautiously. To keep vigilance about social media from turning into conspiracy thinking, these perspectives may help.

Bad outcomes do not always come from bad intentions. Dismissing social media companies as an “axis of evil” cannot lead to constructive discussion.
Slow-looking progress from outside does not mean nobody inside is trying. Criticism of wrong directions should accompany encouragement and support for positive changes. Facebook, for example, continues investing in technology to stop fake news and technology to stop hate speech. Google is also experimenting with technology for targeted advertising that minimizes privacy intrusion.
Almost everything has multiple sides. “Personalization,” for instance, can mean privacy intrusion, spam, or exploitation of attention, but can also provide useful, convenient features and information. “What should we be more careful about when doing X?” is more constructive than “Let’s not do X.”

Conclusion: Technology for humans

Near the end, the documentary quotes a scene from Steve Jobs’s 1980 interview. Jobs says:

I remember reading a magazine article when I was about twelve, probably Scientific American. It measured locomotion efficiency for various species on Earth, in calories consumed moving from one place to another. The condor came first; humans ranked far below. But someone creatively calculated the efficiency of “a human riding a bicycle,” too. The cycling human beat the condor by an enormous margin. This left a powerful impression on me. We humans make tools. By making tools, we can enormously amplify our innate abilities. In this context, to me, computers have always been bicycles for the mind: tools that greatly amplify our innate intelligence. And we are only just getting started.

Computers can profoundly affect humans. Their influence can be profoundly positive or profoundly negative. If you are a growth hacker, UX designer, data analyst, or performance marketer, you are positioned to influence that direction at least in part.

The “bicycle for the mind” metaphor connects naturally with the research tradition of embodied/embedded cognition. This tradition argues that the “mind” is not confined to the brain. Humans are not floating brains but agents with particular bodies (embodied), and those bodies are always situated in particular social and environmental contexts (embedded/situated).

From this perspective, designing means designing part of a user’s mind. To exaggerate, designers’ work may resemble neurosurgeons’ work, differing chiefly in its constant influence on a much larger public.

We need to think more seriously about design ethics.