Computers as Bad Social Actors

Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially

Abstract

The Computers Are Social Actors paradigm suggests people exhibit social/anthropomorphic biases in their treatment of technology. Such insights have encouraged the design of interfaces that interact with users in more social (chatty or even friend-like) ways. However, in typical dark pattern fashion, social-emotional responses to systems as (seemingly sentient) agents can be harnessed to manipulate user behaviour. An increasingly common example is app notifications that assume person-like tones to persuade or pressure users into compliance. Regardless of being manipulative, difficulties meeting contextual social expectations can make automated social acting seem rude, invasive, tactless, and even disrespectful - constituting ‘social’ anti-patterns. This paper explores ways to improve how automated systems treat people in interactions. We mixed four qualitative methods to elicit user experiences and preferences regarding how automated systems “talk” to/at them. We identify an emerging ‘social’ class of dark and anti-patterns, and propose guidelines for helping (social) interfaces treat users in more respectful, tactful, and autonomy-supportive ways.

According to the CASA paradigm, people tend to anthropomorphize computers. This tendency is often exploited through deceptive patterns or takes the form of anti-patterns that unintentionally cause rudeness and similar problems.
This paper analyzes how systems “talk” to users and offers guidelines for designing them to treat users more respectfully.

1. Introduction

Overview:

Concern about dark patterns has grown over the past decade, with conversational push notifications particularly common.
Violating subtle social conventions can unintentionally make a UI rude (an anti-pattern).
Respectful interfaces have been discussed, but specific contextual/design factors have not been explicitly analyzed.

Research questions:

RQ1: Which social design patterns do people find manipulative or autonomy-infringing, and why?
RQ2: What kinds of social design patterns do people find unpleasant, and why?
RQ3: What criteria define people’s preferred ways for systems to “talk”?
RQ4: What contextual factors influence these preferences?

Contributions:

Identifies and characterizes an emerging category of social dark patterns.
Uses qualitative research to analyze when and why the CASA paradigm takes the form of social anti-patterns.
Offers guidelines for preventing anti-patterns.
Suggests what “respect” means for social interfaces, in connection with its recent introduction as a value in HCI.

2. Background and Related Work

2.1. When is social acting (in)appropriate?

2.2. Distinguishing nudges, dark and anti-patterns

Nudge: Originally an ethical, paternalistic intervention that neither restricts choices at all nor significantly changes economic incentives.
Dark patterns: Strategies that use nudges, coercion, and other means to make users do things they otherwise would not. Most discussion has focused on GUIs, but recently it has broadened to robots, AUI/VUI, and other areas.
Anti-pattern: A design pattern that unintentionally produces negative outcomes.

2.3. Respect as a value in HCI

Respect has been viewed as a matter of interpersonal relationships, but is emerging as an important value in HCI too.
Even fair, just, and safe systems can sometimes be exploitative, coercive, or norm-violating.

3. Identifying ‘Social’ Forms of Dark and Anti-Patterns

Lacey and Caudwell: “Cute” robots can be dark patterns. Strong emotional bonds with users can prevent them from recognizing potentially harmful issues such as privacy violations.
Shamsudhin and Jotterand: Social robot design is inherently persuasive design.
The authors define social anti-patterns as social design patterns that are misapplied or used in inappropriate contexts, causing unintended negative experiences.

4. Methods

4.1. Exploratory survey of 83 smartphone users, using open questions that prompted them to recall experiences.
4.2. Selected 11 consenting participants from the 24 who completed the survey, and collected data for a week through experience sampling.
4.3. Semi-structured interviews: 15–30-minute online meetings with each participant after ESM.
4.4. Exploratory workshops in groups of 3–4, using Microsoft Teams and Miro.

5. Results and Analysis

‘Social’ dark patterns: manipulative uses of social cues (RQ1)
‘Social’ anti-patterns: counterproductive uses of social cues (RQ2)
Participant suggestions for improved social acting (RQ3, RQ4)

6. Discussion and Implications

The misuse and abuse of ‘social’ design patterns

Towards a more respectful approach to CASA

Be transparent about intentions and capacities.
Offer users the means to negotiate how they are treated.
Don’t help without being asked to.
Don’t pigeonhole users
Don’t breach personal boundaries

7. Limitations and Future Work

Our participant pool was limited to those aged 18-60 in English-speaking countries
Another potential limitation stems from anchoring a large part of our investigation on smartphone notifications
There is a significant risk, as with any qualitative study, of investigator bias

8. Conclusion

Research related to CASA has encouraged interaction designers to make systems act in increasingly socially-conforming (chatty, even friend-like) ways to elicit favourable responses from users. However, researchers are increasingly noticing backfiring effects of using automated forms of social acting: in the wrong context, or without the appropriate level of social nuance, even well-intentioned, friendly behaviour can be experienced badly. This paper contributes to the literature on dark patterns/nudging by identifying and characterising an emerging ‘social’ class of dark patterns. Drawing from a series of qualitative engagements with end-users, we critically engage with CASA research by proposing when, and why, automated forms of social acting may backfire–constituting ‘social’ anti-patterns. Based on end-user preferences and suggestions, we also proposed guidelines for interaction designers to help prevent undesirable forms of automated social acting. Finally, building on recent suggestions to introduce ‘respect’ as a value in HCI, we offer user-led advice on what it may mean for (socially) interactive systems to treat people respectfully. Overall, we hope our work inspires serious discussions on improving how interfaces treat people in interactions.