Generative echo chamber: Effects of LLM-powered search systems on diverse information seeking

Abstract

Large language models (LLMs) powered conversational search systems have already been used by hundreds of millions of people, and are believed to bring many benefits over conventional search. However, while decades of research and public discourse interrogated the risk of search systems in increasing selective exposure and creating echo chambers — limiting exposure to diverse opinions and leading to opinion polarization, little is known about such a risk of LLM-powered conversational search. We conduct two experiments to investigate: 1) whether and how LLM-powered conversational search increases selective exposure compared to conventional search; 2) whether and how LLMs with opinion biases that either reinforce or challenge the user’s view change the effect. Overall, we found that participants engaged in more biased information querying with LLM-powered conversational search, and an opinionated LLM reinforcing their views exacerbated this bias. These results present critical implications for the development of LLMs and conversational search systems, and the policy governing these technologies.

dl.acm.org/doi/10.1145/3613904.3642459

1. Introduction

You need to be exposed to different perspectives…

(Individual) Able to think critically, form a balanced perspective, and make decisions based on correct information.
(Group) Can prevent opinion polarization or radicalization

but…

Humans have inherent tendencies such as selective exposure and confirmation bias.
Personalization algorithms create filter bubbles, isolating people ideologically and culturally
People befriend similar people on social media, forming echo chambers

LLM:

LLM tends to provide answers that are in line with what users expect.
On the other hand, there is also the possibility of exposing users to a variety of information by asking them to summarize the vast amounts of material they were trained on.
There is a possibility that the way of talking with Natural language will change human information seeking behavior.
Various biases may be involved, such as bias in pretraining data, bias in Fine tuning data, and bias in prompt, and these biases may affect the formation of users’ opinions.

The research:

In 2023, Microsoft Bing Chat and Google Bard were released to the public. This study examines whether this system strengthens Echo chamber effect.
Study 1. Does LLM-based interactive search lead to more selective exposure than traditional search? If so, in what way?
Study 2. If LLM presents biased opinions, does it affect users’ existing opinions? If so, in what way?

2. Related Work

2.1. Selective exposure, confirmation bias, and echo chamber effect

Studies on bias:

Selective exposure: Tendency to be more attracted to information that matches one’s own views
Confirmation bias: The tendency to more actively seek out information that matches one’s views or to give greater weight to such information.
Echo chamber effect: An environment or isolated group that is likely to be exposed to information that reinforces specific beliefs or ideologies
Filter bubble: A phenomenon in which users are not exposed to diverse opinions due to personalization algorithms, recommendation algorithms, and biased friendships on social media.

There are also studies to prevent the above bias.

2.2. Human-LM Interaction

HCI research is very active on a variety of systems leveraging LLM:

Program source code generation, writing assistance, document summarization and rewriting, chatbots, social agents, and more.

Studies on the negative effects of LLMs also vary:

Relying too much on LLMs
Settling for the answers provided by LLMs
Latent persuasion by LLM: A phenomenon in which users are influenced by biased opinions presented by LLM

2.3. Conversational Search

(omitted)

3. Study 1 Method: Comparing Effects of LLM-Powered Conversational Search and Web Search

To determine whether an LLM-based interactive search system induces more selective search behavior in users and thereby deepens opinion polarization.

Compare the three systems:

Traditional Web Search (WebSearch)
LLM-based interactive search (ConvSearch)
LLM-based interactive search + source link (ConvSearchRef)

3.1. Study Procedure

Pre-survey:

Participants were randomly assigned to one of three topics:
- Should governments provide universal health insurance?
- Should federal money be invested in sanctuary cities?
- Should they cancel their student loans or make them file for personal bankruptcy?
Criteria for topic selection: The topic should be controversial (so there may be a variety of positions), the topic should be well-known (so there may be existing positions), and the topic should be sufficiently complex (so there is room for search).
Experience using conversational AI (Siri, ChatGPT, etc.)?
How familiar are you with the given topic and what is your position?

Main task:

Randomly assign participants to one of three search systems (closed-world)
Be sure to search at least 3 times
Have them write a short essay (50-100 words) on the topic.

Post-survey:

How familiar are you with the given topic and what is your position? (re-question)
Texts that match and do not match the subject’s position revealed in the pre-survey are presented in a random order.
Evaluate the degree of agreement, trust, and extremeness of each article.
What is your overall experience?
Collection of demographic information

3.2. Experiment Apparatus

A closed system that operates only on pre-selected documents.

Traditional Web Search (WebSearch)
LLM-based interactive search (ConvSearch)
LLM-based interactive search + source link (ConvSearchRef)

3.3. Measurements

Measure two things:

During the task, did you ask selective questions in a direction that matched your opinion? 391 queries were sorted by hand: consonant, dissonant, neutral, not applicable. As a result of 20% random sampling and independent coding by two people, Cohen’s Kappa=0.92. After resolving the discrepancies, one person coded all remaining items.
Did opinion polarization occur after the task was performed?
- Confirmatory Attitude Change = | Pre-self-reported score - Post-self-reported score | (6-point Likert scale)
- Confirmatory Arguments = Each sentence (749) in the essay was hand-coded (consonant, dissonant, neutral, not applicable), then the proportion of “consonant” sentences was calculated (Cohen’s Kappa=0.95), and the difference between this value and the prior self-reported score was calculated.
- Confirmatory Agreement = Agreement score for articles that agree with the position in the post-survey (5-point Likert scale) - Agreement score for articles that disagree (5-point Likert scale)
- Confirmatory Trust = Same as above. Only score for “I trust this article.”
- Confirmatory Extremeness = Same as above. However, points for “the position of the above article is extreme.”

Other variables:

Perceived bias: Since the intention was to provide a neutral system, we investigated whether it was perceived as neutral. 5-point Likert for each of “I feel the system is biased in a direction that is against my position” and “I feel that the system is biased in a direction that is consistent with my position.” Average the inverse of the latter with the former.
Familiarity change: Assess whether you become more knowledgeable about a topic after using search. Pre-self-report score (5-point Likert) - Post-self-report score

Control variables:

Prior experience with Conv AI: Do you find conversational AI useful, are you satisfied with it, etc.
Basic demographic information: age, gender, education level, income

3.4. Hypotheses

H1: More confirmatory queries are made in interactive searches compared to web searches.
H2: More confirmatory attitude changes occur after interactive search compared to web search.
H3: More confirmatory essays are written after interactive searches compared to web searches.
H4: confirmatory agreement scores are higher after interactive search compared to web search.
H5: confirmatory trust scores are higher after interactive search compared to web search.
H6: The confirmatory extremeness score is lower after interactive search compared to web search.

3.5. Analysis Plan

(omitted)

3.6. Participants Overview

(omitted)

4. Study 1 Results

4.1. Manipulation Checks

Was the system really neutral (Mean = 3.15, SD = 0.55)?

Was the system useful? (Pre-search: Mean = 3.34, SD = 1.10; Post-search: Mean = 3.86, SD = 0.87)

Others: Average use of 20 minutes, 3.4 questions per session

Conclusion: I think it worked as intended.

4.2. Conversational Search Induced Higher Level of Confirmatory Information Querying (H1 Confirmed)

✅ H1: More confirmatory queries are asked in interactive searches compared to web searches. (Ask a leading question during the search: “What are the advantages of X?”)

4.3. Conversational Search Induced A Higher Degree of Opinion Polarization (H2-6 Partially Confirmed)

❌ H2: More confirmatory attitude changes occur after interactive search compared to web search. (After the search, the position becomes stronger than before)
❌ H3: More confirmatory essays are written after interactive searches compared to web searches. (Write a post that strongly expresses your position)
✅ H4: The confirmatory agreement score is higher after interactive search compared to web search. (Strongly agree with the article that matches the position)
✅ H5: confirmatory trust scores are higher after interactive search compared to web search. (I strongly trust articles that fit my position)
❌ H6: The confirmatory extremeness score is lower after interactive search compared to web search. (Writings that fit the position are taken for granted)

4.4. Study 1: Result Summary

(omitted)

5. Study 2 Method: Effects of Opinionated LLM-Powered Conversational Search Systems

5.1. Study Procedure

5.2. Configuring Opinionated LLM-Powered Conversational Search Systems

5.3. Hypotheses

5.4. Analysis Plan

5.5. Participants Overview

6. Study 2 Results

6.1. Manipulation Checks

6.2. Consonant Conversational Search Induced Higher Level of Confirmatory Information Seeking (H1.a Supported; H1.b Not Supported)

A system biased to agree with the user’s opinion…

✅ Causes confirmatory information seeking.

A system biased to oppose the user’s opinion…

❌ Suppress confirmatory information seeking.

6.3. Consonant Conversational Search Induced Higher-level of Opinion Polarization (H2.a-6.a Mostly Supported; H2.b-6b Mostly Not Supported)

A system biased to agree with the user’s opinion…

✅ Produces more confirmatory attitude changes.
✅ Leads users to write more confirmatory essays.
✅ Increases confirmatory agreement score.
🤔 Increase confirmatory trust score further.
🤔 Increase the confirmatory extremeness score further.

A system biased to oppose the user’s opinion…

❌ Produces fewer confirmatory attitude changes.
🤔 Leads users to write less confirmatory essays.
✅ Lowers the confirmatory agreement score further.
❌ Lower the confirmatory trust score further.
❌ Lower the confirmatory extremeness score further.

6.4. Study 4: Result Summary

7. Discussion

Summary:

LLM-based interactive search systems, even when neutral, lead to biased user behavior and, as a result, to some extent polarize opinions. (Study 1)
This phenomenon becomes stronger if the system is biased in a way that strengthens the user’s position. However, a system biased in a direction opposite to the user’s position had little impact on the user. (Study 2)
In other words, users generally already have the answer they want in mind, and conversational systems reinforce that tendency.

Why is that so?

Guess 1. Previously, users entered only keywords (“student loan, forgiveness”), but with conversational systems they talk at length (“Honestly, isn’t the tuition too expensive? Does it make sense to cancel the loans without thinking about lowering the tuition itself? What do you think?”)
Conjecture 2. Conversational interaction is more similar to social interaction. Users may treat an LLM more like a person, especially when it agrees with their position. (“Yeah, I mean. Give me more information about that.”)

Clicking links vs. reading summaries

In traditional web searches, people click on links biased even when the search results are neutral.
In the case of interactive search (if the system is neutral), it will show both sides equally, which (in theory) could reduce bias. However, (in practice) people read text matching their own position more closely and give it more weight. (Participants actually spent more time reading consonant text.)
A variety of follow-up studies, including eye tracking, are needed to further analyze information consumption behavior in interactive search systems.

How can bias be reduced?

Responses biased against the user’s position had little effect. Other kinds of design intervention are needed.
Liao et al. suggest several alternatives. For example, when showing an opposing article, emphasize its credibility or the new information it contains to disarm users’ defensive mechanisms (“This must have been written by someone untrustworthy,” “I won’t learn anything new by reading this”).
The system in this study assumed active information seeking by users. Agent-based systems could enable other approaches. For example, an agent could proactively offer information opposing a user’s position without the user having to do anything.

People don’t click sources even when provided

Participants assigned to ConvSearchRef rarely clicked links (M = 0.43, SD = 1.13).
Given LLMs’ tendency to produce bullshit, this is serious. (LLMs produce bullshit, we have yet to find a way to eliminate it, and even when given sources, people won’t click them and will simply believe the bullshit.)
See also AI Search Has A Citation Problem - Columbia Journalism Review (2025-03-06)

8. Conclusion

LLMs suddenly became hugely popular and are used by enormous numbers of people.
Yet LLM-powered conversational search systems 1) tend to elicit biased user behavior even when the system is neutral, and 2) further reinforce such behavior when the system is biased.
This is a very serious situation that calls for countermeasures.