Conversational AI Search Engines: Implications for Usability and the User Experience

The landscape of digital information retrieval is undergoing a profound transformation with the emergence of conversational AI search engines. These innovative tools, powered by advanced large language models (LLMs), represent a significant departure from traditional keyword-based search methodologies. As reported by Fast Company in February 2024 [1], these systems are designed to directly answer user questions by retrieving, synthesizing, and summarizing information from across the internet, fundamentally altering the search experience. This shift has ignited fervent enthusiasm within academic and scientific research circles, spearheading a wave of experimentation and development aimed at providing an alternative to the long-established Google model of search [2]. The rapid proliferation of these applications online since the rise of generative AI signals not merely an incremental improvement but a paradigm shift, prompting a critical examination of their implications for usability and the broader user experience.

The Evolution of Search: From Keywords to Conversations

For decades, the internet search experience has been synonymous with typing keywords into a search bar and sifting through a list of hyperlinked results. This model, while effective, often requires users to engage in a multi-step process: query formulation, result evaluation, page navigation, and manual information synthesis. The advent of conversational AI search engines like Perplexity AI and Andi introduces a new mental model, one that mirrors human conversation and directly addresses the user’s implicit goal: obtaining an answer.

Perplexity AI, for instance, presents a home page that initially appears familiar, featuring an input field and a results display. However, the underlying interaction model diverges significantly. Instead of a list of links, the user receives a direct, summarized answer, compiled from multiple sources. This approach resonates with the conversational chatbot interfaces popularized by applications like ChatGPT, which have familiarized users with generative AI’s ability to process natural language queries and produce coherent responses. The fundamental change lies in the system’s ability to act as an intelligent intermediary, performing the collation and summarization tasks that previously fell to the user.

Andi, another contender in this burgeoning space, takes this conversational mental model even further in its layout and interaction design. While its visual presentation is more akin to a chat interface, its information architecture, as will be discussed, retains elements that bridge the gap between traditional and AI-driven search, offering a hybrid approach to content display.

From an innovation perspective, the rise of these platforms exemplifies a "technology push" [3]. The extraordinary capabilities of generative AI have created new possibilities, driving the evolution of the digital experience market. However, the ultimate benefit to users, and the strategic risks associated with deploying powerful new technologies without a primary human-centric focus, remain subjects of ongoing scrutiny. The market, estimated to reach significant valuations in the coming years, is attracting substantial investment, underscoring the perceived potential of this disruptive technology to redefine how billions access information.

Enhancing the User Experience: Immediate Answers and Simplified Journeys

Conversational AI search engines offer tangible improvements in usability and interaction, streamlining the search experience. The core enhancement is the direct provision of answers, eliminating the need for users to navigate multiple web pages and extract information manually. This directly addresses a crucial usability heuristic: "matching between system and the real world" [4], as human communication frequently involves direct questions and answers.

Consider the user journey with a traditional search engine:

1. User types query.
2. User reviews a Search Engine Results Page (SERP) containing numerous links.
3. User evaluates titles and snippets to guess relevance.
4. User clicks a link, loads a new page.
5. User scans the page for the answer.
6. (Often) User returns to the SERP to try another link if the first was unhelpful.

In contrast, a conversational AI search engine like Perplexity AI aims to condense this into:

1. User types query.
2. System provides a direct, summarized answer, often with inline citations.

This simplification drastically reduces cognitive load and interaction friction. By collating snippets from multiple sources, the AI delivers a synthesized response that more closely matches the format of a web page article, rather than a mere list of potential sources. This eliminates the laborious task of individual search result evaluation, conjecture, and page scanning. The goal of finding an answer within a text remains constant, but the path to achieving it becomes significantly shorter and more efficient.

The benefits extend beyond mere speed. These systems embody several other Nielsen usability heuristics:

• Visibility of system status: Many AI search engines show their "thinking" process (e.g., "Searching…", "Generating answer…"), keeping the user informed.
• Flexibility and efficiency of use: They cater to both novice users seeking quick answers and expert users who can still dive into sources.
• Error prevention: By synthesizing information, they aim to reduce the chance of users misinterpreting partial information from a single source.
• Help and documentation: While not always explicit, the conversational nature itself acts as a form of inherent guidance, allowing users to refine queries naturally.

While these improvements undeniably enhance the search journey, the question arises whether these gains necessarily justify their widespread adoption as the future standard for information retrieval.

Broader Implications: The Triad of Trust, Explainability, and Human Agency

The shift toward conversational AI search engines, while offering convenience, introduces complex challenges related to trust, the explainability of AI systems, and the long-term impact on human cognitive processes. When using tools like Perplexity AI, users inherently cede a significant degree of decision-making to the AI. The system employs generative technology to construct an answer by selecting, extracting, and summarizing content it deems most relevant. This process, however, often lacks transparency regarding why specific sources were chosen over others, neglecting a crucial aspect of AI explainability (XAI) [5]. This deficit in transparency directly impacts the trustworthiness of the system.

Francesca Rossi, IBM’s Global Ethics Leader, emphasizes the critical role of trust in AI systems, stating that AI raises concerns about "its ability to make important decisions in a way that humans would perceive as fair, to be aware and aligned to human values that are relevant to the problems being tackled, and the capability to explain its reasoning and decision-making" [6]. In the enterprise world, the risks of an untrustworthy AI system—including impending regulations, substantial fines, reputational damage, and legal liabilities—are becoming increasingly pressing. However, the implications for individual "Internauts"—highly skilled, habitual internet users—are more subtle but equally profound.

Roberta Katz, an expert on Gen Z, uses an architectural analogy to explain this impact: "first you make the building and then the building makes you." Just as physical environments shape behavior, the architectures of our digital applications profoundly influence their users [7]. It is reasonable to hypothesize that a significant danger of tools like Perplexity AI is the potential for users to become accustomed to readily accepting convenient, authoritative-sounding answers without critically questioning their accuracy or veracity.

While these tools generally provide links to source documents, several critical questions remain unanswered:

• How were these specific sources prioritized among potentially thousands of others?
• To what extent did the AI system interpret or even alter the original meaning of the source material during summarization?

The first issue, common in traditional search to some extent, becomes amplified by the second, which stems from the deeper delegation of decision-making inherent in generative AI. Even with full explainability, this simplified user experience could, over time, diminish users’ critical thinking skills, much as text messaging has been linked to a decline in formal writing proficiency.

From a usability perspective, the traditional act of sifting through Google’s SERP, comparing information from various articles, and synthesizing an answer is undeniably more tedious. However, these tasks serve as crucial exercises for analytical and creative skills, fostering a degree of critical engagement that discourages uncritical over-reliance on AI models. If the mental model of a search engine evolves into that of an infallible oracle, the implications for our collective ability to discern truth from falsehood could be immense. While this may be inconsequential for mundane searches like finding a new pair of trousers, it becomes critically important for accessing knowledge in fields such as policymaking, scientific research, or civil discourse.

It is important to note that an AI search experience does not have to be designed this way. Andi, for example, adopts a different architectural approach that retains a closer resemblance to the traditional search model, despite its chatbot-like user interface. As seen in a query example, Andi often presents a brief answer that is a direct snippet from the most relevant source (e.g., Wikipedia), alongside a list of all sources with direct links, typically displayed on the left side of the page. Conceptually, this structure is not far removed from a traditional SERP, despite its distinct layout.

Andi’s structure also addresses a crucial concern articulated by Kevin Roose of The New York Times: "If AI search engines can reliably summarize what’s happening in Gaza or tell users which toaster to buy, why would anyone visit a publisher’s Web site ever again?" [8]. This concern highlights the potential threat to the content ecosystem and the revenue streams of publishers who rely on web traffic. Andi’s design, with its prominent display of clickable source links, actively encourages users to visit the original websites, demonstrating how thoughtful UX and UI design can elicit more desirable user behaviors that support content creators.

The Imperative of Trustworthy AI in Search

The future trajectory of AI search engines hinges on the development and deployment of "Trustworthy AI." The central question is not if AI will be the future of search—as it most likely will, given its generative capabilities to enhance user experience and expedite knowledge acquisition—but rather:

• How can we ensure that AI search engines are built to be reliable, accurate, and transparent?
• What design principles and regulatory frameworks are necessary to foster critical thinking and prevent an over-reliance on AI-generated information?
• How can the AI search experience sustain and support the broader content ecosystem, including publishers and creators?

A study published in Nature [9] highlighted the divergent views on AI science search engines among researchers: some found them incredibly useful and accurate, while others expressed deep distrust due to inconsistent retrieval performance. This underscores that trust is the central factor influencing AI adoption. To cultivate this trust, two key areas of focus are paramount:

1. Technological Advancement in Reliability and Explainability:

   • Retrieval-Augmented Generation (RAG): Enhancing RAG systems [10] is crucial for grounding LLM responses in verifiable external data, thereby reducing "AI hallucinations" [11]—instances where AI generates plausible but factually incorrect information.
   • Source Citation and Transparency: Clearly indicating the sources used for each part of a generated answer, along with confidence scores or relevance indicators, can provide users with the means to verify information.
   • Explainable AI (XAI) Integration: Developing methods for AI systems to articulate their reasoning and decision-making processes—e.g., why certain sources were prioritized, or how different pieces of information were weighted during summarization—is essential for building user confidence. Projects like Microsoft Research’s GraphRAG [12] aim to unlock LLM discovery on narrative private data by building a knowledge graph of relationships, which could enhance explainability and accuracy.

2. User Education and Responsible Design:

   • Fostering Critical Digital Literacy: Educational initiatives are needed to equip users with the skills to critically evaluate AI-generated information, understand its limitations, and recognize the importance of source verification.
   • Designing for Human Agency: AI search interfaces should balance convenience with features that encourage users to engage with original sources and exercise their critical faculties. This could involve prominent source displays, options to delve deeper into research, or even gentle nudges to cross-reference information.
   • Ethical Design Principles: Developers must embed ethical considerations from the outset, prioritizing fairness, accountability, and user well-being in the design and deployment of these systems.

As new regulations, such as the EU AI Act, emerge to ensure safer and more ethical uses of AI, the question of whether a "good" search experience is also the "right" experience becomes increasingly relevant. Within a context where users are delegating a significant portion of their decision-making to LLMs, design efforts must pivot beyond mere user-friendliness to prioritize accuracy, trustworthiness, and the comprehensiveness of search outputs. This necessitates much more than building a super user-friendly conversational interface; it requires a holistic approach that integrates advanced AI capabilities with robust ethical frameworks and user empowerment strategies to shape a responsible and beneficial future for AI search.