Zum Hauptinhalt springen Skip to page footer

AI Agents x Digital Product Design

How AI Agents Will Change Digital Product Design

The disruptive impact of AI is felt across all domains. It already feels surreal when a computer seems to ‘understand’ what you're prompting. However, the new capabilities of multimodal AI are mind-blowing. The sheer fact that you can natively talk to an AI model, ‘show’ your surrounding to it with a camera, and literally ask what you are looking for is groundbreaking. With such disruptive nature comes the question: how will AI change our interaction paradigms with digital services and products?

AI Agents enter the floor

While many technical innovations and aspects can be attributed to the disruptive nature of AI, the concept of AI agents has the biggest impact on interaction design paradigms. Put simply, an AI agent is a piece of software that can act autonomously. A key feature is their ability to interact with their environment, meaning they can receive inputs from their surroundings, make decisions, and perform actions to achieve specific goals. In practice, AI agents can range from simple software performing repetitive tasks to complex systems using several AI models to perform a complex set of tasks. For example, ChatGPT by OpenAI is, by definition, an AI agent that makes use of different AI models to understand text, images, or voice. Former Microsoft CEO Bill Gates is so optimistic about the concept of AI agents that he stated at the end of 2023:

„[AI] Agents are not only going to change how everyone interacts with computers. They’re also going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons.“ (Gates Notes)

Although that seemed like a pretty bold statement, the transformation is tangible now. The latest flagship models by OpenAI and Google showcase how AI agents are reshaping our interactions with digital products.

Figure 1: Current interaction paradigm

Figure 2: AI Agent Interaction paradigm

Guided by their wisdom.

Let's start with a quick look at the different levels of current interaction paradigms (Fig. 1):

  1. You have a need, like finding a train route, checking your bank account, or planning a physical training session.
  2. You pick up your phone.
  3. You decide which app or website to use to meet your need.

All of your decisions are highly influenced by the design, functionality, and your personal brand experiences with the digital service. AI agents, however, move right into the sweet spot between you and your services. As AI agents rapidly advance in multimodal comprehension of inputs (text, voice, visual, spatial), the way you interact with your digital services is undergoing profound transformations. Instead of navigating through a predefined user experience by tapping on menus, lists, links, and buttons, you can simply instruct or speak to your device: “Show me train connections from here to Berlin for next Monday, arriving in the late afternoon.” And because AI agents possess enhanced reasoning and creative capabilities, they produce responses that more closely mimic human outputs.

In simpler terms, interacting with an AI agent is becoming increasingly natural and intuitive.

But as these agents demonstrate their capability to autonomously understand, address, and fulfill increasingly complex tasks, people will increasingly rely less on traditional interactions with websites or apps. We will hand over our decision-making and control of the data level to the AI agents. They will autonomously decide which service to use, which information to show, and which decisions to make to satisfy your needs.




The king is dead. Long live the king.

Looking at the current landscape of operating systems for consumer mobile devices, desktops, laptops, and tablets, there are only three major tech companies dominating our world: Alphabet, Apple, and Microsoft. And all three are strongly moving towards the implementation of AI agents at the core of their products:

  • Google has unveiled its Gemini AI ecosystem with different models, like the efficient 'Nano' model, aimed at enhancing on-device tasks within its existing mobile operating system. Additionally, Google believes AI is the future of search.
  • Apple has released only incrementally AI-improved tools, such as photo editing and image recognition. However, rumors hint at a future reveal of its full AI strategy (and a potential cooperation with OpenAI) at the Worldwide Developers Conference on June 10.
  • Microsoft has made significant investments in OpenAI and leverages its AI technologies to enhance its own products and services, such as Azure and Office. This collaboration allows both companies to remain leaders in the field of artificial intelligence and to jointly develop innovative solutions.

Despite these advances, OpenAI has started to carve out a unique position within these ecosystems. Adding to its existing apps, OpenAI introduced a new desktop application (as of May, initially for Mac only) that allows users to engage in voice conversations directly from their computers – with promises of new audio and video capabilities in the near future.

It is designed to integrate seamlessly into anything you’re doing on your computer. (OpenAI)

ChatGPT’s versatility and multimodal capabilities position it as a potential gateway to your digital world, seamlessly integrating into daily computing activities and challenging traditional app-based ecosystems. Its natural interaction style not only complements but also extends beyond the existing frameworks, making it feel like a natural portal to many digital experiences.

As all the dominant tech companies push forward with the integration of AI agents, the digital product and service landscape will undergo massive changes. These changes bring about a range of questions and challenges for digital product teams and organisations.

User Experience and Interaction

  • Generative UI: AI agents’ output needs to be flexible enough to display complex information and data. Enhancing current UX patterns and leveraging design systems will be a key task for digital product teams.
  • Autonomy vs. Automation: While automation can enhance efficiency, over-reliance on it can lead to issues if the AI makes incorrect decisions. Users need an easy way to override AI decisions and take manual control when necessary.
  • Accessibility Challenges: AI-driven interfaces must be designed to be accessible to all users, including those with disabilities. Ensuring that voice, text, and visual inputs are all accessible requires thoughtful design and testing.
  • Consistency in Interaction: Ensuring a consistent and predictable user experience across different AI-driven interfaces is essential to avoid confusion and frustration among users. The unpredictable nature of AI and our current mental model, that computers are predictable and reliable, are conflicting realities.

Market and Competition

  • Market Changes and Competition: Traditional apps and websites might lose significance as AI systems become the primary interfaces for digital services. Companies and brands have to rethink their business models and question wether to integrate or protect themselves from AI data crawlers. Both options come with different strategic and technical challenges.
  • Monopolization: The dominance of a few AI platforms could lead to monopolization, with a handful of companies controlling the digital product and service landscape. This could stifle innovation and limit competition.

Data and Privacy Concerns

  • Data Ownership and Privacy: AI-focused operating systems could effortlessly assimilate and tailor data to meet their needs, fundamentally reshaping most of the landscape of apps, websites, and data services as we currently know them, essentially boiling them down to mere data points. With AI-focused operating systems seamlessly assimilating and tailoring data, users might lose control over their personal information. More than ever, it remains hidden from the user how AI handles their data: How is it processed? How does the AI model arrive at its conclusions? Who receives which data? These questions about data access and usage need to be addressed transparently to maintain user trust.
  • Security Risks: Centralizing and processing large amounts of data through AI systems increases the risk of data breaches and cyberattacks. If these systems are compromised, the consequences could be extensive and severe.
  • Data Integrity: Ensuring the accuracy and reliability of data processed by AI systems is crucial. By nature AI models have a squishy nature being prone for errors or biases in data processing.

Ethics and Trust

  • Ethics and Bias: Bias is a known issue with AI models depending on their training data. An increased use of AI agents could reinforce existing biases and discrimination. Ensuring that AI is designed to be fair and ethical is crucial.
  • User Trust and Transparency: Users need to trust AI systems, and this trust is built through transparent UI/UX design. Explaining how AI makes decisions, providing feedback, and allowing users to understand and control their data are crucial elements.
  • Accountability: Establishing clear accountability for decisions made by AI systems is necessary. Users and organizations need to know who is responsible when AI makes mistakes or causes harm.

Things are moving fast in the AI space, and many hypotheses waft how the concept of AI agents will unfold. Despite all the challenges ahead, we are excited about the opportunities for digital products. And we believe Bill will be right: AI agents will be a crucial game changer for all individuals, corporations and organizations involved in the digital economy landscape.

Matthäus Eckert

Matthäus Eckert
Head of Digital Design

Let's discuss!

Do you have a digital product or service? Let's discuss the challenges and opportunities that AI agents bring for you.

Write us