Demystifying A2UI: How to Make AI Agents “Speak UI” in Your App

Demystifying A2UI: How to Make AI Agents “Speak UI” in Your App

I’ve spent a fair amount of time experimenting with A2UI, a protocol for building agentic interfaces. Why? Quite simply, because I think the idea of dynamically generating user interfaces is awesome, and despite the fact that this is a relatively new protocol (currently 1.0 RC), I wanted to answer a fundamental question: “How do I incorporate A2UI into my own app?”

If you’re new to working with generative UIs, it may be challenging to wrap your head around the core concepts. This post is my mental model for how everything works from a developer’s implementation perspective so that you can start using it in your own applications.

I’m incredibly excited about the future of web apps. So much so that last November, I hosted a livestream called Exploring the future of web apps during which I built my own framework for generating agentic web apps with Angular.

A Quick Side Note: While the framework I built for that livestream isn’t nearly as flexible or extensible (to use with other web frameworks, for example) as A2UI, the architectural patterns are very similar. If you’re still struggling to understand the high-level concepts after reading the A2UI docs and this blog post, give that livestream a watch. The shared concepts will help make it click.

What is A2UI?

A2UI enables AI agents to generate rich, interactive user interfaces that render natively across web, mobile, and desktop - without executing arbitrary code. This is exciting to me because A2UI enables:

• Seamless in-app experiences: Interfaces that adapt dynamically to user intent.
• Portable UIs: Views that can appear virtually anywhere, serving as a vital part of Gemini web app experiences.
• Protection: No arbitrary code execution.

But to understand A2UI, you first have to understand what it isn’t. It’s not a framework; it’s a protocol. The implications of this are massive. Because it’s a protocol, using GenUI doesn’t lock you into a single stack. It defines how an LLM agent and a client application communicate about user interfaces, leaving the actual rendering up to whatever frontend tech you prefer.

High-Level Architecture of an A2UI Application

Before diving into the inner workings and implementation, let’s cover the high-level concepts and requirements. At its most basic level, A2UI operates around a shared protocol or schema.

• The Server (Agent-Side): Generates a message following this schema to define the UI layout. What makes this an “agentic” UI is that this message is generated dynamically by an AI agent based on a user’s query.
• The Client (Web/Mobile): Reads that message and knows exactly how to construct the UI.

The beauty of the protocol is that you can implement clients using a variety of different web and mobile frameworks. For today, we’re going to focus on an Angular implementation.

To give you an idea of what we are passing back and forth, here is an example of what an A2UI message looks like. Don’t worry if this looks foreign to you right now - we’ll get into more detail later - but it’s basically just a JSON definition of the app structure:

{
  "version": "v0.9",
  "updateComponents": {
    "surfaceId": "main",
    "components": [
      {
        "id": "root",
        "component": "Column",
        "children": ["header", "body"]
      },
      {
        "id": "header",
        "component": "Text",
        "text": "Welcome"
      },
      {
        "id": "body",
        "component": "Card",
        "child": "content"
      },
      {
        "id": "content",
        "component": "Text",
        "text": {"path": "/message"}
      }
    ]
  }
}

Developer Input (What Data You Need)

In an effort to better understand this, let’s assume we can strip out all of the setup and boilerplate so all you need to provide is the actual data (or metadata) to implement A2UI in your app. We’ll come back to how this all fits into your app later on. But for now, here is what you, the developer, need to provide.

Agent (Server) Side

Let’s start with the agent configuration on the agent (server) side.

1. Catalog: Available Building Blocks

The catalog is the dictionary of elements available to use in your A2UI app. It is maintained server-side so the agent knows what “building blocks” it can draw from when creating messages to send to the client. These building blocks can range from basic elements like text fields and icons to sophisticated custom components. This catalog uses JSON Schema and contains the definition of the elements - such as their descriptions, variants, and expected children.

The A2UI team open-sourced a “basic catalog” of commonly used elements to make it easier to get started, but you can (and probably should) also define your own. For the sake of this blog post, we’ll be looking at two samples provided by the A2UI team.

In the A2UI repository, the restaurant_finder sample uses only the basic catalog. Conversely, the rizzcharts example implements its own custom catalog because it relies on highly customized components, like a Chart:

"Chart": {
  "type": "object",
  "allOf": [
    { "$ref": "common_types.json#/$defs/ComponentCommon" },
    { "$ref": "#/$defs/CatalogComponentCommon" },
    {
      "type": "object",
      "description": "An interactive chart that uses a hierarchical list of objects for its data.",
      "properties": {
        "component": { "const": "Chart" },
        "type": {
          "type": "string",
          "description": "The type of chart to render.",
          "enum": ["doughnut", "pie"]
        },
        "title": {
          "$ref": "common_types.json#/$defs/DynamicString",
          "description": "The title of the chart."
        },
        "chartData": {
          "description": "The data for the chart, provided as a list of items. Can be a literal array or a data model path.",
          "oneOf": [
            { "type": "array", "items": { "$ref": "#/$defs/ChartItem" } },
            { "$ref": "common_types.json#/$defs/DataBinding" }
          ]
        }
      },
      "required": ["component", "type", "chartData"]
    }
  ],
  "unevaluatedProperties": false
}

2. UI Description

This is a natural language description telling the agent which UI(s) the app can or should use, when to use them, and what they should look like. For example, look at how this is set up in the restaurant_finder prompt builder:

UI_DESCRIPTION = """
- If the query is for a list of restaurants, use the restaurant data you have already received from the `get_restaurants` tool to populate the `updateDataModel` message.
- IMPORTANT: When using updateDataModel to update items, you MUST specify `path: "/items"` in `updateDataModel`, and the `value` MUST be an array of restaurants.
- IMPORTANT: Always specify the path when using updateDataModel. The part message is ignored when the path is missing.
- If the number of restaurants is 5 or fewer, you MUST use the `SINGLE_COLUMN_LIST_EXAMPLE` template.
- If the number of restaurants is more than 5, you MUST use the `TWO_COLUMN_LIST_EXAMPLE` template.
- If the query is to book a restaurant (e.g., "USER_WANTS_TO_BOOK..."), you MUST use the `BOOKING_FORM_EXAMPLE` template.
- If the query is a booking submission (e.g., "User submitted a booking..."), you MUST use the `CONFIRMATION_EXAMPLE` template.
"""

3. Examples

Now the agent knows what kind of UIs to generate and what building blocks it can use, but that still leaves a lot open to interpretation. To guide the agent, you can provide examples using a few-shot prompting technique. This helps the agent better understand what the JSON message structure (and thus the rendered UI) should actually look like.

As a great example of this, check out the booking form example from the restaurant_finder sample. Since that app only uses the basic catalog, there is no single “booking form” component available. Instead, the example shows the agent exactly how to assemble a booking form out of standard elements like inputs, columns, and buttons.

4. Agent Skills

You can augment your UI descriptions with “skills” to map out specific user capabilities, variations, and use cases. For instance, the rizzcharts sample defines a view_sales_by_category skill:

AgentSkill(
    id="view_sales_by_category",
    name="View Sales by Category",
    description=(
        "Displays a pie chart of sales broken down by product category for"
        " a given time period."
    ),
    tags=["sales", "breakdown", "category", "pie chart", "revenue"],
    examples=[
        "show my sales breakdown by product category for q3",
        "What's the sales breakdown for last month?",
    ],
)

5. Action Mappings

Generating dynamic views is only half the battle. To make an app useful, we have to handle user behavior - like button clicks or form submissions. That’s where action mappings come in. An action mapping is a named action triggered by a user event that the agent can use to do something. Since that’s a bit vague, let’s look at the restaurant_finder submit_booking action:

elif action == "submit_booking":
    restaurant_name = ctx.get("restaurantName", "Unknown Restaurant")
    party_size = ctx.get("partySize", "Unknown Size")
    reservation_time = ctx.get("reservationTime", "Unknown Time")
    dietary_reqs = ctx.get("dietary", "None")
    image_url = ctx.get("imageUrl", "")
    query = (
        f"User submitted a booking for {restaurant_name} for {party_size} people at"
        f" {reservation_time} with dietary requirements: {dietary_reqs}. The image"
        f" URL is {image_url}"
    )

Notice what is happening here: we are saying “if the agent receives the submit_booking action, extract these parameters and submit this new natural language query back to the agent.” We aren’t hardcoding database writes or redirect logic here; we are feeding a structured state update back to the AI. And how does the agent know what to do with that query? Well, it’s an AI agent, so it figures it out - often by calling tools.

6. Agent Tools

Tools are functions or external resources that an agent can utilize to interact with its environment. They allow agents to perform complex tasks - like fetching live data, calling APIs, or interacting with databases. While agent tools aren’t technically part of A2UI specifically, and are more of an agentic concept more generally, they still play an important role in the story.

For instance, the rizzcharts sample defines tools like get_store_sales and get_sales_data. You can also use agent skills to tie these tools and mappings together, as shown in the restaurant_finder skill registration:

skill = AgentSkill(
    id="find_restaurants",
    name="Find Restaurants Tool",
    description=(
        "Helps find restaurants based on user criteria (e.g., cuisine, location)."
    ),
    tags=["restaurant", "finder"],
    examples=["Find me the top 10 chinese restaurants in the US"],
)

7. Additional Metadata and Configuration Options

Beyond the core logic, your server-side agents require a little extra metadata to function, such as an app name, ID, and description. You’ll also configure options for session management, memory, and whether to include few-shot examples in the prompt context.

Client-Side in an Angular App

Everything we’ve covered so far lives on the server. Now let’s talk about the client side. There are two main non-boilerplate pieces of information you need to provide in your frontend:

1. Custom Components Catalog

If your server-side agent is generating custom components (like charts or maps), your client needs to know how to render them. If you’re only using standard elements, this step isn’t even necessary.

For example, in the restaurant_finder app, we simply configure the Angular app to use the basic catalog, which contains basic UI elements including rows, columns, text elements, and cards:

{
  provide: A2UI_RENDERER_CONFIG,
  useFactory: () => {
    const injector = inject(Injector);
    return {
      catalogs: [new BasicCatalog()],
      actionHandler: (action: A2uiClientAction) =>
        injector.get(Client).handleAction(action),
    };
  },
}

However, if you have stricter requirements like the rizzcharts app, you’ll need custom components. That project includes an a2ui-catalog directory containing the actual Angular implementations of the custom components defined in the JSON schemas. The project also contains a catalog.ts file that extends the basic catalog to register these custom components. This custom catalog is registered in the app config in the same way.

Crucial Rule: The component definitions in your client must perfectly match the catalog schemas and examples provided to the agent on the server. If they drift, the agent will generate JSON that your client doesn’t know how to render.

2. Theming

To ensure these agent-generated views feel like a native part of your application, you can provide CSS custom properties and themes. For example, the restaurant_finder app styles its dynamic views using restaurant-theme.css. Keep in mind that the CSS variables used in this file pertain to the “basic catalog” implementation of the widgets. In custom catalogs, you can implement styling however you’d like.

Bringing It All Together

I decided to focus this deep dive on the specific developer inputs for a few reasons:

• Concept over Boilerplate: Focusing on the data schemas makes the core concepts much easier to grasp. You don’t have to get bogged down in directory structures or class interfaces right away. The official A2UI documentation covers the physical setup - like how to set up your Angular app to use A2UI - exceptionally well. My hope is that the pointers and code samples in this article can act as the practical bridge.
• Declarative Power: This approach highlights the clean separation between setup and implementation. Personally, I prefer highly declarative frameworks. In my own sandbox app, I took most of the metadata we discussed and shoved it into a single JSON file, which gave me an incredibly high degree of flexibility and maintainability.
• Architecture Sync: It illustrates how all these moving pieces lock together. When designing your own application, I highly encourage you to establish a single source of truth for your agentic views. From that single definition, you should derive your UI descriptions, JSON catalogs, agent examples, and client-side components to keep them in lockstep.
• Automation Potential: Finally, it gets us thinking about how we can automate these developer workflows.

The Agentic Automated Future

The future of dynamic, user-first interfaces is bright, and I can’t wait to see what you build.