<!-- Source: https://docs.marimo.io/api/inputs/chat/ -->

# Chat

> **Looking for example notebooks?**
>
> For example notebooks, check out [`examples/ai/chat` on our
> GitHub](https://github.com/marimo-team/marimo/tree/main/examples/ai/chat).

[Interactive marimo example](https://marimo.app/?embed=true&mode=edit&show-chrome=false#code/JYWwDg9gTgLgBCAhlUEBQixjgXgc1AOgEEsAKAd2ABMYALHAIhAFNrgBXERgSjTQACmMIQDGLADYS01FgDM4AfUVkeALjRwtcUJFj4UICHEQBnBOm1woLGBygA7QcLGTpshctUarHuKd0JFkUWUToIRSNZCTJWU1NEAHMWUwAaOFEIBzlgRPVNK20bO0c4OUYATQgOf0QaNTgAbziE5NMAbQBaAEYAXTEsmBYHGABfRn4rI0IOYDE6RBgyAsL-QODQ8MiIaNSVwrAoCHAYUxx2xgAJNwhGdKuIChMbOABPaoB+Rl691a1TcIURSZbK5eyLYBZYGDI4SM4AFSgHBY+z4VmK9icQA)

The chat UI element provides an interactive chatbot interface for
conversations. It can be customized with different models, including built-in
AI models from popular providers or custom functions.

## marimo.ui.chat

```python
chat(
    model: Callable[
        [list[ChatMessage], ChatModelConfig], object
    ],
    *,
    prompts: list[str] | None = None,
    on_message: Callable[[list[ChatMessage]], None]
    | None = None,
    show_configuration_controls: bool = False,
    config: ChatModelConfigDict | None = DEFAULT_CONFIG,
    allow_attachments: bool | list[str] = False,
    max_height: int | None = None,
    disabled: bool = False,
)
```

Bases: `UIElement[dict[str, Any], list[[ChatMessage](#marimo.ai.ChatMessage)]]`

A chatbot UI element for interactive conversations.

Define a chatbot by implementing a function that takes a list of ChatMessages and
optionally a config object as input, and returns the chat response. The response
can be any object, including text, plots, or marimo UI elements.

Examples:

Using a custom model:
```python
def my_rag_model(messages, config):
    # Each message has a `content` attribute, as well as a `role`
    # attribute ("user", "system", "assistant");
    question = messages[-1].content
    docs = find_docs(question)
    prompt = template(question, docs, messages)
    response = query(prompt)
    if is_dataset(response):
        return dataset_to_chart(response)
    return response


chat = mo.ui.chat(my_rag_model)
```

Async functions and async generators are also supported:
```python
async def my_rag_model(messages):
    return await my_async_function(messages)
```

Regular (sync) generators for streaming:
```python
def my_streaming_model(messages, config):
    for chunk in process_stream():
        yield chunk  # Each yield updates the UI
```

Async generators for streaming with async operations:
```python
async def my_async_streaming_model(messages, config):
    async for chunk in async_process_stream():
        yield chunk  # Each yield updates the UI
```

The last value yielded by the generator is treated as the model
response. Streaming responses are automatically streamed to the frontend
as they are generated.

Using a built-in model:
```python
from pydantic_ai import Agent

chat = mo.ui.chat(
    mo.ai.llm.pydantic_ai(
        Agent(
            "openai:gpt-5",
            system_prompt="You are a helpful assistant.",
        )
    ),
)
```

Using attachments:
```python
chat = mo.ui.chat(
    mo.ai.llm.pydantic_ai(
        Agent(
            "openai:gpt-5",
            system_prompt="You are a helpful assistant.",
        )
    ),
    allow_attachments=["image/png", "image/jpeg"],
)
```

Custom model with Vercel AI SDK streaming (reasoning, tool calls):
```python
import pydantic_ai.ui.vercel_ai.response_types as vercel


async def custom_model(messages, config):
    # Stream reasoning/thinking
    yield vercel.ReasoningStartChunk(id="reasoning-1")
    yield vercel.ReasoningDeltaChunk(
        id="reasoning-1", delta="Let me think..."
    )
    yield vercel.ReasoningEndChunk(id="reasoning-1")

    # Stream text response (can also use plain dicts)
    yield {"type": "text-start", "id": "text-1"}
    yield vercel.TextDeltaChunk(
        id="text-1", delta="Here is my answer."
    )
    yield vercel.TextEndChunk(id="text-1")

    yield vercel.FinishChunk(finish_reason="stop")


chat = mo.ui.chat(custom_model)
```

Refer to examples/ai/chat/pydantic-ai-chat.py for a complete example.

| ATTRIBUTE | DESCRIPTION |
| --- | --- |
| `[value](#marimo.ui.chat.value)` | The current chat history, a list of ChatMessage objects. **TYPE:** `List[[ChatMessage](#marimo.ai.ChatMessage)]` |

| PARAMETER | DESCRIPTION |
| --- | --- |
| `model` | A callable that takes in the chat history and returns a response. **TYPE:** `Callable[[List[[ChatMessage](#marimo.ai.ChatMessage)], [ChatModelConfig](#marimo.ai.ChatModelConfig)], object]` |
| `prompts` | Optional list of initial prompts to present to the user. Defaults to None. **TYPE:** `List[str]` **DEFAULT:** `None` |
| `on_message` | Optional callback function to handle new messages. Defaults to None. **TYPE:** `Callable[[List[[ChatMessage](#marimo.ai.ChatMessage)]], None]` **DEFAULT:** `None` |
| `show_configuration_controls` | Whether to show the configuration controls. Defaults to False. **TYPE:** `bool` **DEFAULT:** `False` |
| `config` | Optional configuration to override the default configuration. Keys include: - max\_tokens. The maximum number of tokens to generate. Defaults to 100. - temperature. Defaults to 0.5. - top\_p. Defaults to 1. - top\_k. Defaults to 40. - frequency\_penalty. Defaults to 0. - presence\_penalty. Defaults to 0. **TYPE:** `ChatModelConfigDict` **DEFAULT:** `DEFAULT_CONFIG` |
| `allow_attachments` | Allow attachments. True for any attachments types, or pass a list of mime types. Defaults to False. **TYPE:** `bool \| List[str]` **DEFAULT:** `False` |
| `max_height` | Optional maximum height for the chat element. Defaults to None. **TYPE:** `int` **DEFAULT:** `None` |
| `disabled` | Whether the chat input is disabled. When True, the user cannot type or send messages. Defaults to False. **TYPE:** `bool` **DEFAULT:** `False` |

### text `property`

```python
text: str
```

A string of HTML representing this element.

### value `property` `writable`

```python
value: T
```

The element's current value.

### batch

```python
batch(**elements: UIElement[Any, Any]) -> batch
```

Convert an HTML object with templated text into a UI element.

This method lets you create custom UI elements that are represented
by arbitrary HTML.

> **Example**
>
> ```python3
> user_info = mo.md(
>     '''
>     - What's your name?: {name}
>     - When were you born?: {birthday}
>     '''
> ).batch(name=mo.ui.text(), birthday=mo.ui.date())
> ```
> In this example, `user_info` is a UI Element whose output is markdown
> and whose value is a dict with keys `'name'` and '`birthday`'
> (and values equal to the values of their corresponding elements).

| PARAMETER | DESCRIPTION |
| --- | --- |
| `elements` | the UI elements to interpolate into the HTML template. **TYPE:** `UIElement[Any, Any]` **DEFAULT:** `{}` |

### callout

```python
callout(
    kind: Literal[
        "neutral", "danger", "warn", "success", "info"
    ] = "neutral",
) -> Html
```

Create a callout containing this HTML element.

A callout wraps your HTML element in a raised box, emphasizing its
importance. You can style the callout for different situations with the
`kind` argument.

Examples:

```python3
mo.md("Hooray, you did it!").callout(kind="success")
```

```python3
mo.md("It's dangerous to go alone!").callout(kind="warn")
```

### center

```python
center() -> Html
```

Center an item.

> **Example**
>
> ```python3
> mo.md("# Hello, world").center()
> ```

| RETURNS | DESCRIPTION |
| --- | --- |
| `[Html](https://docs.marimo.io/api/html/#marimo.Html)` | An `Html` object. |

### form

```python
form(
    label: str = "",
    *,
    bordered: bool = True,
    loading: bool = False,
    submit_button_label: str = "Submit",
    submit_button_tooltip: str | None = None,
    submit_button_disabled: bool = False,
    clear_on_submit: bool = False,
    show_clear_button: bool = False,
    clear_button_label: str = "Clear",
    clear_button_tooltip: str | None = None,
    validate: Callable[[JSONType | None], str | None]
    | None = None,
    on_change: Callable[[T | None], None] | None = None,
) -> form[S, T]
```

Create a submittable form out of this `UIElement`.

Creates a form that gates submission of a `UIElement`'s value until a submit button is clicked.
The form's value is the value of the underlying element from the last submission.

Examples:

Convert any `UIElement` into a form:
```python
prompt = mo.ui.text_area().form()
```

Combine with `HTML.batch` to create a form made out of multiple `UIElements`:
```python
form = (
    mo.md(
        '''
    **Enter your prompt.**

    {prompt}

    **Choose a random seed.**

    {seed}
    '''
    )
    .batch(
        prompt=mo.ui.text_area(),
        seed=mo.ui.number(),
    )
    .form()
)
```

| PARAMETER | DESCRIPTION |
| --- | --- |
| `label` | A text label for the form. **TYPE:** `str` **DEFAULT:** `''` |
| `bordered` | Whether the form should have a border. **TYPE:** `bool` **DEFAULT:** `True` |
| `loading` | Whether the form should be in a loading state. **TYPE:** `bool` **DEFAULT:** `False` |
| `submit_button_label` | The label of the submit button. **TYPE:** `str` **DEFAULT:** `'Submit'` |
| `submit_button_tooltip` | The tooltip of the submit button. **TYPE:** `str \| None` **DEFAULT:** `None` |
| `submit_button_disabled` | Whether the submit button should be disabled. **TYPE:** `bool` **DEFAULT:** `False` |
| `clear_on_submit` | Whether the form should clear its contents after submitting. **TYPE:** `bool` **DEFAULT:** `False` |
| `show_clear_button` | Whether the form should show a clear button. **TYPE:** `bool` **DEFAULT:** `False` |
| `clear_button_label` | The label of the clear button. **TYPE:** `str` **DEFAULT:** `'Clear'` |
| `clear_button_tooltip` | The tooltip of the clear button. **TYPE:** `str \| None` **DEFAULT:** `None` |
| `validate` | A function that takes the form's value and returns an error message if invalid, or `None` if valid. **TYPE:** `Callable[[JSONType \| None], str \| None] \| None` **DEFAULT:** `None` |
| `on_change` | Optional callback to run when this element's value changes. Defaults to None. **TYPE:** `Callable[[T \| None], None] \| None` **DEFAULT:** `None` |

### from\_args `classmethod`

```python
from_args(
    data: dict[str, int],
    args: InitializationArgs[S, T],
    memo: dict[int, Any] | None = None,
    basis: UIElement[S, T] | None = None,
) -> UIElement[S, T]
```

### left

```python
left() -> Html
```

Left-justify.

> **Example**
>
> ```python3
> mo.md("# Hello, world").left()
> ```

| RETURNS | DESCRIPTION |
| --- | --- |
| `[Html](https://docs.marimo.io/api/html/#marimo.Html)` | An `Html` object. |

### right

```python
right() -> Html
```

Right-justify.

> **Example**
>
> ```python3
> mo.md("# Hello, world").right()
> ```

| RETURNS | DESCRIPTION |
| --- | --- |
| `[Html](https://docs.marimo.io/api/html/#marimo.Html)` | An `Html` object. |

### style

```python
style(
    style: dict[str, Any] | None = None, **kwargs: Any
) -> Html
```

Wrap an object in a styled container.

> **Example**
>
> ```python
> mo.md("...").style({"max-height": "300px", "overflow": "auto"})
> mo.md("...").style(max_height="300px", overflow="auto")
> ```

| PARAMETER | DESCRIPTION |
| --- | --- |
| `style` | an optional dict of CSS styles, keyed by property name **TYPE:** `dict[str, Any] \| None` **DEFAULT:** `None` |
| `**kwargs` | CSS styles as keyword arguments **TYPE:** `Any` **DEFAULT:** `{}` |

## Basic Usage

Here's a simple example using a custom echo model:

```python
import marimo as mo

def echo_model(messages, config):
    return f"Echo: {messages[-1].content}"

chat = mo.ui.chat(echo_model, prompts=["Hello", "How are you?"])
chat
```

Here, `messages` is a list of [`ChatMessage`](#marimo.ai.ChatMessage) objects,
which has `role` (`"user"`, `"assistant"`, or `"system"`) and `content` (the
message string) attributes; `config` is a
[`ChatModelConfig`](#marimo.ai.ChatModelConfig) object with various
configuration parameters, which you are free to ignore.

## Using Pydantic AI

marimo has first class support for [pydantic-ai](https://ai.pydantic.dev). Use the [`Agent`](https://ai.pydantic.dev/agents) class to build your chatbot and the Chat UI will display reasoning steps, tool calls and more.

```python
from pydantic_ai import Agent
import marimo as mo

assistant = Agent(
    "openai:gpt-5",
    system_prompt="You are a helpful assistant.",
)

chat = mo.ui.chat(mo.ai.llm.pydantic_ai(assistant))
chat
```

## Using a Built-in AI Model

You can use marimo's built-in AI models, such as OpenAI's GPT:

```python
import marimo as mo

chat = mo.ui.chat(
    mo.ai.llm.openai(
        "gpt-4",
        system_message="You are a helpful assistant.",
    ),
    show_configuration_controls=True
)
chat
```

## Accessing Chat History

You can access the chat history using the `value` attribute:

```python
chat.value
```

This returns a list of [`ChatMessage`](#marimo.ai.ChatMessage) objects, each
containing `id`, `role`, `parts` and `metadata` attributes. The `content` and `attachments` attributes are supported for basic models.

> **Note**
>
> For pydantic-ai, the messages are mapped to [Vercel UI messages](https://github.com/pydantic/pydantic-ai/blob/9aa6dd40efafd93c04c19c2ef5596a454906ca53/pydantic_ai_slim/pydantic_ai/ui/vercel_ai/request_types.py). To convert to Pydantic messages, use the
> adapter function.
> ```python
> from pydantic_ai.ui.vercel_ai import VercelAIAdapter
>
> messages = VercelAIAdapter.load_messages(chat.value)
> ```

## marimo.ai.ChatMessage

Bases: `Struct`

A message in a chat.

### attachments `class-attribute` `instance-attribute`

```python
attachments: list[ChatAttachment] | None = None
```

### content `instance-attribute`

```python
content: Any
```

### id `class-attribute` `instance-attribute`

```python
id: str = ''
```

### metadata `class-attribute` `instance-attribute`

```python
metadata: Any | None = None
```

### parts `class-attribute` `instance-attribute`

```python
parts: list[ChatPart] = []
```

### role `instance-attribute`

```python
role: Literal['user', 'assistant', 'system']
```

### create `classmethod`

```python
create(
    *,
    role: Literal["user", "assistant", "system"],
    message_id: str,
    content: str | None,
    parts: list[ChatPart],
    part_validator_class: Any | None = None,
) -> ChatMessage
```

Helper method to create a ChatMessage object.
If part\_validator\_class is provided, the parts will be converted to the given class.

## Custom Model with Additional Context

Here's an example of a custom model that uses additional context:

```python
import marimo as mo

def rag_model(messages, config):
    question = messages[-1].content
    docs = find_relevant_docs(question)
    context = "\n".join(docs)
    prompt = f"Context: {context}\n\nQuestion: {question}\n\nAnswer:"
    response = query_llm(prompt, config)
    return response

mo.ui.chat(rag_model)
```

This example demonstrates how you can implement a Retrieval-Augmented
Generation (RAG) model within the chat interface.

## Templated Prompts

You can pass sample prompts to `mo.ui.chat` to allow users to select from a
list of predefined prompts. By including a `{{var}}` in the prompt, you can
dynamically insert values into the prompt; a form will be generated to allow
users to fill in the variables.

```python
mo.ui.chat(
    mo.ai.llm.openai("gpt-4o"),
    prompts=[
        "What is the capital of France?",
        "What is the capital of Germany?",
        "What is the capital of {{country}}?",
    ],
)
```

## Including Attachments

You can allow users to upload attachments to their messages by passing an
`allow_attachments` parameter to `mo.ui.chat`.

```python
mo.ui.chat(
    rag_model,
    allow_attachments=["image/png", "image/jpeg"],
    # or True for any attachment type
    # allow_attachments=True,
)
```

## Streaming Responses

Chatbots can stream responses in real-time, creating a more interactive experience
similar to ChatGPT where you see the response appear word-by-word as it's generated.

Responses from built-in models (OpenAI, Anthropic, Google, Groq, Bedrock) are streamed by default.

### How Streaming Works

marimo uses **delta-based streaming**, which follows the industry-standard pattern used by OpenAI, Anthropic, and other AI providers. Your generator function should yield **individual chunks** (deltas) of new content, which marimo automatically accumulates and displays progressively.

### With Custom Models

For custom models, you can use either regular (sync) or async generator functions that yield delta chunks:

**Sync generator (simpler):**

```python
import marimo as mo
import time

def streaming_model(messages, config):
    """Stream responses word by word."""
    response = "This response will appear word by word!"
    words = response.split()

    for word in words:
        yield word + " "  # Yield delta chunks
        time.sleep(0.1)  # Simulate processing delay

chat = mo.ui.chat(streaming_model)
chat
```

**Async generator (for async operations):**

```python
import marimo as mo
import asyncio

async def async_streaming_model(messages, config):
    """Stream responses word by word asynchronously."""
    response = "This response will appear word by word!"
    words = response.split()

    for word in words:
        yield word + " "  # Yield delta chunks
        await asyncio.sleep(0.1)  # Async processing delay

chat = mo.ui.chat(async_streaming_model)
chat
```

Each `yield` sends a new chunk (delta) to marimo, which accumulates and displays
the progressively building response in real-time.

> **Delta vs Accumulated**
>
> **Yield deltas, not accumulated text.** Each yield should be **new content only**:
> ✅ **Correct (delta mode):**
> ```python
> yield "Hello"
> yield " "
> yield "world"
> # Result: "Hello world"
> ```
> ❌ **Incorrect (accumulated mode, deprecated):**
> ```python
> yield "Hello"
> yield "Hello "
> yield "Hello world"
> # Inefficient: sends duplicate content
> ```
> Delta mode is more efficient (reduces bandwidth by ~99% for long responses) and aligns with standard streaming APIs.

> **See streaming examples**
>
> For complete working examples, check out:
> - [`openai_example.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/openai_example.py) - OpenAI chatbot with streaming (default)
> - [`streaming_custom.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/streaming_custom.py) - Custom streaming chatbot

## Built-in Models

marimo provides several built-in AI models that you can use with the chat UI
element.

### Pydantic AI

```python
import marimo as mo
from pydantic_ai import Agent

assistant = Agent(
    "openai:gpt-5",
    system_prompt="You are a helpful assistant.",
)

mo.ui.chat(mo.ai.llm.pydantic_ai(assistant))
```

## marimo.ai.llm.pydantic\_ai

```python
pydantic_ai(agent: Agent[Any, Any])
```

Bases: `ChatModel`

[Pydantic AI](https://ai.pydantic.dev/) ChatModel

| PARAMETER | DESCRIPTION |
| --- | --- |
| `agent` | A pydantic\_ai Agent instance. [See docs](https://ai.pydantic.dev/agents/) **TYPE:** `Agent[Any, Any]` |

> **Example**
>
> ```python
> from pydantic_ai import Agent
>
> agent = Agent(
>     model="gpt-5", system_prompt="You are a helpful assistant."
> )
> chatbot = mo.ui.chat(
>     mo.ai.llm.pydantic_ai(agent),
>     prompts=["What is the capital of France?", "What is marimo?"],
> )
> chatbot
> ```

### agent `instance-attribute`

```python
agent = agent
```

### OpenAI

```python
import marimo as mo

mo.ui.chat(
    mo.ai.llm.openai(
        "gpt-4o",
        system_message="You are a helpful assistant.",
        api_key="sk-proj-...",
    ),
    show_configuration_controls=True
)
```

## marimo.ai.llm.openai

```python
openai(
    model: str,
    *,
    system_message: str = DEFAULT_SYSTEM_MESSAGE,
    api_key: str | None = None,
    base_url: str | None = None,
)
```

Bases: `ChatModel`

OpenAI ChatModel

| PARAMETER | DESCRIPTION |
| --- | --- |
| `model` | The model to use. Can be found on the [OpenAI models page](https://platform.openai.com/docs/models) **TYPE:** `str` |
| `system_message` | The system message to use **TYPE:** `str` **DEFAULT:** `DEFAULT_SYSTEM_MESSAGE` |
| `api_key` | The API key to use. If not provided, the API key will be retrieved from the OPENAI\_API\_KEY environment variable or the user's config. **TYPE:** `str \| None` **DEFAULT:** `None` |
| `base_url` | The base URL to use **TYPE:** `str \| None` **DEFAULT:** `None` |

### api\_key `instance-attribute`

```python
api_key = api_key
```

### base\_url `instance-attribute`

```python
base_url = base_url
```

### model `instance-attribute`

```python
model = model
```

### system\_message `instance-attribute`

```python
system_message = system_message
```

### Anthropic

```python
import marimo as mo

mo.ui.chat(
    mo.ai.llm.anthropic(
        "claude-3-5-sonnet-20240620",
        system_message="You are a helpful assistant.",
        api_key="sk-ant-...",
    ),
    show_configuration_controls=True
)
```

## marimo.ai.llm.anthropic

```python
anthropic(
    model: str,
    *,
    system_message: str = DEFAULT_SYSTEM_MESSAGE,
    api_key: str | None = None,
    base_url: str | None = None,
)
```

Bases: `ChatModel`

Anthropic ChatModel

| PARAMETER | DESCRIPTION |
| --- | --- |
| `model` | The model to use. Can be found on the [Anthropic models page](https://docs.anthropic.com/en/docs/about-claude/models) **TYPE:** `str` |
| `system_message` | The system message to use **TYPE:** `str` **DEFAULT:** `DEFAULT_SYSTEM_MESSAGE` |
| `api_key` | The API key to use. If not provided, the API key will be retrieved from the ANTHROPIC\_API\_KEY environment variable or the user's config. **TYPE:** `str \| None` **DEFAULT:** `None` |
| `base_url` | The base URL to use **TYPE:** `str \| None` **DEFAULT:** `None` |

### api\_key `instance-attribute`

```python
api_key = api_key
```

### base\_url `instance-attribute`

```python
base_url = base_url
```

### model `instance-attribute`

```python
model = model
```

### system\_message `instance-attribute`

```python
system_message = system_message
```

### supports\_temperature

```python
supports_temperature(model: str) -> bool
```

### Google AI

```python
import marimo as mo

mo.ui.chat(
    mo.ai.llm.google(
        "gemini-1.5-pro-latest",
        system_message="You are a helpful assistant.",
        api_key="AI..",
    ),
    show_configuration_controls=True
)
```

## marimo.ai.llm.google

```python
google(
    model: str,
    *,
    system_message: str = DEFAULT_SYSTEM_MESSAGE,
    api_key: str | None = None,
)
```

Bases: `ChatModel`

Google AI ChatModel

| PARAMETER | DESCRIPTION |
| --- | --- |
| `model` | The model to use. Can be found on the [Gemini models page](https://ai.google.dev/gemini-api/docs/models/gemini) **TYPE:** `str` |
| `system_message` | The system message to use **TYPE:** `str` **DEFAULT:** `DEFAULT_SYSTEM_MESSAGE` |
| `api_key` | The API key to use. If not provided, the API key will be retrieved from the GOOGLE\_AI\_API\_KEY environment variable or the user's config. **TYPE:** `str \| None` **DEFAULT:** `None` |

### api\_key `instance-attribute`

```python
api_key = api_key
```

### model `instance-attribute`

```python
model = model
```

### system\_message `instance-attribute`

```python
system_message = system_message
```

### Groq

```python
import marimo as mo

mo.ui.chat(
    mo.ai.llm.groq(
        "llama-3.1-70b-versatile",
        system_message="You are a helpful assistant.",
        api_key="gsk-...",
    ),
    show_configuration_controls=True
)
```

## marimo.ai.llm.groq

```python
groq(
    model: str,
    *,
    system_message: str = DEFAULT_SYSTEM_MESSAGE,
    api_key: str | None = None,
    base_url: str | None = None,
)
```

Bases: `ChatModel`

Groq ChatModel

| PARAMETER | DESCRIPTION |
| --- | --- |
| `model` | The model to use. Can be found on the [Groq models page](https://console.groq.com/docs/models) **TYPE:** `str` |
| `system_message` | The system message to use **TYPE:** `str` **DEFAULT:** `DEFAULT_SYSTEM_MESSAGE` |
| `api_key` | The API key to use. If not provided, the API key will be retrieved from the GROQ\_API\_KEY environment variable or the user's config. **TYPE:** `str \| None` **DEFAULT:** `None` |
| `base_url` | The base URL to use **TYPE:** `str \| None` **DEFAULT:** `None` |

### api\_key `instance-attribute`

```python
api_key = api_key
```

### base\_url `instance-attribute`

```python
base_url = base_url
```

### model `instance-attribute`

```python
model = model
```

### system\_message `instance-attribute`

```python
system_message = system_message
```

## Types

Chatbots can be implemented with a function that receives a list of
[`ChatMessage`](#marimo.ai.ChatMessage) objects and a
[`ChatModelConfig`](#marimo.ai.ChatModelConfig).

## marimo.ai.ChatMessage

Bases: `Struct`

A message in a chat.

### attachments `class-attribute` `instance-attribute`

```python
attachments: list[ChatAttachment] | None = None
```

### content `instance-attribute`

```python
content: Any
```

### id `class-attribute` `instance-attribute`

```python
id: str = ''
```

### metadata `class-attribute` `instance-attribute`

```python
metadata: Any | None = None
```

### parts `class-attribute` `instance-attribute`

```python
parts: list[ChatPart] = []
```

### role `instance-attribute`

```python
role: Literal['user', 'assistant', 'system']
```

### create `classmethod`

```python
create(
    *,
    role: Literal["user", "assistant", "system"],
    message_id: str,
    content: str | None,
    parts: list[ChatPart],
    part_validator_class: Any | None = None,
) -> ChatMessage
```

Helper method to create a ChatMessage object.
If part\_validator\_class is provided, the parts will be converted to the given class.

## marimo.ai.ChatModelConfig `dataclass`

```python
ChatModelConfig(
    max_tokens: int | None = None,
    temperature: float | None = None,
    top_p: float | None = None,
    top_k: int | None = None,
    frequency_penalty: float | None = None,
    presence_penalty: float | None = None,
)
```

### frequency\_penalty `class-attribute` `instance-attribute`

```python
frequency_penalty: float | None = None
```

### max\_tokens `class-attribute` `instance-attribute`

```python
max_tokens: int | None = None
```

### presence\_penalty `class-attribute` `instance-attribute`

```python
presence_penalty: float | None = None
```

### temperature `class-attribute` `instance-attribute`

```python
temperature: float | None = None
```

### top\_k `class-attribute` `instance-attribute`

```python
top_k: int | None = None
```

### top\_p `class-attribute` `instance-attribute`

```python
top_p: float | None = None
```

[`mo.ui.chat`](#marimo.ui.chat) can be instantiated with an initial
configuration with a dictionary conforming to the config.

`ChatMessage`s can also include attachments.

## marimo.ai.ChatAttachment `dataclass`

```python
ChatAttachment(
    url: str,
    name: str = "attachment",
    content_type: str | None = None,
)
```

### content\_type `class-attribute` `instance-attribute`

```python
content_type: str | None = None
```

### name `class-attribute` `instance-attribute`

```python
name: str = 'attachment'
```

### url `instance-attribute`

```python
url: str
```

## Supported Model Providers

We support any OpenAI-compatible endpoint. If you want any specific provider added explicitly (ones that don't abide by the standard OpenAI API format), you can file a [feature request](https://github.com/marimo-team/marimo/issues/new?template=feature_request.yaml).

Normally, overriding the `base_url` parameter should work. Here are some examples:

**Cerebras:**

```python
chatbot = mo.ui.chat(
    mo.ai.llm.openai(
        model="llama3.1-8b",
        api_key="csk-...", # insert your key here
        base_url="https://api.cerebras.ai/v1/",
    ),
)
chatbot
```

**Groq:**

```python
chatbot = mo.ui.chat(
    mo.ai.llm.openai(
        model="llama-3.1-70b-versatile",
        api_key="gsk_...", # insert your key here
        base_url="https://api.groq.com/openai/v1/",
    ),
)
chatbot
```

**xAI:**

```python
chatbot = mo.ui.chat(
    mo.ai.llm.openai(
        model="grok-beta",
        api_key=key, # insert your key here
        base_url="https://api.x.ai/v1",
    ),
)
chatbot
```

> **Note**
>
> We have added examples for GROQ and Cerebras. These providers offer free API keys and are great for trying out Llama models (from Meta). You can sign up on their platforms and integrate with various AI integrations in marimo easily. For more information, refer to the [AI completion documentation in marimo](https://docs.marimo.io/guides/editor_features/ai_completion/).