# How to Set Up an AI Agent with the Agno Framework: A Complete Guide

> Learn to set up an AI agent with the Agno framework. This guide covers installation, agent instantiation with models and tools, and running queries for powerful AI applications.

- Repository: [Shubham Saboo/awesome-llm-apps](https://github.com/shubhamsaboo/awesome-llm-apps)
- Tags: how-to-guide
- Published: 2026-02-16

---

**You can set up an AI agent with the Agno framework by installing the dependencies, instantiating the `Agent` class with a model like `OpenAIChat` and optional tools, then calling the `run()` method to process user queries.**

The Agno framework (formerly Phidata) provides a lightweight, plug-and-play architecture for building AI agents that interact with LLMs and external tools. In the `Shubhamsaboo/awesome-llm-apps` repository, you'll find production-ready examples demonstrating how to set up an AI agent with the Agno framework for data analysis, media processing, and autonomous task execution.

## Core Concepts and Architecture

### The Agent Class

The `Agent` class serves as the central orchestrator in the Agno framework. Located throughout the repository (notably in [`starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py)), this class accepts a model, tools list, system message, and provides `run()` and `stream()` methods for executing queries.

### Models

Agno supports multiple providers through unified wrappers. You can use `OpenAIChat` for OpenAI models, `Gemini` for Google models, or `DeepSeek` for reasoning tasks. Each wrapper accepts an API key and model identifier.

### Tools

Tools extend agent capabilities beyond text generation. The repository demonstrates `DuckDbTools` for SQL queries, `PandasTools` for dataframe operations, and custom browser automation tools. These are passed as a list to the `Agent` constructor.

### Media Handling

For multimodal inputs, Agno provides `AgnoImage` and `AgnoAudio` classes in `agno/media/`. These wrap file paths to ensure proper encoding for vision-capable models.

## Step-by-Step Setup Guide

### 1. Install Dependencies

Clone the repository and install the required packages:

```bash
git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git
cd awesome-llm-apps/starter_ai_agents/ai_data_analysis_agent
pip install -r requirements.txt

```

The [`requirements.txt`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/requirements.txt) files pin Agno to version ≥2.0.4.

### 2. Configure API Keys

Obtain API keys from your chosen provider:

- **OpenAI**: platform.openai.com/account/api-keys
- **Gemini**: makersuite.google.com/app/apikey
- **DeepSeek**: deepseek.com/api-keys

Store these securely in environment variables or Streamlit session state. Never hard-code secrets in production code.

### 3. Initialize the Agent

Create your agent by combining a model, tools, and system instructions:

```python
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.duckdb import DuckDbTools
from agno.tools.pandas import PandasTools

data_analyst_agent = Agent(
    model=OpenAIChat(id="gpt-4o", api_key=st.session_state.openai_key),
    tools=[DuckDbTools(), PandasTools()],
    system_message=(
        "You are an expert data analyst. Use the 'uploaded_data' table to answer user queries. "
        "Generate SQL queries using DuckDB tools to solve the user's query."
    ),
    markdown=True,
)

```

The `system_message` parameter defines the agent's persona and constraints.

### 4. Handle Media Inputs (Optional)

For image analysis, wrap files using `AgnoImage`:

```python
from agno.media import Image as AgnoImage
from pathlib import Path

image = AgnoImage(filepath=Path("screenshot.png"))
response = agent.run("Describe this image", images=[image])

```

This pattern appears in [`starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py) for processing user-uploaded images.

### 5. Execute and Display Results

Trigger the agent from your UI:

```python
if st.button("Analyze"):
    with st.spinner("Processing..."):
        result = data_analyst_agent.run(user_query)
    st.markdown(result.content)

```

The `run()` method returns a response object with a `.content` attribute containing the formatted output.

## Production Examples from the Repository

### AI Data Analyst

Located in [`starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py), this agent combines `DuckDbTools` and `PandasTools` to let users query CSV/Excel files using natural language. The agent automatically generates and executes SQL queries based on user questions.

### Breakup Recovery Squad

Found in [`starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py), this demonstrates multi-agent orchestration and image handling via `AgnoImage`. It showcases how to build empathetic AI interactions with multimodal inputs.

### 3D PyGame Code Generator

In [`advanced_ai_agents/autonomous_game_playing_agent_apps/ai_3dpygame_r1/ai_3dpygame_r1.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/advanced_ai_agents/autonomous_game_playing_agent_apps/ai_3dpygame_r1/ai_3dpygame_r1.py), this example shows mixed-model orchestration where DeepSeek handles reasoning tasks while OpenAI manages code extraction. It also implements async browser automation tools.

## Common Pitfalls and Solutions

| Issue | Cause | Solution |
|-------|-------|----------|
| Agent ignores tools | Missing `tools=` parameter or uninstantiated tool classes | Pass instantiated tool objects like `[DuckDbTools(), PandasTools()]` to the `Agent` constructor |
| Images not processed | Using raw file paths instead of `AgnoImage` wrapper | Import `AgnoImage` from `agno.media` and wrap file paths before passing to `run()` |
| API rate limits | Concurrent calls exceeding quota | Enable `debug_mode=True` to inspect raw calls, implement retry logic, or upgrade API tier |
| UI freezing | Synchronous blocking without feedback | Wrap `agent.run()` in `st.spinner()` or execute in background threads |

## Summary

- **Install** Agno framework via pip (version ≥2.0.4) from the repository requirements.
- **Configure** API keys for OpenAI, Gemini, or DeepSeek without hard-coding secrets.
- **Instantiate** the `Agent` class with a model wrapper, tool list, and system message.
- **Execute** queries using `run()` for synchronous responses or `stream()` for real-time output.
- **Extend** capabilities by adding tools like `DuckDbTools` or media handlers like `AgnoImage`.

## Frequently Asked Questions

### What is the Agno framework and how does it differ from other agent frameworks?

Agno (formerly Phidata) is a lightweight, plug-and-play Python framework designed specifically for building LLM agents with minimal boilerplate. Unlike heavier orchestration frameworks, Agno focuses on unified model wrappers and first-class tool integration, allowing you to set up an AI agent with just a few lines of code in [`starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_data_analysis_agent/ai_data_analyst.py).

### Can I use Agno with models other than OpenAI?

Yes. Agno supports multiple providers including Gemini, DeepSeek, and others through provider-specific wrappers. The repository demonstrates mixed-model orchestration in [`advanced_ai_agents/autonomous_game_playing_agent_apps/ai_3dpygame_r1/ai_3dpygame_r1.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/advanced_ai_agents/autonomous_game_playing_agent_apps/ai_3dpygame_r1/ai_3dpygame_r1.py), where DeepSeek handles reasoning tasks while OpenAI manages code extraction.

### How do I add custom tools to my Agno agent?

You instantiate tool classes like `DuckDbTools()` or `PandasTools()` and pass them as a list to the `tools=` parameter when creating your `Agent`. For custom functionality, you can write Python functions and register them as tools using Agno's decorator pattern, though the repository primarily uses built-in tool classes for database and media operations.

### What is the best way to handle images in Agno agents?

Always wrap image file paths using the `AgnoImage` class from `agno.media` before passing them to the `run()` method. This ensures proper encoding and metadata formatting for multimodal models. The breakup-recovery agent in [`starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py`](https://github.com/Shubhamsaboo/awesome-llm-apps/blob/main/starter_ai_agents/ai_breakup_recovery_agent/ai_breakup_recovery_agent.py) demonstrates this pattern for processing user-uploaded images.