# What Kind of Data Does embabel-agent Process? A Complete Guide to Multimodal Content Handling

> Discover what kind of data embabel-agent processes. This guide covers multimodal content handling including text, images, and documents via a type-safe Kotlin API.

- Repository: [Embabel/embabel-agent](https://github.com/embabel/embabel-agent)
- Tags: deep-dive
- Published: 2026-08-14

---

**embabel-agent processes multimodal messages combining plain text, images, and documents through a type-safe Kotlin API with strict validation and size limits.**

The embabel-agent framework, developed by [Embabel](https://github.com/embabel), provides a structured approach to handling diverse data types for LLM interactions. Its data model centers on the `ContentPart` hierarchy defined in [`embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt), enabling seamless integration of textual and binary content in a single conversation flow.

## Core Data Types in embabel-agent

The framework recognizes three primary data kinds, each implemented as a sealed subtype of `ContentPart`. This design ensures exhaustive pattern matching and compile-time safety when processing multimodal inputs.

### Plain Text (TextPart)

The simplest content type, **TextPart** wraps a non-empty string:

```kotlin
data class TextPart(val text: String) : ContentPart {
    init {
        require(text.isNotEmpty()) { "Text must not be empty" }
    }
}

```

According to the embabel-agent source code at [lines 41-44](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt#L41-L44), empty strings trigger an `IllegalArgumentException`. This prevents null or blank messages from propagating through the pipeline.

### Images (ImagePart)

**ImagePart** handles binary image data with strict validation requirements:

| Attribute | Specification |
|-----------|---------------|
| MIME type | Must satisfy `isImageMimeType()` (e.g., `image/png`, `image/jpeg`) |
| Payload | Non-empty `ByteArray` |
| Size limit | 20 MiB maximum (`MAX_IMAGE_SIZE`) |

The constructor validates all constraints at instantiation:

```kotlin
val imgBytes = Files.readAllBytes(Paths.get("diagram.png"))
val image = ImagePart("image/png", imgBytes)  // throws if >20 MiB or invalid MIME

```

Implementation details reside in [lines 50-62](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt#L50-L62) of [`ContentPart.kt`](https://github.com/embabel/embabel-agent/blob/main/ContentPart.kt). The `MediaPart` interface (sealed under `ContentPart`) provides the common contract for all binary types.

### Documents (DocumentPart)

**DocumentPart** extends support to generic document formats:

| Attribute | Specification |
|-----------|---------------|
| MIME type | Must satisfy `isDocumentMimeType()` (e.g., `application/pdf`, `text/plain`) |
| Payload | Non-empty `ByteArray` |
| Filename | Optional, but non-blank if provided |
| Size limit | 20 MiB maximum (`MAX_DOCUMENT_SIZE`) |

```kotlin
val pdfBytes = Files.readAllBytes(Paths.get("contract.pdf"))
val document = DocumentPart(
    mimeType = "application/pdf",
    data = pdfBytes,
    filename = "contract.pdf"  // optional parameter
)

```

The validation logic at [lines 85-99](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt#L85-L99) enforces these constraints consistently.

## Validation and Type Safety Architecture

### MIME Type Verification

embabel-agent delegates MIME type validation to helper functions in the common module:

- `isImageMimeType(mimeType: String)` — defined in [`MediaMimeTypeUtils.kt`](https://github.com/embabel/embabel-agent/blob/main/MediaMimeTypeUtils.kt)
- `isDocumentMimeType(mimeType: String)` — defined in [`MediaMimeTypeUtils.kt`](https://github.com/embabel/embabel-agent/blob/main/MediaMimeTypeUtils.kt)

These functions reference canonical lists maintained in [`MimeTypes.kt`](https://github.com/embabel/embabel-agent/blob/main/MimeTypes.kt) (`embabel-agent-common/embabel-agent-common/src/main/kotlin/com/embabel/common/ai/media/`). The separation of concerns allows supported formats to evolve without modifying core data classes.

### Sealed Interface Extensibility

`MediaPart` is declared as a sealed interface beneath the `ContentPart` root. This architecture enables future expansion:

```kotlin
// Current hierarchy (simplified)
sealed interface ContentPart
sealed interface MediaPart : ContentPart  // mimeType: String, data: ByteArray
data class TextPart(...) : ContentPart
data class ImagePart(...) : MediaPart
data class DocumentPart(...) : MediaPart
// Future: AudioPart, VideoPart, etc.

```

The sealed constraint ensures all implementations are known at compile time, while the interface abstraction permits polymorphic handling of binary content.

## Practical Usage: Composing Multimodal Messages

Individual `ContentPart` instances assemble into collections for LLM submission:

```kotlin
import com.embabel.chat.*

// Build a mixed-content message
val messageParts: List<ContentPart> = listOf(
    TextPart("Analyze this quarterly report and accompanying chart."),
    DocumentPart("application/pdf", reportBytes, "Q3_Financials.pdf"),
    ImagePart("image/png", chartBytes)
)

// Pass to agent for processing
val response = embabelAgent.send(messageParts)

```

Each element undergoes validation at construction, failing fast with descriptive exceptions rather than deferring errors to the LLM layer.

## Key Implementation Files

| Path | Responsibility |
|------|----------------|
| [`embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-api/src/main/kotlin/com/embabel/chat/ContentPart.kt) | `ContentPart` sealed class, `TextPart`, `ImagePart`, `DocumentPart` definitions |
| [`embabel-agent-common/embabel-agent-common/src/main/kotlin/com/embabel/common/ai/media/MimeTypes.kt`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-common/embabel-agent-common/src/main/kotlin/com/embabel/common/ai/media/MimeTypes.kt) | Supported MIME type constants |
| [`embabel-agent-common/embabel-agent-common/src/main/kotlin/com/embabel/common/ai/media/MediaMimeTypeUtils.kt`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-common/embabel-agent-common/src/main/kotlin/com/embabel/common/ai/media/MediaMimeTypeUtils.kt) | `isImageMimeType()`, `isDocumentMimeType()` validation functions |

## Summary

- **Text** — `TextPart` with non-empty string requirement
- **Images** — `ImagePart` with 20 MiB limit and image-specific MIME validation
- **Documents** — `DocumentPart` with 20 MiB limit, optional filename, and document-specific MIME validation
- **Architecture** — Sealed `ContentPart` hierarchy with `MediaPart` interface for extensible binary handling
- **Validation** — Eager constructor checks using `require()` blocks and dedicated MIME type utilities

## Frequently Asked Questions

### What is the maximum file size for images and documents in embabel-agent?

Both images and documents are limited to **20 MiB** per part, enforced by `MAX_IMAGE_SIZE` and `MAX_DOCUMENT_SIZE` constants. The `ImagePart` and `DocumentPart` constructors throw `IllegalArgumentException` if this threshold is exceeded.

### Can embabel-agent process audio or video files?

Not currently in the core distribution. However, the `MediaPart` sealed interface architecture permits extension with new binary types. Implementing `AudioPart` or `VideoPart` would require defining appropriate MIME type validators in [`MediaMimeTypeUtils.kt`](https://github.com/embabel/embabel-agent/blob/main/MediaMimeTypeUtils.kt) and size constraints analogous to existing media parts.

### How does embabel-agent handle unsupported MIME types?

The framework rejects unknown MIME types at instantiation. Image parts must satisfy `isImageMimeType()`; document parts must satisfy `isDocumentMimeType()`. These functions check against enumerated lists in [`MimeTypes.kt`](https://github.com/embabel/embabel-agent/blob/main/MimeTypes.kt), preventing invalid content from reaching downstream processors.

### What happens if I provide an empty filename for a DocumentPart?

The `filename` parameter is optional (nullable), but if supplied it must be non-blank. The constructor at `ContentPart.kt:85-99` validates `require(filename == null || filename.isNotBlank())`, throwing an exception for empty or whitespace-only filenames.