# How the Marin API Works: Architecture and Endpoints Explained

> Explore the Marin API architecture and endpoints. Learn how it converts JSON payloads into dataclasses and orchestrates pipeline execution via a modular step runner. Understand the Marin API's inner workings.

- Repository: [The Marin Project/marin](https://github.com/marin-community/marin)
- Tags: architecture
- Published: 2026-08-27

---

**The Marin API provides a FastAPI-based HTTP interface that converts JSON payloads into internal dataclasses and orchestrates pipeline execution through a modular step runner.**

The marin-community/marin repository implements a modular pipeline framework designed for data-processing and model-training workflows. The **Marin API** exposes these capabilities through a programmatic REST interface, allowing users to create experiments, monitor distributed sweep progress, and retrieve profiling data through standardized HTTP endpoints.

## Core Architecture and Entry Points

The Marin API follows a layered architecture that separates HTTP transport from business logic. The web layer resides in the `lib/marin/src/marin/web/` package and handles request routing, validation, and dispatch.

### FastAPI Application Registration

The primary entry point lives in [`lib/marin/src/marin/web/__init__.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/__init__.py). This module instantiates the FastAPI application and mounts the route handlers that constitute the public API. When the server starts, this file registers endpoints such as `POST /v1/experiments` and `GET /v1/sweeps/{sweep_id}`.

### Request Conversion and Validation

Incoming JSON undergoes type conversion in [`lib/marin/src/marin/web/convert.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/convert.py). This utility module translates raw HTTP payloads into internal Marin objects—specifically `Experiment`, `Sweep`, and `StepSpec` instances. The conversion relies on Pydantic models defined in the `lib/marin/src/marin/experiment` module to enforce schema validation before objects pass to the execution engine.

## API Request Lifecycle

When a client issues a request to the Marin API, the framework processes it through three distinct stages:

1. **Routing**: The FastAPI router receives the request and validates the HTTP method and path against handlers registered in [`lib/marin/src/marin/web/__init__.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/__init__.py).
2. **Deserialization**: The [`convert.py`](https://github.com/marin-community/marin/blob/main/convert.py) module transforms validated JSON into strongly-typed dataclasses using the Pydantic models.
3. **Execution**: The request dispatches to the engine in [`lib/marin/src/marin/execution/step_runner.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/execution/step_runner.py), which orchestrates the actual work and optionally delegates to external backends such as Iris or VLLM.

## CRUD Operations and Endpoint Examples

The Marin API supports full **CRUD operations** for experiments, sweeps, checkpoints, and profiling data. The following examples demonstrate authenticated requests using Python's `requests` library.

### Creating an Experiment

Submit a new experiment configuration via `POST /v1/experiments`:

```python
import requests

exp_payload = {
    "name": "my_experiment",
    "config": {"model": "gpt-2", "batch_size": 32},
}
resp = requests.post(
    "https://api.marin.dev/v1/experiments",
    json=exp_payload,
    headers={"Authorization": "Bearer <TOKEN>"},
)
print(resp.json())   # => {"id": "...", "status": "created"}

```

### Monitoring Sweep Status

Query the execution state of a running sweep using `GET /v1/sweeps/{sweep_id}`:

```python
sweep_id = "sweep-123"
status = requests.get(
    f"https://api.marin.dev/v1/sweeps/{sweep_id}",
    headers={"Authorization": "Bearer <TOKEN>"},
).json()
print(status)        # => {"state": "RUNNING", "progress": 0.57}

```

### Uploading Profiling Data

Submit performance metrics via `POST /v1/profiling`:

```python
profile = {"step": "train", "latency_ms": 123, "memory_mb": 2048}
requests.post(
    "https://api.marin.dev/v1/profiling",
    json=profile,
    headers={"Authorization": "Bearer <TOKEN>"},
)

```

## Response Serialization

The Marin API returns JSON-serialized responses using a custom encoder defined in [`lib/marin/src/marin/utilities/json_encoder.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/utilities/json_encoder.py). This encoder handles complex datatypes that standard JSON libraries cannot process, ensuring consistent serialization of experiment states, sweep configurations, and checkpoint metadata.

## Summary

- **Entry Point**: [`lib/marin/src/marin/web/__init__.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/__init__.py) registers the FastAPI application and public routes.
- **Type Conversion**: [`lib/marin/src/marin/web/convert.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/convert.py) transforms JSON payloads into validated `Experiment`, `Sweep`, and `StepSpec` objects.
- **Execution Engine**: [`lib/marin/src/marin/execution/step_runner.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/execution/step_runner.py) orchestrates pipeline steps and delegates to external backends when required.
- **Serialization**: [`lib/marin/src/marin/utilities/json_encoder.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/utilities/json_encoder.py) provides custom JSON encoding for API responses.
- **Endpoint Coverage**: The API exposes CRUD operations for experiments, sweeps, checkpoints, and profiling data through standard HTTP methods.

## Frequently Asked Questions

### What authentication method does the Marin API require?

The Marin API implements Bearer token authentication. Clients must include an `Authorization` header with the value `Bearer <TOKEN>` in every HTTP request to authenticate against the endpoints.

### Which file handles the conversion from JSON to internal Marin objects?

The [`lib/marin/src/marin/web/convert.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/web/convert.py) module contains the utility functions that translate incoming JSON payloads into internal dataclasses after they pass Pydantic validation in the `lib/marin/src/marin/experiment` module.

### Can the Marin API delegate execution to external backends?

Yes. According to the source code in [`lib/marin/src/marin/execution/step_runner.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/execution/step_runner.py), the execution engine orchestrates workloads and can optionally delegate processing to external backends such as Iris or VLLM for specialized computation.

### What content type does the Marin API return?

The Marin API returns JSON-serialized responses using the custom encoder located in [`lib/marin/src/marin/utilities/json_encoder.py`](https://github.com/marin-community/marin/blob/main/lib/marin/src/marin/utilities/json_encoder.py), which handles complex datatypes beyond standard JSON serialization capabilities.