Calliope Hardware and Client Applications: ESP32-Sparrow, Clio, and Custom API Clients

Calliope supports three client categories: the ESP32-Sparrow hardware device, the Clio TypeScript web application, and any generic HTTP client that implements the /v1/frames/ or /v2/stories/ API contract.

The chrisimmel/calliope repository defines a flexible architecture designed to generate interactive stories from any HTTP-capable device. While the server accepts requests from virtually any client, it officially maintains and documents two specific implementations—the ESP32-Sparrow embedded hardware and the Clio browser client—alongside a generic API that enables custom integrations.

Officially Supported Client Types

Calliope categorizes clients based on their hardware capabilities and interaction models. Each client type registers itself using a unique client_id and optional client_type parameter, allowing the server to apply appropriate defaults for media handling and display formatting.

ESP32-Sparrow Hardware

The ESP32-Sparrow is a bespoke embedded device built around the ESP32 microcontroller. According to docs/config.md, this hardware family represents "the primary family of sparrows" within the Calliope ecosystem.

Key hardware characteristics include:

• A built-in screen for displaying story frames
• Optional camera module for capturing images
• Optional microphone for audio recording
• Native RGB565 image format support for efficient display rendering

When a Sparrow sends a request to /v1/frames/, it includes base64-encoded media that calliope/utils/image.py decodes—handling the RGB565 format specifically when the client_type is set to "sparrow". The server maintains per-device state in calliope/tables/sparrow_state.py, tracking the current story_id, last frame timestamp, and uploaded media metadata.

Clio Web Client

Clio is a TypeScript single-page application that runs in any modern web browser (desktop or mobile). As documented in docs/Clio.md, Clio provides a complete interactive interface without requiring dedicated hardware.

Clio capabilities include:

• Webcam photo capture directly from the browser
• Audio recording through the device microphone
• Touch or click navigation between story frames
• Real-time display of generated text and images

Unlike the ESP32-Sparrow, Clio clients use standard JPEG or PNG image formats rather than RGB565. The browser client identifies itself with client_type: "clio" and stores configuration data in calliope/tables/config.py alongside Sparrow records.

Generic API Clients

Any application capable of issuing HTTP/HTTPS requests can function as a Calliope client. The API contract, defined in docs/api.md, requires only two parameters:

• client_id (required): A unique string identifying the device or application instance
• client_type (optional): A discriminator value ("sparrow", "clio", or custom) that selects default configuration parameters

Custom clients may optionally include base64-encoded images or audio in the JSON payload. Because the client_type field primarily influences default strategy selection and media preprocessing, developers can implement clients in Python, JavaScript, mobile native code, or any language supporting HTTP requests.

Architecture Integration

Understanding how Calliope processes client requests reveals why these three client types coexist seamlessly within the same infrastructure.

Client Registration and State Management

Upon receiving a request, Calliope validates the client_id and retrieves or creates a state record. For hardware devices, this uses the SparrowState model in calliope/tables/sparrow_state.py, which persists:

• Current story progression
• Last frame identifier
• Media metadata and temporary storage paths

Web and custom clients utilize SparrowConfig (defined in calliope/tables/config.py) for device-agnostic configuration storage. Both tables map the client_id to story context, ensuring continuity across disconnected sessions.

Media Processing Pipeline

When a client uploads media, the request flows through calliope/utils/image.py for decoding. The utility detects RGB565 payloads for ESP32-Sparrow clients and converts them to standard formats for processing by the story generation strategies. Audio data follows a similar path, with format normalization occurring before the StoryRequest model passes the content to the active strategy.

Story Generation and Response

The server invokes a story strategy—such as strategies/simple_one_frame.py—which receives the normalized request including any client-supplied media. After generating the next narrative frame and optional images, Calliope:

1. Persists the frame to the database
2. Updates the client's SparrowState or SparrowConfig record
3. Returns a JSON payload containing the story text, image URLs, and updated story_id

Implementation Examples

The following examples demonstrate how to interact with Calliope from each supported client category.

Command-Line API Request

Send a story request from any shell using curl:

curl -X POST "http://localhost:8008/v1/frames/?client_id=sparrow_01&client_type=sparrow" \
     -H "Content-Type: application/json" \
     -d '{
           "text": "Once upon a time...",
           "image": "<base64-encoded-jpeg>"
         }'

The response contains the next story frame and updated state, as documented in docs/api.md.

Python Client Implementation

A minimal Python client using httpx works with any client_type:

import httpx
import base64
import pathlib

image_path = pathlib.Path("photo.jpg")
b64_image = base64.b64encode(image_path.read_bytes()).decode()

payload = {
    "text": "The robot looked at the sky.",
    "image": b64_image,
}

resp = httpx.post(
    "http://localhost:8008/v1/frames/",
    params={"client_id": "my_custom_app", "client_type": "custom"},
    json=payload,
)
print(resp.json())

This approach works because Calliope requires only the client_id and accepts optional client_type and media fields.

ESP32-Sparrow Firmware Pattern

Embedded firmware for the ESP32-Sparrow follows the same JSON contract, encoding RGB565 frames:

// Inside the ESP32-Sparrow sketch
const char* SERVER = "http://calliope.local:8008";
const char* CLIENT_ID = "sparrow_42";

void send_frame(const char* text, const uint8_t* img_rgb565, size_t img_len) {
    httpClient.begin(SERVER "/v1/frames/");
    httpClient.addHeader("Content-Type", "application/json");
    String json = "{\"text\":\"" + String(text) + "\",\"image\":\"";
    json += base64::encode(img_rgb565, img_len);
    json += "\"}";
    httpClient.POST(json);
    // handle response …
}

The firmware sends RGB565 data that calliope/utils/image.py specifically handles for client_type: "sparrow" requests.

Summary

• ESP32-Sparrow provides dedicated hardware with screen, camera, and microphone support, using RGB565 image encoding as defined in docs/config.md.
• Clio delivers a browser-based TypeScript client for desktop and mobile users, documented in docs/Clio.md.
• Generic HTTP clients require only a client_id parameter and optional client_type to interact with /v1/frames/ or /v2/stories/ endpoints.
• State persistence occurs in calliope/tables/sparrow_state.py for hardware and calliope/tables/config.py for all client types.
• Media handling in calliope/utils/image.py adapts to hardware-specific formats while maintaining a unified API contract.

Frequently Asked Questions

Can I build a custom mobile app for Calliope?

Yes. Any mobile application that can send HTTP POST requests to the Calliope server functions as a valid client. Include a unique client_id in the query parameters and optionally set client_type to a custom value for configuration tracking. The API accepts JSON payloads with text and base64-encoded image or audio fields, requiring no specific SDK or library dependencies.

What image formats does the ESP32-Sparrow hardware use?

The ESP32-Sparrow transmits images in the RGB565 format, a 16-bit color representation optimized for embedded displays. When calliope/utils/image.py detects a Sparrow client through the client_type parameter, it decodes this hardware-specific format before passing standardized data to the story generation strategies. Other clients like Clio should send standard JPEG or PNG images.

How does Calliope track story state across different clients?

Calliope uses the client_id parameter to maintain continuity. For hardware devices, it stores state in the SparrowState table (calliope/tables/sparrow_state.py), tracking the active story and last frame. For web and custom clients, configuration and state reside in SparrowConfig (calliope/tables/config.py). Both mechanisms ensure that returning clients resume their stories from the correct narrative position.

Is the Clio web client required to use Calliope?

No. While Clio provides a convenient browser interface documented in docs/Clio.md, Calliope operates as a standalone HTTP API server. You can interact with it exclusively through the REST endpoints defined in docs/api.md using curl, Python scripts, or embedded firmware. The Clio client is simply one officially supported implementation among many possible client applications.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}