# How GenerationBackend Communicates with External AI Services in Palmier Pro

> Discover how GenerationBackend communicates with external AI services in Palmier Pro. Learn how the RPC layer forwards requests to Convex for seamless AI model interaction and real-time updates.

- Repository: [Palmier/palmier-pro](https://github.com/palmier-io/palmier-pro)
- Tags: internals
- Published: 2026-06-23

---

**The GenerationBackend functions as a thin RPC layer that forwards all generation requests to a Convex serverless backend, which handles the actual communication with external AI models and streams real-time updates back to the client via the ConvexMobile SDK.**

The GenerationBackend in Palmier Pro orchestrates AI-powered media generation by acting as an intermediary between the Swift client application and external AI services. Unlike direct API integration, this architecture delegates model inference and third-party API management to a serverless Convex backend, while the Swift client manages authentication, file uploads, and result consumption. Understanding this communication flow reveals how modern macOS applications can leverage serverless architectures to interact with complex AI pipelines.

## Architecture Overview

The **GenerationBackend** does not communicate directly with external AI providers such as OpenAI or Stability AI. Instead, it implements a remote procedure call (RPC) pattern through the **ConvexMobile** SDK, forwarding requests to a **Convex** serverless deployment that hosts the actual AI model integrations. This design separates the client-side orchestration logic from the heavy lifting of model inference and third-party API management.

The communication flow follows five distinct stages: client authentication, asset upload, job submission, real-time status monitoring, and result retrieval. Each stage utilizes specific Convex mutations and subscriptions defined in the [`GenerationBackend.swift`](https://github.com/palmier-io/palmier-pro/blob/main/GenerationBackend.swift) source file.

## Configuration and Client Initialization

Before initiating any AI requests, the backend establishes an authenticated connection to Convex using configuration values from the app bundle. In [`Sources/PalmierPro/Account/BackendConfig.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Account/BackendConfig.swift) (lines 3-7), the `BackendConfig` struct provides the deployment URL and authentication parameters required for initialization.

The `AccountService` creates a `ConvexClientWithAuth` instance using these credentials, which the GenerationBackend then uses for all subsequent RPC calls. This authenticated client handles JWT token management and request signing automatically, ensuring secure communication with the Convex backend without exposing API keys in the client code.

## The Three-Step Upload Workflow

When a generation requires reference media, the `uploadReference` method in [`Sources/PalmierPro/Generation/GenerationBackend.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/GenerationBackend.swift) executes a three-phase upload process that bypasses the Convex functions for the actual data transfer, optimizing for large file sizes.

### Ticket Minting

First, the client requests a pre-signed upload URL by calling the Convex mutation `uploads:generateUploadTicket` (lines 29-33). This mutation returns a temporary, authenticated storage URL that grants direct write access to the underlying storage bucket without exposing credentials to the client.

### Direct Upload via PUT/POST

With the pre-signed URL obtained, the client performs a standard HTTP upload using `URLSession.upload` (lines 35-43). This direct upload to cloud storage removes the bandwidth bottleneck of proxying large media files through the Convex functions, allowing efficient handling of high-resolution images and video frames.

### Commit Phase

After the upload completes successfully, the client notifies Convex via the `uploads:commitUpload` mutation (lines 48-53). This mutation validates the upload and returns a permanent URL for the asset, which the system stores for association with the generation job. The commit phase ensures data integrity before the asset enters the AI processing pipeline.

## Job Submission and Model Routing

Once assets are uploaded, the `submit` method (lines 56-73) initiates the actual AI generation by calling the Convex mutation `generations:submit`. This method constructs a dictionary containing:

- **model**: A string identifier specifying the external AI service (e.g., "gpt-4-vision", "stable-diffusion", "palmier-video-v2")
- **params**: A `BackendGenerationParams` enum value encoding model-specific payload data for video, image, audio, or upscale operations
- **projectId**: An optional identifier linking the generation to a specific project

The Convex backend receives this payload, routes it to the appropriate external AI API, and immediately returns a `jobId` that the client uses to track the asynchronous operation.

## Real-Time Result Subscriptions

After submitting a job, the `subscribe` method (lines 8-18) establishes a real-time subscription to the `generations:byId` table in Convex. This subscription utilizes Convex's reactive query system to push status updates to the client without polling.

The subscription yields `BackendGenerationJob` objects containing status fields (`queued`, `running`, `succeeded`, `failed`). When the job reaches the `succeeded` state, the `resultUrls` array contains the URLs of the generated assets, ready for download. If the job fails, the subscription delivers error details that allow the client to display appropriate messaging.

## Downloading and Finalizing Results

While the GenerationBackend handles the RPC communication, the **GenerationService** manages the final asset retrieval. In [`Sources/PalmierPro/Generation/GenerationService.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/GenerationService.swift) (lines 92-106), the `downloadAndFinalize` method receives the `resultUrls` from the completed job, downloads each asset using standard HTTP requests, performs file extension normalization if required, and imports the media into the local project workspace.

## Implementation Example

The following Swift code demonstrates the complete workflow from uploading a reference image to receiving generation results:

```swift
// 1️⃣ Upload a reference image (handled inside GenerationService)
let uploadedURL = try await GenerationBackend.uploadReference(
    fileURL: localImageURL,
    contentType: "image/jpeg"
)

// 2️⃣ Submit a generation request (e.g., for a video model)
let jobId = try await GenerationBackend.submit(
    model: "palmier-video-v2",
    params: .video(myVideoParams),
    projectId: editor.projectId
)

// 3️⃣ Subscribe to job updates
if let publisher = GenerationBackend.subscribe(jobId: jobId) {
    let cancellable = publisher
        .sink(receiveCompletion: { _ in … },
              receiveValue: { job in
                  guard let job = job else { return }
                  switch job.status {
                  case .succeeded:
                      // Process `job.resultUrls`
                  case .failed:
                      // Show error
                  default: break
                  }
              })
}

```

## Summary

- The **GenerationBackend** operates as an RPC wrapper around the Convex serverless platform, never communicating directly with external AI APIs.
- File uploads use a three-phase ticket system (mint, direct upload, commit) to efficiently handle large media without proxying through Convex functions.
- Job submission relies on the `generations:submit` mutation, which accepts a model identifier string and `BackendGenerationParams` payload.
- Real-time status updates flow through Convex subscriptions to `generations:byId`, delivering job state changes including final result URLs.
- **GenerationService** completes the workflow by downloading assets from the provided URLs and importing them into the project.

## Frequently Asked Questions

### How does the GenerationBackend authenticate with external AI services?

The GenerationBackend does not authenticate directly with external AI services. Instead, it uses the `ConvexClientWithAuth` established by `AccountService` to communicate with the Convex backend. The Convex server holds the API keys and authentication credentials for third-party AI providers, keeping sensitive tokens out of the client application and reducing the attack surface.

### What happens if a file upload fails during the three-step process?

If the direct upload to the pre-signed URL fails, the client can retry the upload without generating a new ticket, provided the URL has not expired. If the upload ticket expires or the commit phase fails, the client must restart the process by requesting a new upload ticket via `uploads:generateUploadTicket`. The commit phase in [`Sources/PalmierPro/Generation/GenerationBackend.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/GenerationBackend.swift) (lines 48-53) only proceeds after successful confirmation of the upload completion.

### Which AI models can the GenerationBackend communicate with?

The GenerationBackend supports any model identifier recognized by the Convex backend deployment. Common identifiers include "gpt-4-vision" for OpenAI's vision models, "stable-diffusion" for image generation, and "palmier-video-v2" for proprietary video generation. The `model` parameter in the `submit` method (lines 56-73) accepts these strings, and the Convex backend routes the request to the appropriate external API based on this identifier.

### How does the real-time subscription handle network interruptions?

The subscription to `generations:byId` uses the ConvexMobile SDK's built-in reconnection logic. If the network connection drops, the SDK automatically attempts to re-establish the WebSocket connection and resume the subscription from the last known state. The client receives any missed updates immediately upon reconnection, ensuring the job status remains synchronized without requiring manual polling or refresh logic in the GenerationBackend code.