How Palmier Pro's Generation Backend Interfaces with External AI Services Like Seedance and Kling

Palmier Pro's generation backend acts as a thin RPC layer that submits typed parameters to a Convex-hosted backend, which routes requests to Seedance, Kling, or other AI providers based on model identifiers, then streams results back via Combine publishers.

The generation backend interface with external AI services in Palmier Pro follows a clean separation of concerns where the Swift client never talks directly to Seedance or Kling. Instead, the application leverages a Convex-hosted backend as a protocol-agnostic bridge, handling everything from model validation and reference media uploads to real-time status streaming. This architecture allows the client to work with uniform Swift types like BackendGenerationParams while the server manages provider-specific API intricacies.

Architecture of the Generation Backend Layer

The generation backend is implemented primarily in Sources/PalmierPro/Generation/GenerationBackend.swift, which provides a type-safe RPC interface to the Convex backend.

The RPC Layer and Type Safety

The GenerationBackend class exposes three core asynchronous methods that encapsulate all network communication:

• uploadReference(fileURL:contentType:) – Handles the three-step upload flow to Convex storage
• submit(model:params:projectId:) – Submits generation jobs to the backend
• subscribe(jobId:) – Returns a Combine publisher that listens for status updates on the generations:byId channel

This design abstracts the underlying HTTP complexity, allowing UI components to work with strongly-typed Swift structures rather than raw JSON payloads.

Model Catalog and Capability Validation

Before any request reaches Seedance or Kling, the client validates parameters against model-specific caps defined in VideoModelConfig.swift and ImageModelConfig.swift. These catalogs specify:

• Maximum reference image counts (e.g., 4 images for Seedance)
• Duration limits and resolution constraints
• Maximum combined video reference seconds (e.g., 20 seconds for certain models)

The VideoCompressor.swift utility enforces size caps (such as Seedance's ~1112px long-side limit) before upload, ensuring the backend receives compliant media.

Submitting Generation Requests to External AI Services

When a user triggers generation from the UI, GenerationService.swift constructs a BackendGenerationParams value that encapsulates all generation settings.

Building the Submission Parameters

The BackendGenerationParams enum wraps either VideoGenerationParams or ImageGenerationParams, containing:

• Model identifier: Strings like "seedance-2-fast" or "kling-v3-motion-control"
• Generation parameters: Prompt, duration, aspect ratio, and resolution
• Reference media URLs: Pre-uploaded Convex storage URLs for images, videos, or audio
• Audio generation flags: Boolean indicating whether to generate audio alongside video

Here is the complete flow for submitting a Seedance video generation:

import PalmierPro

// Step 1: Build generation parameters
let videoParams = VideoGenerationParams(
    prompt: "A sunrise over a misty forest, cinematic lighting",
    duration: 8,
    aspectRatio: "16:9",
    resolution: "720p",
    referenceImageURLs: [],
    referenceVideoURLs: [],
    referenceAudioURLs: [],
    generateAudio: true
)

let backendParams = BackendGenerationParams.video(videoParams)

// Step 2: Submit to Convex backend (lines 56-74 in GenerationBackend.swift)
let jobId = try await GenerationBackend.submit(
    model: "seedance-2-fast",      // Routes to Seedance API
    params: backendParams,
    projectId: currentProject.id
)

// Step 3: Subscribe to status updates
let cancellable = GenerationBackend.subscribe(jobId: jobId)?
    .sink { job in
        guard let job = job else { return }
        switch job.status {
        case .succeeded:
            print("Result URLs: \(job.resultUrls)")
        case .failed:
            print("Generation failed: \(job.errorMessage ?? "Unknown error")")
        default:
            print("Status: \(job.status)")
        }
    }

The Convex backend receives this mutation via the generations:submit function, inspects the model string, and forwards the payload to the appropriate external AI service.

Uploading Reference Media for Kling and Seedance

Both Seedance and Kling support reference images and videos for style transfer or motion control. Palmier Pro handles these through a three-step upload process implemented in GenerationBackend.uploadReference (lines 20-54):

// Upload a local image to use as reference for Kling-v3
let uploadedReferenceURL = try await GenerationBackend.uploadReference(
    fileURL: localImageURL,
    contentType: "image/png"
)

// Use the returned Convex storage URL in generation parameters
let klingParams = VideoGenerationParams(
    prompt: "Character walks through a cyberpunk city",
    duration: 10,
    aspectRatio: "9:16",
    resolution: "1080p",
    referenceImageURLs: [uploadedReferenceURL],
    referenceVideoURLs: [],
    referenceAudioURLs: [],
    generateAudio: false
)

let jobId = try await GenerationBackend.submit(
    model: "kling-v3-elements",
    params: .video(klingParams),
    projectId: projectId
)

The upload process obtains a Convex storage ticket, POSTs the raw bytes to Convex storage, and commits the upload, returning a URL that the AI services can access when processing the job.

Real-Time Status Updates via Combine

The generation backend interface supports reactive programming through Combine publishers. Once a job is submitted, the client subscribes to the generations:byId channel to receive real-time updates as the job progresses through states: queued → running → succeeded or failed.

This subscription mechanism is implemented in GenerationBackend.subscribe (lines 8-18):

let subscription = GenerationBackend.subscribe(jobId: jobId)

subscription?
    .receive(on: DispatchQueue.main)
    .sink(
        receiveCompletion: { completion in
            if case .failure(let error) = completion {
                print("Subscription error: \(error)")
            }
        },
        receiveValue: { job in
            // Update UI with job.status, job.progress, or job.resultUrls
            self.updateGenerationStatus(job)
        }
    )
    .store(in: &cancellables)

The external AI services write generated media to Convex storage upon completion, and the updated resultUrls field flows through this subscription channel to the client.

Backend Routing and Provider Abstraction

The Convex backend functions as a protocol-agnostic router. When generations:submit receives a request, it:

1. Validates the model identifier against the server-side catalog
2. Routes to the specific provider:
   • Seedance: seedance-2-fast, seedance-2-regular
   • Kling: kling-v3-motion-control, kling-v3-elements
   • Other providers: Grok, etc.
3. Transforms the standardized BackendGenerationParams into provider-specific API calls
4. Polls the external service for completion
5. Writes result files to Convex storage
6. Notifies clients via the subscription channel

This abstraction means UI code in GenerationView.swift remains agnostic to whether the user selected Seedance or Kling—the same Swift types and methods work uniformly across all providers.

Summary

• Palmier Pro's generation backend acts as a thin RPC layer in GenerationBackend.swift, interfacing with Convex rather than directly with AI providers.
• Model identifiers like "seedance-2-fast" and "kling-v3-motion-control" determine routing, while VideoModelConfig.swift enforces client-side validation of provider limits.
• Reference media undergoes a three-step upload process to Convex storage before being passed to external services via URL.
• Real-time updates stream through Combine publishers listening to the generations:byId channel, enabling reactive UI updates without polling.
• Backend abstraction allows the client to use uniform Swift types (BackendGenerationParams) regardless of whether Seedance, Kling, or other AI services process the generation.

Frequently Asked Questions

How does Palmier Pro handle authentication with Seedance and Kling?

Palmier Pro never stores or manages API keys for external AI services. Authentication is handled entirely by the Convex backend, which securely stores provider credentials and manages token refresh cycles. The Swift client only authenticates with Convex using its existing session, and the backend uses its own credentials when forwarding requests to Seedance or Kling.

What happens if an uploaded reference image exceeds the AI service's size limits?

The VideoCompressor.swift utility preprocesses reference media before upload, enforcing model-specific constraints such as Seedance's ~1112px long-side limit. If compression cannot reduce the file to acceptable parameters, the upload fails early with a validation error before reaching the external service, preventing wasted API calls and incomplete jobs.

Can I switch between Seedance and Kling after starting a generation job?

No, once a job is submitted via GenerationBackend.submit(model:params:projectId:), the model identifier is immutable. The Convex backend uses this identifier to route to the specific provider API immediately upon receiving the mutation. To use a different provider, you must cancel the existing job (if still queued) and submit a new request with the desired model identifier, such as "kling-v3-elements" instead of "seedance-2-fast".

How does the generation backend handle failed requests from external AI services?

When Seedance or Kling returns an error, the Convex backend writes a failed status to the job record along with an error message in the errorMessage field. This update flows through the generations:byId subscription channel to the client, where the Combine publisher emits the updated job state. The UI can then display the failure reason and allow the user to retry with modified parameters if appropriate.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}