heygen

Package

@ai-sdk/heygen

Factories

AIProviders.heyGen

Capabilities

• Video

Generate a video

The HeyGen adapter follows @ai-sdk/heygen@3.0.0. Set HEYGEN_API_KEY or pass ProviderSettings(apiKey:); baseURL, custom headers and transport support proxies and testing. Swift resolves credentials when AIProviders.heyGen() is created. The JavaScript factory resolves them lazily at request time.

import SwiftAISDK

let heyGen = try AIProviders.heyGen()
let model = try heyGen.video("heygen-video-1")
let video = try await AI.generateVideo(
    model: model,
    request: VideoGenerationRequest(
        prompt: "A paper boat drifting through a rain-soaked city at dusk.",
        aspectRatio: "16:9",
        durationSeconds: 5,
        providerOptions: HeyGenVideoModelOptions(
            resolution: .p768,
            promptEnhancement: .turbo
        ).providerOptions
    ),
    poll: VideoGenerationPollOptions(
        intervalMilliseconds: 5_000,
        timeoutMilliseconds: 600_000
    )
)
print(video.urls.first ?? "No video URL returned")

video(...) and videoModel(...) are aliases. Future model IDs are forwarded unchanged. HeyGen uses asynchronous start/status operations, which AI.generateVideo selects automatically; its unary model call is unsupported. Each operation generates one MP4 with audio at 24 fps. count above one on the facade starts separate operations and merges results in order. AI.startVideo accepts one video per operation. Direct model starts warn for a count above one, non-24 fps, or generateAudio: false and omit those controls from the request.

First-frame image and reference video

Use either image or one .firstFrame in frameImages for image-to-video. URL inputs must use HTTPS. Local Data is encoded once as base64; it needs an explicit image/* media type. HeyGen fetches asset URLs itself.

let firstFrame = VideoGenerationRequest(
    prompt: "Slowly pan across this landscape.",
    frameImages: [VideoFrameImage(
        image: ImageInputFile(url: "https://example.com/landscape.png", mediaType: "image/png"),
        frameType: .firstFrame
    )]
)

let references = VideoGenerationRequest(
    prompt: "Match the subject and camera movement in these references.",
    inputReferences: [
        ImageInputFile(url: "https://example.com/subject.png", mediaType: "image/png"),
        ImageInputFile(url: "https://example.com/motion.mp4", mediaType: "video/mp4"),
        ImageInputFile(url: "https://example.com/sound.mp3", mediaType: "audio/mpeg")
    ],
    providerOptions: HeyGenVideoModelOptions(
        referenceImages: [.assetID("your-workspace-image-asset-id")]
    ).providerOptions
)

Reference inputs need an explicit image/*, video/*, or audio/* media type; they are grouped by kind, retaining order within each group. Provider-option references append to the corresponding standard-input group. Typed assets support .url(...), .assetID(...), and .base64(data:mediaType:) under image, referenceImages, referenceVideos, and referenceAudio. Existing base64 text is forwarded unchanged. Raw JSON options use type: "asset_id" with assetId, or type: "base64" with mediaType and data.

The mode is inferred from the inputs, or specified with .textToVideo, .imageToVideo, or .referenceToVideo. The SDK rejects conflicting modes, image plus references, standard plus provider-option first images, last frames, and multiple first frames before I/O. Reference-to-video requires at least one image or video; audio-only conditioning is invalid. Limits are 9 images, 3 videos, 3 audio references, and 12 references total.

Output controls

Prompts require 1–32,000 UTF-16 code units. Duration is an integer from 5–15 seconds and defaults to 5. Seeds range from 0 to 4,294,967,295. promptEnhancement accepts .turbo, .quality, or .disabled; omission lets HeyGen apply its API default.

Typed resolution selects .p480, .p768, .p1080, or .twoK and overrides the standard resolution frame size. The default is 768p. These standard frame sizes map to a size class and aspect ratio:

Class	Frame sizes
480p	960x416, 832x480, 640x480, 480x480, 480x640, 480x832
768p	1536x672, 1344x768, 1024x768, 768x768, 768x1024, 768x1344
1080p	1890x1080, 1080x1890
2k	2688x1536, 1536x2688

Text-to-video defaults to 16:9; reference-to-video defaults to adaptive. Supported explicit ratios are 21:9, 16:9, 4:3, 1:1, 3:4, and 9:16. Text/reference generation at 1080p or 2k requires 16:9 or 9:16; select a ratio explicitly when using the reference default. Image-to-video inherits the first frame ratio, omits aspect_ratio from the API request, and warns when a standard ratio or mapped frame size was supplied. Its resolution still applies.

Persist and resume an operation

import Foundation

let started = try await AI.startVideo(model: model, request: firstFrame)
let saved = try JSONEncoder().encode(started.operation)
// Persist saved with application storage; it contains videoId, mode and resolution.
let restored = try JSONDecoder().decode(JSONValue.self, from: saved)
let status = try await AI.getVideoStatus(model: model, operation: restored)
switch status {
case .pending: print("Check again later")
case let .failed(message, _, _): print(message)
case let .completed(result): print(result.urls)
}

Resume with a newly created HeyGen model using the same model ID and workspace credentials. Status maps pending/processing to .pending and failed/cancelled to .failed with HeyGen’s message and failure code. Unexpected states or completion without a URL fail explicitly. HTTP errors preserve status, response headers/body, retryability, and structured code/param details. Start retries retain one idempotency key; a caller-supplied header wins.

Completion returns the signed MP4 URL without downloading the file. Check the operation again to refresh an expired URL. Reported duration, dimensions, aspect ratio, seed, inference timing and failure code are preserved under providerMetadata["heygen"]?["videos"]?[0] for completed videos; start/pending/ failure metadata uses providerMetadata["heygen"]. Duration is the actual reported value, which can be fractional. The adapter reports no invented token or credit usage. Credentials are removed from status redirects to other origins.

HeyGen’s upstream package has no remote cancellation or video webhook adapter. Stopping polling aborts local waiting and leaves the operation resumable; it does not cancel the remote job. Supplying a webhook factory warns and falls back to polling without invoking it. Avatar, voice, translation APIs and JavaScript workflow/model serialization are outside this video provider.

The compile-checked example is Examples/Sources/GenerateWithHeyGen/main.swift. Run swift run --package-path Examples GenerateWithHeyGen for usage without API calls. The text, image, reference, start, and resume commands call HeyGen.

Notes

HeyGen mirrors @ai-sdk/heygen 3.0.10: HEYGEN_API_KEY / x-api-key authentication, custom base URL/headers/transport and ai-sdk-heygen/3.0.10 user-agent; video/videoModel aliases and open model IDs; heygen-video-1 text-to-video, first-frame image-to-video and grouped image/video/audio reference conditioning. Typed HeyGenVideoModelOptions and HeyGenVideoAsset validate HTTPS URL/asset/base64 inputs, mode conflicts, 9 image/3 video/3 audio/12 total reference limits, prompt/duration/seed bounds, all documented frame sizes and size-class/aspect-ratio constraints before I/O. AsyncVideoModel start/status operations retain videoId/mode/resolution, support durable JSON persistence and shared polling/idempotent retries, and return refreshed signed MP4 URLs without downloading. Reported output duration/dimensions/seed/inference timing and failure codes are preserved without inventing token or credit usage. Native credentials resolve at factory creation. Unary generation, remote cancellation, webhooks, avatar/voice/translation APIs and JavaScript model/workflow serialization are absent from this provider surface; unsupported fps/audio/count settings warn.

Factory argument requirements are defined by the public Swift factory signatures. Use Public symbols when you need the exact initializer or factory declaration.

Return to the provider matrix.