Durable batches

Use Batch V4 for durable, asynchronous generation. The batch belongs to the provider rather than one model: every request supplies its model ID and its text or image modality. The returned AIBatchReference is persistable, so another process can resume status and result reads without keeping the start request alive.

Anthropic Messages Batch, OpenAI Responses Batch, xAI Responses Batch, Google Generative AI Batch, and Gateway Batch implement the shared provider surface:

let anthropic = try AIProviders.anthropic()
let batch = anthropic.experimentalBatch()

let started = try await AI.startBatch(
    provider: batch,
    requests: [
        .text(TextBatchRequest(
            id: "summary-1",
            modelID: "claude-sonnet-4-5",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        ))
    ]
)

// Persist started.batch.reference if another process will resume the batch.
let status = try await AI.getBatchStatus(
    provider: batch,
    batch: started.batch.reference
)

The provider factory determines the wire adapter:

let openAI = try AIProviders.openAI()
let openAIBatch = try openAI.experimentalBatch()

let xAI = try AIProviders.xAI()
let xAIBatch = try xAI.experimentalBatch()

let google = try AIProviders.google()
let googleBatch = google.experimentalBatch()

let gateway = try AIProviders.gateway()
let gatewayBatch = gateway.experimentalBatch()

Each provider validates its supported request set before network I/O:

• Anthropic: text, with independent model IDs per request.
• OpenAI: text, with one common model per batch.
• xAI: text and image, including mixed models and modalities.
• Google Generative AI: text and image, with one common model endpoint.
• Gateway: text, with one common model per batch.

Use provider.supportedURLs with isURLSupported(...) to decide whether a remote input can be forwarded directly. Convert unsupported remote inputs to the request type’s inline data form before starting the batch.

Mixed text and image requests

xAI accepts a mixed provider-owned batch:

let started = try await AI.startBatch(
    provider: xAIBatch,
    requests: [
        .text(TextBatchRequest(
            id: "summary-1",
            modelID: "grok-4",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )),
        .image(ImageBatchRequest(
            id: "cover-1",
            modelID: "grok-imagine-image",
            request: ImageGenerationRequest(prompt: "A minimal release cover")
        ))
    ]
)

Google also supports text and image requests, but all items in one batch must select the same model. It sends small payloads inline and uses the resumable file-upload flow once the encoded body reaches the provider limit.

Lifecycle

Starting is intentionally not retried because it may create billable work. Pass a stable idempotencyKey when the provider or gateway honors one. Status, listing, and result setup use the normal retry policy:

let status = try await AI.getBatchStatus(
    provider: batch,
    batch: started.batch.reference
)

let page = try await AI.listBatches(
    provider: batch,
    limit: 20
)

_ = try await AI.cancelBatch(
    provider: batch,
    batch: started.batch.reference
)

Pass webhookURL to request a completion callback. Gateway and Google forward the URL through their native callback fields; direct Anthropic, OpenAI, and xAI adapters return an unsupported warning so the caller can fall back to status polling without silently assuming webhook delivery.

Results

getBatchResults is an async stream of independent terminal items. One failed, cancelled, or expired request does not terminate results for the other IDs:

let results = try AI.getBatchResults(
    provider: batch,
    batch: started.batch.reference
)

for try await item in results {
    switch item {
    case let .text(.succeeded(id, generation)):
        print(id, generation.text, generation.content)
    case let .image(.succeeded(id, generation)):
        print(id, generation.urls, generation.base64Images.count)
    case let .text(.failed(id, error, _)),
         let .image(.failed(id, error, _)):
        print(id, error.message)
    default:
        print(item)
    }
}

Stopping result consumption cancels the active provider stream. Batch references are validated against provider identity before status or result I/O, preventing accidental cross-provider reads. Provider-native request counts are normalized before they reach the shared status type. Start and terminal results retain provider metadata such as uploaded file identifiers, expiry timestamps, and raw finish reasons. A failed, cancelled, or expired item does not terminate the other results.

Result content is authoritative and can contain text, reasoning, files, sources, tool calls, and tool results. The text property remains a convenience projection. Google merges empty-text thought signatures onto the associated reasoning or tool part so they survive later model-message replay.

Successful text items also materialize URL-backed generated files and reasoning files before they are emitted. The same private-host, redirect, size, and abort protections used by generateText apply; provider file references remain opaque. Resolver-backed DNS address pinning remains deferred at the transport layer.

Legacy text-only API

The model-owned API remains source compatible for existing callers:

let model = try anthropic.messages("claude-sonnet-4-5")
let started = try await AI.startTextBatch(
    model: model,
    requests: [TextBatchRequest(
        id: "summary-1",
        request: LanguageModelRequest(messages: [.user("Summarize this.")])
    )]
)

This compatibility surface remains text only. New integrations should prefer experimentalBatch() plus AI.startBatch(provider:requests:) so model identity and modality live on each request and the provider owns the full lifecycle.

Bounded result downloads

Anthropic, OpenAI Responses, Google and Gateway batch JSONL readers enforce a UTF-8 byte limit per line before accumulating a large result. The default is 64 MiB; set a positive ProviderSettings.batchResultDownloads limit to choose a smaller or larger bound. Invalid limits fail before the download starts, and termination or oversize lines cancel the response body.

var settings = ProviderSettings()
settings.batchResultDownloads = AIBatchResultDownloadSettings(maxLineBytes: 8 * 1024 * 1024)
let provider = try AIProviders.openAI(settings: settings)