Stream text

Use streamText when the UI or caller should receive partial output as it is generated.

Providers may finish with interrupted when the stream ends without its completion signal. Partial output remains available. The facade skips local tool execution and automatic continuation for that response; the caller decides whether to retry. Explicit provider/parse errors retain their error semantics.

import SwiftAISDK

@main
struct StreamTextExample {
    static func main() async throws {
        let provider = try AIProviders.openAI()
        let model = try provider.languageModel("gpt-4.1-mini")

        for try await part in model.streamText("Write a short haiku about APIs.") {
            switch part {
            case .textDelta(let text):
                print(text, terminator: "")
            case .finish(let reason, let usage):
                print("\nFinished: \(reason ?? "unknown")")
                print("Tokens: \(usage?.totalTokens ?? 0)")
            default:
                break
            }
        }
    }
}

Streaming emits structured lifecycle parts, not just raw strings. That lets you render text, reasoning, tool input deltas, tool calls, tool approval requests and responses, tool results, source citations, files, raw chunks, response metadata, and finish metadata with one loop.

Built-in HTTP language providers parse the response incrementally, so the first complete provider event can become a LanguageStreamPart while the HTTP response is still open. A provider end marker such as [DONE], caller abort, or stopping iteration cancels the remaining response-body read instead of waiting for EOF.

URLSessionTransport converts response bytes to Data only when the body consumer requests the next element. An unread or paused body no longer creates an unbounded queue of one-byte Data values. Abort signals, explicit cancelBody(), task cancellation during a read, and dropping the body release the underlying URLSession task. Response byte limits still apply.

This bounds the SDK’s byte-adapter queue only. Foundation can buffer network data, and higher-level parsers, tool loops, telemetry and application code can retain content. The SDK does not claim a whole-stream constant-memory bound. The offline macOS probe python3 Scripts/measure-http-streaming-memory.py measures an unread 1 MiB response in fresh processes; compare measurements on the same machine and toolchain.

Canonical Stream Contract

Built-in language models emit one semantic family for generated content:

textStart, zero or more textDeltaPart values, then textEnd or textEndWithCitations;
reasoningStart, zero or more reasoningDeltaPart values, then reasoningEnd.

The legacy textDelta, reasoningDelta, and finish enum cases remain available for source compatibility with custom models. Built-in providers never pair a legacy delta with a part-aware delta for the same provider token. The high-level facade normalizes legacy deltas into the canonical lifecycle and legacy finish into finishMetadata; a custom stream that mixes both delta families in one semantic channel is rejected because equal adjacent strings can be legitimate provider output and cannot be safely deduplicated by value.

Each logical built-in model response emits exactly one terminal finishMetadata part after its open content and tool-input blocks have been closed and its final usage and provider metadata are known. A protocol that can carry multiple logical responses on one connection, such as OpenAI Responses, emits one terminal part for each response. A multi-step tool loop likewise has one terminal part per model step. Custom models should emit the same terminal; clean custom EOF without one is preserved rather than assigned a guessed finish reason.

The full stream forwards repeatable provider error parts as in-band events. Setup, HTTP, framing, and network failures terminate by throwing. If an in-band error closes a provider stream without a terminal event, the facade completes that step with a single finishMetadata whose reason is error. By default an attempt is not replayed once it has exposed a public part. Callers can opt into the narrower post-start recovery described below.

Use toTextStream() when only text is needed. It emits canonical text deltas, ignores lifecycle, reasoning, tool, metadata, and in-band error parts, and still propagates thrown failures.

Citations attached to text

AICitation identifies the URL or document supporting a text block, with optional startIndex, exclusive endIndex, and citedText. OpenAI and Anthropic populate citations independently of the complete retrieved sources list. Ranges are provider offsets into the containing text; preserve their original coordinate system when applying text transformations.

TextGenerationResult.citations flattens the attached citations. Content uses .citedText when citations are present, while uncited output retains .text. Streams close cited blocks with .textEndWithCitations; its citations property also exposes the references without matching the case. UI snapshots retain them in AIUITextPart.citations. Existing .text and .textEnd constructors remain available. Exhaustive switches should handle the companion cases as well.

The JavaScript experimental_streamTextLowMemory result facade and its replay/final-getter retention policy are not exposed by this native API. Swift returns async sequences without those JavaScript result getters; no equivalent memory bound or retention-performance guarantee is claimed.

Custom Transports

Provider settings continue to accept any AITransport, which keeps unary generate calls source-compatible with send-only transports. A custom transport used by stream must also conform to AIStreamingTransport and return response headers plus an incremental AsyncThrowingStream<Data, Error> body. SwiftAISDK reports a non-retryable AIError.invalidArgument when streaming is requested with a send-only transport; it never falls back to buffering through send.

Prodia language generation and the default protocol implementation remain intentionally simulated streams because their underlying operation is unary. Provider-backed SSE and Amazon EventStream language models use the incremental transport path.

Tool input can arrive before the final tool call. Use toolInputStart / toolInputStartText, toolInputDelta, and toolInputEnd parts when you want to show streamed arguments in a UI, and handle toolApprovalRequest / toolApprovalResponse parts when a tool requires human or policy approval.

Retry Behavior

Facade calls retry transient setup failures according to retryPolicy. The default post-start behavior is still conservative: after the first public part, a stream failure ends that attempt.

Set streamRetries to a positive count to recover from retryable in-band provider error events after streaming has started. The failed attempt’s tool input, tool calls/results, finish, usage, and provider metadata do not become step state for the replacement attempt. When recovery succeeds, the provider error event and failed terminal state are suppressed. Text and reasoning already delivered to the caller cannot be retracted, so a replacement attempt can repeat a prefix; consumers that enable this option should reconcile visible text accordingly.

Thrown transport/framing failures after public output, non-retryable provider errors, clean empty streams, and output-schema failures do not consume this post-start retry budget. retryPolicy continues to govern setup attempts and backoff independently.

for try await part in AI.streamText(
    model: model,
    prompt: "Stream this carefully.",
    retryPolicy: .none,
    streamRetries: 1
) {
    // Render parts.
}

Semantic Stream Timeouts

Use AIStreamTimeoutConfiguration when a streaming caller needs to distinguish the complete-operation deadline, each model-call step, a stalled first output, or a stall between output parts:

for try await part in AI.streamText(
    model: model,
    prompt: "Stream this carefully.",
    timeout: AIStreamTimeoutConfiguration(
        totalNanoseconds: 60_000_000_000,
        stepNanoseconds: 30_000_000_000,
        firstChunkNanoseconds: 10_000_000_000,
        chunkNanoseconds: 15_000_000_000
    )
) {
    // Render parts.
}

The step/first/inter-chunk timers restart for every model-call step. Only semantic output disarms or resets them: non-empty text, reasoning, or tool-input deltas, files, reasoning files, and complete tool calls. Metadata, raw keep-alives, lifecycle markers, tool results, and empty deltas do not extend the deadline. A terminal model finish/error disarms first/inter-chunk timers before output postprocessing or tool work. Every replacement provider attempt starts a fresh first-output timer. Explicit aborts preserve their caller reason and reason name. First/inter-chunk failures throw AIStreamTimeoutError with .firstChunk or .chunk; step and total failures use AIError.timeout. The legacy flat timeoutNanoseconds remains available as the total deadline.

Total and step budgets include time spent in retry backoff. A step deadline stays active after the model stream while client-side tools execute; when it fires, both the provider request and tool context see an aborted signal whose reason name is TimeoutError. Typed AI.streamText(..., output: Output.*, timeout: ...) overloads use the same timeout lifecycle.

Files and UI snapshots

Streamed file and reasoning-file parts use the same materialization policy as generateText: data: and HTTP(S) URLs become inline data before the part is emitted, while provider references stay opaque. Downloads honor aborts, redirect validation, private-host rejection, and byte limits. Resolver-backed DNS address pinning is still a transport-level gap.

AIUIMessageStreamReducer.snapshots can continue an existing originalMessage instead of starting a new assistant message. Its async onEnd callback fires at most once with the final message and an outcome of completed, aborted, failed, or unknown. Consumer cancellation reports unknown with isCancelled == true; an AIAbortError reports aborted with isAborted == true.