google.generative-ai
Package
Section titled “Package”@ai-sdk/google
Factories
Section titled “Factories”AIProviders.google
Capabilities
Section titled “Capabilities”- Language
- Embedding
- Image
- Speech
- Video
- Evaluation
- Decision
- Files
Structured JSON with application tools
Section titled “Structured JSON with application tools”A request with .json(schema: ...) and function tools can return JSON while retaining real application tool calls. Pre-Gemini 3 and forced-tool cases use a collision-safe synthetic response tool. Its arguments become JSON text/stream deltas; its internal tool call and unrelated prose are omitted. Gemini 3 automatic selection uses native JSON output. Reasoning, signatures and application tool calls remain available.
Tool failures and denials use functionResponse.error. Generic embedding dimensions apply unless provider-specific output dimensions override them. nano-banana-2.1 uses the generateContent image route; retrieved-context custom metadata stays in provider metadata.
decisionModel(...) uses DecisionLanguageModel; see Decision. Batch downloads use the configurable per-row limit.
Gemini image requests merge the typed aspect ratio with provider imageConfig; configuring output settings does not discard the caller’s ratio.
Gemini 4 long decode
Section titled “Gemini 4 long decode”GenerateContent automatically follows CONTINUATION responses for gemini-4-argon and other compatible model IDs. Each hop resends the original request with the returned token, keeps maxOutputTokens cumulative, and appends newly generated content. Streams retain one text lifecycle across hops. Usage totals and modality details sum all requests to match billing; providerMetadata["google"]["initialPromptTokenCount"] retains the first prompt’s count for context-window checks.
After automatic continuation, providerMetadata["google"]["groundingMetadataPerRequest"] contains one grounding payload per successful request, in order, with null where that request returned none. This preserves earlier search evidence even when the final response has no grounding. The existing groundingMetadata field contains the final response’s payload for generation and the latest non-null payload for streaming. Single-request calls, including manual resume calls that do not continue automatically, omit the per-request array.
A response ending with MAX_TOKENS exposes providerMetadata["google"]["continuationToken"]. Resume with the original prompt, a larger maxOutputTokens, and that token in providerOptions["google"]["continuationToken"]. Failed streaming continuation emits an error followed by finish metadata containing the resumable token. Cancellation propagates through every hop.
responseModalities accepts AUDIO alongside TEXT and IMAGE. Explicit thinkingConfig.thinkingBudget or thinkingConfig.thinkingLevel takes precedence over the shared reasoning setting. Gemini 3+ Pro models normalize none and minimal reasoning to their supported low floor. Routed IDs such as eu.gemini-2.5-flash retain the correct model capabilities. Interactions model calls with background: true poll until terminal, including Gemini 4 long decode.
Continuation and streamed tool arguments
Section titled “Continuation and streamed tool arguments”Each successful automatic continuation starts with a fresh completion state; an empty next response remains interrupted instead of inheriting the previous finish reason. A failed continuation retains the preceding response’s state. Partial function arguments, including response tools, preserve nested empty arrays and objects and produce valid concatenated JSON deltas. Legacy tool-call snapshot parts remain available alongside the incremental tool-input lifecycle.
Google Generative AI’s implemented Gemini language, embedding, image, TTS speech, Veo video, files, interactions, and agents surface was audited against upstream Google 4.0.105. The current pass adds the published 100-input embedding preflight limit. Imagen factories remain locally available for source compatibility after their removal from the current upstream package. Gemini 3.7 Flash normalizes none and minimal reasoning to the supported low thinking floor. Primitive JSON Schema enum/const values and local $ref expansion, including boolean-schema references, map to Gemini’s OpenAPI-compatible wire form, while strict forced/named tools use ANY and strict automatic tools retain VALIDATED. Interactions forwards topK and seed while warning for unsupported penalties and agent-dropped settings; Files resumable uploads leave payload length to the transport. Generate and stream results surface Gemini responseId; multiple code-execution results retain their originating call ID; parallel unsigned function calls reuse an earlier signed standard call; and unknown future Gemini IDs default to the newest supported tool, thinking, file-search, and multimodal response capabilities while known legacy families stay conservative. Speech maps Gemini TTS generateContent requests with WAV-wrapped PCM by default, raw PCM opt-in, multi-speaker provider options, response sample-rate metadata, and upstream warnings. Gemini 3.5 unary/live transcription, Files URL recognition, recursive-tool behavior, and Gemini 2.5 penalty warnings remain deferred to a dedicated provider pass. Realtime/WebSocket speech translation and lifecycle events and JS ProviderV4 runtime helpers also remain gaps. The 4.0.64 pass adds JSON Schema minItems and maxItems, preserves caller function aliases, forwards file headers and cancellation, supplies a durable Batch V4 adapter with tool content and input-file metadata, and supports Interactions video plus processing-call/result request, response, stream, warning, and metadata handling. Final integration also round-trips Interactions response history, function and built-in tool items, reasoning items, compaction state, provider references, mediaResolution, and systemInstruction precedence; Google Batch rejects empty result keys and preserves thought signatures even when their associated text is empty. The 2026-09-20 pass adds lossless schema conversion, corrected block reasons, accumulated stream metadata and usage, exact image-count behavior, custom provider names, and a Gemini-backed Evaluation V4 adapter. The 2026-09-27 pass adds embedding provider-option slicing, image metadata, video offsets, code-execution replay, JSON Schema $ref URIs, and current TTS behavior. The 2026-10-04 pass reports Gemini image file/mask capabilities and retains speech usageMetadata. Gemini transcription and Google Realtime remain deferred. The 2026-10-11 pass adds collision-safe JSON response tools with application tools, incremental nested JSON parsing and real-tool/signature preservation, Gemini 3 native automatic selection, shared embedding dimensions and Decision V4. Batch downloads default to a configurable 64 MiB UTF-8 line limit. The 2026-10-18 pass: GenerateContent auto-continues long decodes with cumulative output budgets, summed per-hop usage and initialPromptTokenCount metadata. Checkpoint tokens support explicit resume. AUDIO output, explicit thinking precedence, dotted route IDs and background Interactions polling are supported.
Factory argument requirements are defined by the public Swift factory signatures. Use Public symbols when you need the exact initializer or factory declaration.
Return to the provider matrix.