---
title: "OpenAI Generative AI with Weaviate"
description: "OpenAI Generative Model Provider"
url: https://docs.weaviate.io/weaviate/model-providers/openai/generative
---

# OpenAI Generative AI with Weaviate

> [!NOTE]
>
> **Looking for Azure OpenAI integration docs?**
>
> For Azure OpenAI integration docs, see [this page instead](https://docs.weaviate.io/weaviate/model-providers/openai-azure/generative).

Weaviate's integration with OpenAI's APIs allows you to access their models' capabilities directly from Weaviate.

[Configure a Weaviate collection](https://docs.weaviate.io/weaviate/model-providers/openai/generative#configure-collection) to use a generative AI model with OpenAI. Weaviate will perform retrieval augmented generation (RAG) using the specified model and your OpenAI API key.

More specifically, Weaviate will perform a search, retrieve the most relevant objects, and then pass them to the OpenAI generative model to generate outputs.

![RAG integration illustration](https://docs.weaviate.io/assets/images/integration_openai_rag-108560294234d45d8df87b22bddd93fe.png)

## Requirements

### Weaviate configuration

Your Weaviate instance must be configured with the OpenAI generative AI integration (`generative-openai`) module.

**For Weaviate Cloud (WCD) users**

This integration is enabled by default on Weaviate Cloud (WCD) instances.

**For self-hosted users**

- Check the [cluster metadata](https://docs.weaviate.io/deploy/configuration/status#cluster-metadata) to verify if the module is enabled.
- Follow the [how-to configure modules](https://docs.weaviate.io/weaviate/configuration/modules) guide to enable the module in Weaviate.

### API credentials

You must provide a valid OpenAI API key to Weaviate for this integration. Go to [OpenAI](https://openai.com/) to sign up and obtain an API key.

Provide the API key to Weaviate using one of the following methods:

- Set the `OPENAI_APIKEY` environment variable that is available to Weaviate.
- Provide the API key at runtime, as shown in the examples below.

**Python**

```py
import weaviate
from weaviate.classes.init import Auth
import os

# Recommended: save sensitive data as environment variables
openai_key = os.getenv("OPENAI_API_KEY")
headers = {
    "X-OpenAI-Api-Key": openai_key,
}

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=weaviate_url,                       # `weaviate_url`: your Weaviate URL
    auth_credentials=Auth.api_key(weaviate_key),      # `weaviate_key`: your Weaviate API key
    headers=headers
)

# Work with Weaviate

client.close()
```

**JavaScript/TypeScript**

```ts
import weaviate from '@weaviate/node'

const openaiApiKey = process.env.OPENAI_API_KEY || '';  // Replace with your inference API key

const client = await weaviate.connectToWeaviateCloud(
  'WEAVIATE_INSTANCE_URL',  // Replace with your instance URL
  {
    authCredentials: new weaviate.ApiKey('WEAVIATE_INSTANCE_API_KEY'),
    headers: {
      'X-OpenAI-Api-Key': openaiApiKey,
    }
  }
)

// Work with Weaviate

client.close()
```

## Configure collection

> [!NOTE]
>
> **Generative model integration mutability**
>
> A collection's `generative` model integration configuration is mutable from `v1.25.23`, `v1.26.8` and `v1.27.1`. See [this section](https://docs.weaviate.io/weaviate/manage-collections/generative-reranker-models#update-the-generative-model-integration) for details on how to update the collection configuration.

[Configure a Weaviate index](https://docs.weaviate.io/weaviate/manage-collections/generative-reranker-models#specify-a-generative-model-integration) as follows to use an OpenAI generative AI model:

**Python**

```py
from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    generative_config=Configure.Generative.openai()
    # Additional parameters not shown
)
```

**JavaScript/TypeScript**

```ts
await client.collections.create({
  name: 'DemoCollection',
  generative: weaviate.configure.generative.openAI(),
  // Additional parameters not shown
});
```

### Select a model

You can specify one of the [available models](https://docs.weaviate.io/weaviate/model-providers/openai/generative#available-models) for Weaviate to use, as shown in the following configuration example:

**Python**

```py
from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    generative_config=Configure.Generative.openai(
        model="gpt-4-1106-preview"
    )
    # Additional parameters not shown
)
```

**JavaScript/TypeScript**

```ts
await client.collections.create({
  name: 'DemoCollection',
  generative: weaviate.configure.generative.openAI({
    model: 'gpt-4-1106-preview',
  }),
  // Additional parameters not shown
});
```

You can [specify](https://docs.weaviate.io/weaviate/model-providers/openai/generative#generative-parameters) one of the [available models](https://docs.weaviate.io/weaviate/model-providers/openai/generative#available-models) for Weaviate to use. The [default model](https://docs.weaviate.io/weaviate/model-providers/openai/generative#available-models) is used if no model is specified.

### Generative parameters

Configure the following generative parameters to customize the model behavior.

**Python**

```py
from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    generative_config=Configure.Generative.openai(
        # # These parameters are optional
        # model="gpt-4",
        # frequency_penalty=0,
        # max_tokens=500,
        # presence_penalty=0,
        # temperature=0.7,
        # top_p=0.7,
        # base_url="<custom_openai_url>",
        # # For reasoning models such as the gpt-5 family:
        # reasoning_effort="medium",  # One of "minimal", "low", "medium", "high"
        # verbosity="medium",  # One of "low", "medium", "high"
    )
    # Additional parameters not shown
)
```

**JavaScript/TypeScript**

```ts
await client.collections.create({
  name: 'DemoCollection',
  generative: weaviate.configure.generative.openAI({
    // These parameters are optional
    // model: 'gpt-4',
    // frequencyPenalty: 0,
    // maxTokens: 500,
    // presencePenalty: 0,
    // temperature: 0.7,
    // topP: 0.7,
  }),
  // Additional parameters not shown
});
```

Two additional parameters are available for reasoning models such as the `gpt-5` family. They were added in `v1.33.0`, and backported to `v1.31.15` and `v1.32.9`:

- `reasoningEffort`: How much reasoning the model does before it answers. One of `minimal`, `low`, `medium`, or `high`. If not set, the model provider default applies.
- `verbosity`: How detailed the generated answer is. One of `low`, `medium`, or `high`. If not set, the model provider default applies.

The Python client exposes these as the `reasoning_effort` and `verbosity` arguments, both when you configure the collection and when you [select a model at runtime](https://docs.weaviate.io/weaviate/model-providers/openai/generative#select-a-model-at-runtime). The TypeScript client does not expose them yet, so set them with another client or through the REST collection configuration API.

For further details on model parameters, see the [OpenAI API documentation](https://platform.openai.com/docs/api-reference/chat).

## Select a model at runtime

Aside from setting the default model provider when creating the collection, you can also override it at query time.

**Python**

```py
from weaviate.classes.config import Configure
from weaviate.classes.generate import GenerativeConfig

collection = client.collections.use("DemoCollection")
response = collection.generate.near_text(
    query="A holiday film",
    limit=2,
    grouped_task="Write a tweet promoting these two movies",
    generative_provider=GenerativeConfig.openai(
        # # These parameters are optional
        # model="gpt-4",
        # frequency_penalty=0,
        # max_tokens=500,
        # presence_penalty=0,
        # temperature=0.7,
        # top_p=0.7,
        # base_url="<custom_openai_url>"
    ),
    # Additional parameters not shown
)
```

**JavaScript/TypeScript**

```ts
import { generativeParameters } from '@weaviate/node';

let response;
const myCollection = client.collections.use("DemoCollection");

response = await myCollection.generate.nearText("A holiday film", {
  groupedTask: "Write a tweet promoting these two movies",
  config: generativeParameters.openAI({
    // These parameters are optional
    // model: 'gpt-4',
    // frequencyPenalty: 0,
    // maxTokens: 500,
    // presencePenalty: 0,
    // temperature: 0.7,
    // topP: 0.7,
    // baseURL: "<custom-openai-url>",
  }),
}, {
  limit: 2,
}
  // Additional parameters not shown
);
```

## Header parameters

You can provide the API key as well as some optional parameters at runtime through additional headers in the request. The following headers are available:

- `X-OpenAI-Api-Key`: The OpenAI API key.
- `X-OpenAI-Baseurl`: The base URL to use (e.g. a proxy) instead of the default OpenAI URL.
- `X-OpenAI-Organization`: The OpenAI organization ID.

Any additional headers provided at runtime will override the existing Weaviate configuration.

Provide the headers as shown in the [API credentials examples](https://docs.weaviate.io/weaviate/model-providers/openai/generative#api-credentials) above.

> [!NOTE]
>
> **How Weaviate builds the request URL**
>
> Use the `X-OpenAI-Baseurl` header, or the `baseURL` parameter, to target an OpenAI-compatible service. Weaviate builds the request URL by appending `/v1/chat/completions` to the base URL, or `/v1/completions` for the legacy `text-davinci-002` and `text-davinci-003` models.
>
> The generative integration has no `endpoint` parameter to override this path, unlike the [OpenAI embeddings integration](https://docs.weaviate.io/weaviate/model-providers/openai/embeddings#header-parameters). If your provider expects a different path, point the base URL at a proxy that rewrites `/v1/chat/completions` to your provider's path.
>
> The [Azure OpenAI integration](https://docs.weaviate.io/weaviate/model-providers/openai-azure/generative) builds a deployment-specific path and ignores the paths above.

## Retrieval augmented generation

After configuring the generative AI integration, perform RAG operations, either with the [single prompt](https://docs.weaviate.io/weaviate/model-providers/openai/generative#single-prompt) or [grouped task](https://docs.weaviate.io/weaviate/model-providers/openai/generative#grouped-task) method.

### Single prompt

![Single prompt RAG integration generates individual outputs per search result](https://docs.weaviate.io/assets/images/integration_openai_rag_single-752b91e4aafc5395e038395d0a4b8af4.png)

To generate text for each object in the search results, use the single prompt method.

The example below generates outputs for each of the `n` search results, where `n` is specified by the `limit` parameter.

When creating a single prompt query, use braces `{}` to interpolate the object properties you want Weaviate to pass on to the language model. For example, to pass on the object's `title` property, include `{title}` in the query.

**Python**

```py
collection = client.collections.use("DemoCollection")

response = collection.generate.near_text(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    single_prompt="Translate this into French: {title}",
    limit=2
)

for obj in response.objects:
    print(obj.properties["title"])
    print(f"Generated output: {obj.generated}")  # Note that the generated output is per object
```

**JavaScript/TypeScript**

```ts
let response;
const myCollection = client.collections.use("DemoCollection");

let myCollection = client.collections.use('DemoCollection');

const singlePromptResults = await myCollection.generate.nearText('A holiday film', {
  singlePrompt: `Translate this into French: {title}`,
}, {
  limit: 2,
});

for (const obj of singlePromptResults.objects) {
  console.log(obj.properties['title']);
  console.log(`Generated output: ${obj.generative?.text}`);  // Note that the generated output is per object
}
```

### Grouped task

![Grouped task RAG integration generates one output for the set of search results](https://docs.weaviate.io/assets/images/integration_openai_rag_grouped-1ce009f2694c9e3597b27a8255c52375.png)

To generate one text for the entire set of search results, use the grouped task method.

In other words, when you have `n` search results, the generative model generates one output for the entire group.

**Python**

```py
collection = client.collections.use("DemoCollection")

response = collection.generate.near_text(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    grouped_task="Write a fun tweet to promote readers to check out these films.",
    limit=2
)

print(f"Generated output: {response.generative.text}")  # Note that the generated output is per query
for obj in response.objects:
    print(obj.properties["title"])
```

**JavaScript/TypeScript**

```ts
let response;
const myCollection = client.collections.use("DemoCollection");

let myCollection = client.collections.use('DemoCollection');

const groupedTaskResults = await myCollection.generate.nearText('A holiday film', {
  groupedTask: `Write a fun tweet to promote readers to check out these films.`,
}, {
  limit: 2,
});

console.log(`Generated output: ${groupedTaskResults.generative?.text}`);  // Note that the generated output is per query
for (const obj of groupedTaskResults.objects) {
  console.log(obj.properties['title']);
}
```

### RAG with images

You can also supply images as a part of the input when performing retrieval augmented generation in both single prompts and grouped tasks.

**Python**

```py
import base64
import requests
from weaviate.classes.generate import GenerativeConfig, GenerativeParameters

src_img_path = "https://upload.wikimedia.org/wikipedia/commons/thumb/b/b0/Winter_forest_silver.jpg/960px-Winter_forest_silver.jpg"
base64_image = base64.b64encode(requests.get(src_img_path).content).decode('utf-8')

prompt = GenerativeParameters.grouped_task(
    prompt="Which movie is closest to the image in terms of atmosphere",
    images=[base64_image],      # A list of base64 encoded strings of the image bytes
    # image_properties=["img"], # Properties containing images in Weaviate
)

jeopardy = client.collections.use("DemoCollection")
response = jeopardy.generate.near_text(
    query="Movies",
    limit=5,
    grouped_task=prompt,
    generative_provider=GenerativeConfig.openai(
        max_tokens=1000
    ),
)

# Print the source property and the generated response
for o in response.objects:
    print(f"Title property: {o.properties['title']}")
print(f"Grouped task result: {response.generative.text}")
```

**JavaScript/TypeScript**

```ts
import { generativeParameters } from '@weaviate/node';

function arrayBufferToBase64(buffer: ArrayBuffer): string {
  const bytes = new Uint8Array(buffer);
  let binary = '';
  const chunkSize = 1024; // Process in chunks to avoid call stack issues

  for (let i = 0; i < bytes.length; i += chunkSize) {
    const chunk = bytes.slice(i, Math.min(i + chunkSize, bytes.length));
    binary += String.fromCharCode.apply(null, Array.from(chunk));
  }

  return btoa(binary);
}

let response;
const myCollection = client.collections.use("DemoCollection");

const srcImgPath = "https://upload.wikimedia.org/wikipedia/commons/thumb/b/b0/Winter_forest_silver.jpg/960px-Winter_forest_silver.jpg"
const responseImg = await fetch(srcImgPath);
const image = await responseImg.arrayBuffer();

const base64String = arrayBufferToBase64(image);

const prompt = {
  prompt: "Which movie is closest to the image in terms of atmosphere",
  images: [base64String],      // A list of base64 encoded strings of the image bytes
  // imageProperties: ["img"], // Properties containing images in Weaviate
}

response = await myCollection.generate.nearText("Movies", {
  groupedTask: prompt,
  config: generativeParameters.openAI({
    maxTokens: 1000,
  }),
}, {
  limit: 5,
})

// Print the source property and the generated response
for (const item of response.objects) {
  console.log("Title property:", item.properties['title'])
}

console.log("Grouped task result:", response.generative?.text)
```

## References

### Available models

Weaviate does not validate the model name, so you can set any model that your OpenAI account can reach. Name validation was removed in `v1.33.0`, and backported to `v1.31.17` and `v1.32.10`.

The server default is `gpt-5-mini`. It changed in `v1.32.3`, and was backported to `v1.30.16` and `v1.31.10`. Earlier releases on each of those lines default to `gpt-3.5-turbo`.

See the [OpenAI model documentation](https://platform.openai.com/docs/models) for the list of available models.

Weaviate stores a token limit for the models below. The limit caps the `maxTokens` value you can set for those models; it does not restrict which models you can use.

**Models with a stored token limit**

- [gpt-5](https://platform.openai.com/docs/models/gpt-5)
- [gpt-5-mini](https://platform.openai.com/docs/models/gpt-5-mini) (server default)
- [gpt-5-nano](https://platform.openai.com/docs/models/gpt-5-nano)
- [gpt-3.5-turbo](https://platform.openai.com/docs/models/gpt-3-5) (previous server default)
- [gpt-3.5-turbo-16k](https://platform.openai.com/docs/models/gpt-3-5)
- [gpt-3.5-turbo-1106](https://platform.openai.com/docs/models/gpt-3-5)
- [gpt-4](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo)
- [gpt-4-1106-preview](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo)
- [gpt-4-32k](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo)
- [gpt-4o](https://platform.openai.com/docs/models#gpt-4o)
- [gpt-4o-mini](https://platform.openai.com/docs/models#gpt-4o-mini)

These older models also have a stored limit, but are not recommended:

- [davinci 002](https://platform.openai.com/docs/models/overview)
- [davinci 003](https://platform.openai.com/docs/models/overview)

## Further resources

### Other integrations

- [OpenAI embedding models + Weaviate](https://docs.weaviate.io/weaviate/model-providers/openai/embeddings).

### Code examples

Once the integrations are configured at the collection, the data management and search operations in Weaviate work identically to any other collection. See the following model-agnostic examples:

- The [How-to: Manage collections](https://docs.weaviate.io/weaviate/manage-collections) and [How-to: Manage objects](https://docs.weaviate.io/weaviate/manage-objects) guides show how to perform data operations (i.e. create, read, update, delete collections and objects within them).
- The [How-to: Query & Search](https://docs.weaviate.io/weaviate/search) guides show how to perform search operations (i.e. vector, keyword, hybrid) as well as retrieval augmented generation.

### References

- OpenAI [Chat API documentation](https://platform.openai.com/docs/api-reference/chat)