--- title: "Inverted index" description: "Learn how-to configure the inverted index parameters and set the tokenization type for properties." url: https://docs.weaviate.io/weaviate/manage-collections/inverted-index --- # Inverted index An **inverted index** is a data structure in Weaviate that enables efficient text search and filtering operations. **Additional information** In Weaviate, the inverted index supports search capabilities such as keyword search, filtering, and range queries. An inverted index maps from terms (*tokens*) back to the objects that contain them. This mapping allows Weaviate to quickly identify which objects contain specific terms or match certain criteria during search queries. You can [enable inverted indexes](https://docs.weaviate.io/weaviate/manage-collections/inverted-index#enable-inverted-index-for-keyword-searches-and-filtering) on properties and adjust various parameters that control [indexing behavior](https://docs.weaviate.io/weaviate/manage-collections/inverted-index#set-inverted-index-parameters) and [tokenization strategies](https://docs.weaviate.io/weaviate/manage-collections/inverted-index#set-tokenization-type-for-property). Proper configuration of these parameters is crucial for optimizing both search performance and storage efficiency. ## Enable inverted index for keyword searches and filtering [Inverted index parameters](https://docs.weaviate.io/weaviate/config-refs/indexing/inverted-index#inverted-index-parameters) control how individual properties are indexed for search and filtering operations. These parameters determine whether specific properties can be searched, filtered, or used in range queries. **Enabling inverted index** The inverted index in Weaviate can be enabled through parameters at the property level. The names below are the REST and camelCase client spellings; the Python client uses the snake\_case equivalent, so `indexFilterable` is `index_filterable`, `indexSearchable` is `index_searchable` and `indexRangeFilters` is `index_range_filters`. **`indexFilterable`** - Controls whether a property can be used in where filters. When set to `true`, the property values are indexed for efficient filtering operations. Disable this for properties that don't need filtering to save storage space. **`indexSearchable`** - Determines whether a property participates in keyword search queries. When `true`, the property's text content is tokenized and indexed for search. Set to `false` for properties that shouldn't be searchable to improve performance. **`indexRangeFilters`** - Enables range filtering capabilities (greater than, less than, etc.) for numerical and date properties. When enabled, additional indexing structures are created to support efficient range queries. **Python** ```py from weaviate.classes.config import Configure, Property, DataType client.collections.create( "Article", # Additional settings not shown properties=[ Property( name="title", data_type=DataType.TEXT, index_filterable=True, index_searchable=True, ), Property( name="chunk", data_type=DataType.TEXT, index_filterable=True, index_searchable=True, ), Property( name="chunk_number", data_type=DataType.INT, index_range_filters=True, ), ], ) ``` **JavaScript/TypeScript** ```ts await client.collections.create({ name: 'Article', properties: [ { name: 'title', dataType: dataType.TEXT, indexFilterable: true, indexSearchable: true, }, { name: 'chunk', dataType: dataType.TEXT, indexFilterable: true, indexSearchable: true, }, { name: 'chunk_number', dataType: dataType.INT, indexRangeFilters: true, }, ], }) ``` **Go (v5)** ```go vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, }, { Name: "chunk", DataType: schema.DataTypeText.PropString(), Tokenization: "word", IndexFilterable: &vTrue, IndexSearchable: &vTrue, }, { Name: "chunk_no", DataType: schema.DataTypeInt.PropString(), IndexRangeFilters: &vTrue, }, }, InvertedIndexConfig: &models.InvertedIndexConfig{ Bm25: &models.BM25Config{ B: 0.7, K1: 1.25, }, IndexNullState: true, IndexPropertyLength: true, IndexTimestamps: true, }, } ``` **Go (v6)** ```go _, err := client.Collections.Create(ctx, collections.Collection{ Name: "Article", Properties: []collections.Property{ {Name: "title", DataType: collections.DataTypeText, IndexFilterable: new(true), IndexSearchable: new(true)}, {Name: "wordCount", DataType: collections.DataTypeInt, IndexRangeable: new(true)}, }, }) if err != nil { // handle error panic(err) } ``` **Java** ```java client.collections.create("Article", col -> col .properties( Property.text("title", p -> p.indexFilterable(true).indexSearchable(true)), Property.text("chunk", p -> p.indexFilterable(true).indexSearchable(true)), Property.integer("chunk_number", p -> p.indexRangeFilters(true))) .invertedIndex(idx -> idx.bm25(b -> b.b(1).k1(2)) .indexNulls(true) .indexPropertyLength(true) .indexTimestamps(true))); ``` **C#** ```csharp await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", indexFilterable: true, indexSearchable: true), Property.Text("chunk", indexFilterable: true, indexSearchable: true), Property.Int("chunk_number", indexRangeFilters: true), ], InvertedIndexConfig = new InvertedIndexConfig { Bm25 = new BM25Config { B = 1, K1 = 2 }, IndexNullState = true, IndexPropertyLength = true, IndexTimestamps = true, }, } ); ``` ## Set inverted index parameters [Inverted index parameters](https://docs.weaviate.io/weaviate/config-refs/indexing/inverted-index#inverted-index-parameters) control the overall behavior of the inverted index for an entire collection. These parameters affect ranking algorithms, null value handling, and timestamp indexing across all properties in the collection. **Inverted index parameters** The inverted index in Weaviate can be configured through various parameters at the collection level. The names below are the REST and camelCase client spellings. In REST, `b` and `k1` are members of the `bm25` object. The Python client uses the snake\_case equivalent, so they are `bm25_b`, `bm25_k1`, `index_null_state`, `index_property_length` and `index_timestamps`. **`bm25`: `b`** - Controls the degree of normalization by document length in the BM25 ranking algorithm. Values range from 0 to 1, where 0 means no length normalization and 1 means full normalization. Higher values favor shorter documents. **`bm25`: `k1`** - Controls term frequency saturation in BM25. Higher values make term frequency more important, while lower values reduce the impact of term frequency on scoring. **`indexNullState`** - Determines whether null values are indexed. When enabled, you can filter for objects that have null values in specific properties. **`indexPropertyLength`** - Controls whether the length of text properties is indexed. When enabled, allows filtering based on text length and can improve certain ranking algorithms. **`indexTimestamps`** - Enables indexing of creation and update timestamps for objects, allowing filtering and sorting operations. **Python** ```py from weaviate.classes.config import Configure, Property, DataType client.collections.create( "Article", # Additional settings not shown inverted_index_config=Configure.inverted_index( bm25_b=0.7, bm25_k1=1.25, index_null_state=True, index_property_length=True, index_timestamps=True, ), ) ``` **JavaScript/TypeScript** ```ts import { dataType } from '@weaviate/node'; await client.collections.create({ name: 'Article', invertedIndex: { bm25: { b: 0.7, k1: 1.25 }, indexNullState: true, indexPropertyLength: true, indexTimestamps: true } }) ``` **Go (v5)** ```go vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, }, { Name: "chunk", DataType: schema.DataTypeText.PropString(), Tokenization: "word", IndexFilterable: &vTrue, IndexSearchable: &vTrue, }, { Name: "chunk_no", DataType: schema.DataTypeInt.PropString(), IndexRangeFilters: &vTrue, }, }, InvertedIndexConfig: &models.InvertedIndexConfig{ Bm25: &models.BM25Config{ B: 0.7, K1: 1.25, }, IndexNullState: true, IndexPropertyLength: true, IndexTimestamps: true, }, } ``` **Go (v6)** ```go _, err := client.Collections.Create(ctx, collections.Collection{ Name: "Article", Properties: []collections.Property{ {Name: "title", DataType: collections.DataTypeText, IndexFilterable: new(true), IndexSearchable: new(true), Tokenization: collections.TokenizationWord}, {Name: "chunk", DataType: collections.DataTypeText, IndexFilterable: new(true), IndexSearchable: new(true), Tokenization: collections.TokenizationField}, {Name: "chunk_number", DataType: collections.DataTypeInt, IndexRangeable: new(true)}, }, InvertedIndex: &collections.InvertedIndexConfig{ BM25: &collections.BM25Config{ B: 0.7, K1: 1.25, }, Stopwords: &collections.StopwordConfig{ Preset: "en", Additions: []string{"example", "stopword"}, Removals: []string{"the", "and"}, }, IndexNullState: true, IndexPropertyLength: true, IndexTimestamps: true, }, }) if err != nil { // handle error panic(err) } ``` **Java** ```java client.collections.create("Article", col -> col .properties( Property.text("title", p -> p.indexFilterable(true).indexSearchable(true)), Property.text("chunk", p -> p.indexFilterable(true).indexSearchable(true)), Property.integer("chunk_number", p -> p.indexRangeFilters(true))) .invertedIndex(idx -> idx.bm25(b -> b.b(1).k1(2)) .indexNulls(true) .indexPropertyLength(true) .indexTimestamps(true))); ``` **C#** ```csharp await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", indexFilterable: true, indexSearchable: true), Property.Text("chunk", indexFilterable: true, indexSearchable: true), Property.Int("chunk_number", indexRangeFilters: true), ], InvertedIndexConfig = new InvertedIndexConfig { Bm25 = new BM25Config { B = 1, K1 = 2 }, IndexNullState = true, IndexPropertyLength = true, IndexTimestamps = true, }, } ); ``` ## Drop an inverted index Drop (delete) an inverted index from a property. This is a destructive operation: the index data is removed from disk. To use the index again, it must be regenerated. The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. **Python** ```py collection = client.collections.get("Article") # Drop the searchable inverted index from the "title" property collection.config.delete_property_index("title", "searchable") # Drop the filterable inverted index from the "title" property collection.config.delete_property_index("title", "filterable") # Drop the range filter index from the "chunk_number" property collection.config.delete_property_index("chunk_number", "rangeFilters") ``` **JavaScript/TypeScript** ```ts const article = client.collections.use('Article') // Drop the searchable inverted index from the "title" property await article.config.dropInvertedIndex('title', 'searchable') // Drop the filterable inverted index from the "title" property await article.config.dropInvertedIndex('title', 'filterable') // Drop the range filter index from the "chunk_number" property await article.config.dropInvertedIndex('chunk_number', 'rangeFilters') ``` **Go (v5)** ```go collection := client.Schema() // Drop the searchable inverted index from the "title" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("title"). WithSearchable(). Do(ctx) // Drop the filterable inverted index from the "title" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("title"). WithFilterable(). Do(ctx) // Drop the range filter index from the "chunk_no" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("chunk_no"). WithRangeFilters(). Do(ctx) ``` **Go (v6)** ```go collection := client.Collections.Use("Article") // Drop the searchable inverted index from the "title" property err := collection.Config.DropPropertyIndex(ctx, collections.DropPropertyIndexOptions{ PropertyName: "title", IndexType: collections.IndexSearchable, }) if err != nil { // handle error panic(err) } // Drop the filterable inverted index from the "title" property err = collection.Config.DropPropertyIndex(ctx, collections.DropPropertyIndexOptions{ PropertyName: "title", IndexType: collections.IndexFilterable, }) if err != nil { // handle error panic(err) } // Drop the range filter index from the "chunk_number" property err = collection.Config.DropPropertyIndex(ctx, collections.DropPropertyIndexOptions{ PropertyName: "chunk_number", IndexType: collections.IndexRangeable, }) if err != nil { // handle error panic(err) } ``` **Java** ```java var collection = client.collections.use("Article"); // Drop the searchable inverted index from the "title" property collection.config.dropPropertyIndex("title", PropertyIndexType.SEARCHABLE); // Drop the filterable inverted index from the "title" property collection.config.dropPropertyIndex("title", PropertyIndexType.FILTERABLE); // Drop the range filter index from the "chunk_number" property collection.config.dropPropertyIndex("chunk_number", PropertyIndexType.RANGE_FILTERS); ``` ## Set tokenization type for property Configure a [tokenization method](https://docs.weaviate.io/weaviate/config-refs/collections#tokenization) for each property individually. **Tokenization methods** Tokenization determines how text content is broken down into individual terms that can be indexed and searched. Weaviate supports several tokenization strategies: **`word`** - The default tokenization that splits text on whitespace and punctuation, converting to lowercase. Best for general text search where you want to match individual words. **`lowercase`** - Splits text on whitespace only, then lowercases each token. Preserves symbols (like `&`, `@`, `_`) that `word` tokenization would strip. Good for case-insensitive matching where punctuation is meaningful, such as code snippets or email addresses. **`whitespace`** - Splits text only on whitespace characters, preserving punctuation and case. Good when punctuation is meaningful for search. **`field`** - Treats the entire property value as a single token without any processing. Use for exact matching of complete field values like IDs, email addresses, or URLs. **`trigram`** - Breaks text into overlapping 3-character sequences. Enables fuzzy matching and is useful for handling typos or partial matches. **`gse`** - Language-aware tokenization for Japanese text. Disabled by default. Enable with the `ENABLE_TOKENIZER_GSE` environment variable. **`gse_ch`** - The same tokenizer with the Chinese dictionary. Use this value, not `gse`, for Chinese text. Disabled by default. Enable with the `ENABLE_TOKENIZER_GSE_CH` environment variable. For Korean text, see the `kagome_kr` option. For the full list of supported tokenizers (including `kagome_ja`, `kagome_kr`, and the per-property text-analyzer options), see the [tokenization reference](https://docs.weaviate.io/weaviate/config-refs/collections#tokenization). **Python** ```py from weaviate.classes.config import Configure, Property, DataType, Tokenization client.collections.create( "Article", vector_config=Configure.Vectors.text2vec_cohere(), properties=[ Property( name="title", data_type=DataType.TEXT, tokenization=Tokenization.LOWERCASE, # Use "lowercase" tokenization description="The title of the article.", # Optional description ), Property( name="body", data_type=DataType.TEXT, tokenization=Tokenization.WHITESPACE, # Use "whitespace" tokenization ), ], ) ``` **JavaScript/TypeScript** ```ts const newCollection = await client.collections.create({ name: 'Article', vectorizers: vectors.text2VecHuggingFace(), properties: [ { name: 'title', dataType: dataType.TEXT, tokenization: tokenization.LOWERCASE }, { name: 'body', dataType: dataType.TEXT, tokenization: tokenization.WHITESPACE }, ], }) ``` **Go (v5)** ```go vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, ModuleConfig: map[string]interface{}{ "text2vec-cohere": map[string]interface{}{ "vectorizePropertyName": true, }, }, }, { Name: "body", DataType: schema.DataTypeText.PropString(), Tokenization: "whitespace", IndexFilterable: &vTrue, IndexSearchable: &vTrue, ModuleConfig: map[string]interface{}{ "text2vec-cohere": map[string]interface{}{ "vectorizePropertyName": false, }, }, }, }, Vectorizer: "text2vec-cohere", } ``` **Go (v6)** ```go _, err := client.Collections.Create(ctx, collections.Collection{ Name: "Article", Properties: []collections.Property{ { Name: "title", DataType: collections.DataTypeText, Tokenization: collections.TokenizationLowercase, // Use "lowercase" tokenization Description: "The title of the article.", // Optional description }, { Name: "body", DataType: collections.DataTypeText, Tokenization: collections.TokenizationWhitespace, // Use "whitespace" tokenization }, }, Vectors: map[string]collections.VectorConfig{ // Vectorize the title only. The body is left out of the vector. "default": {Vectorizer: model2vec.Text2Vec{Properties: []string{"title"}}}, }, }) if err != nil { // handle error panic(err) } ``` **Java** ```java client.collections.create("Article", col -> col.properties( Property.text("title", p -> p.description("The title of the article.") .tokenization(Tokenization.LOWERCASE) .vectorizePropertyName(false)), Property.text("body", p -> p.skipVectorization(true) .tokenization(Tokenization.WHITESPACE)))); ``` **C#** ```csharp await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", tokenization: PropertyTokenization.Lowercase), Property.Text("body", tokenization: PropertyTokenization.Whitespace), ], } ); ``` ## Further resources - [References: Collection definition](https://docs.weaviate.io/weaviate/config-refs/collections) - [Concepts: Inverted index](https://docs.weaviate.io/weaviate/config-refs/indexing/inverted-index)