Improve your cluster
This page is linked from the banner Weaviate logs once a cluster is up. If you just ran it for the first time, the checklist is for later: start with the quickstart and connect a client, and come back once you have data you would mind losing.
Most errors in the error messages reference come from clusters that were short on headroom, unmonitored, or behind on versions. This page lists what to change so the next one does not happen. Do it once at setup and again when the workload changes shape: a new collection, many more tenants, a much larger import.
Memory, disk, memory mappings, and vector memory are all knowable in advance.
More than one copy of anything you cannot rebuild quickly.
Every limit announces itself in the logs and metrics before it is an outage.
Replication copies mistakes perfectly. Backups are how you undo them.
Old versions carry every bug fixed since, and the upgrade comes anyway.
Give the cluster headroom
Weaviate fails safely when it runs out of a resource: shards go read-only or refuse to load rather than corrupting data. That is the behavior you want, but it is still an outage, and every one of these limits is knowable in advance.
Size memory for the working set, not the averageResource planning →
Set GOMEMLIMIT, or LIMIT_RESOURCES and let Weaviate derive it. Without one, the runtime assumes it may use the whole machine and gets killed by the container limit instead of collecting garbage harder.
Treat 90% disk as the ceiling, not the targetDisk thresholds →
Weaviate warns above 80% and marks shards read-only above 90%, both tunable. Compaction and backups need free space too.
Raise the memory-mapping limitThe fix →
Every shard costs kernel memory mappings, and the limit is a host setting, not a container one. Thousands of active tenants on one node pass a distribution default.
Compress vectorsCompression →
Vector indexes are the largest consumer and quantization is the biggest single lever: a small amount of recall for a large reduction in memory, and it can be enabled on a collection you already have.
Keep inactive tenants out of memoryTenant states →
Tenants you are not serving should not be resident. Deactivating or offloading them returns memory, mappings, and file handles.
Survive the loss of a node
A single-node cluster has no bad days, only outages. If Weaviate holds data you cannot rebuild quickly from somewhere else, it needs more than one copy.
Set a replication factor above oneReplication →
On collections that matter, and when you create them: raising it later copies data across the network at a moment you probably did not choose.
Use an odd number of nodes
So the cluster can still form a majority when one is lost.
Choose consistency levels deliberatelyConsistency →
Weaviate lets you pick per request, so the write path and the read path can make different trade-offs.
Turn on async replicationAsync replication →
Replicas that fall behind, or miss writes while restarting, repair themselves in the background instead of serving stale results until someone notices.
Find out before your users do
Every resource limit above announces itself in the logs and in the metrics well before it becomes an outage. That warning is only worth having if something is reading it.
Scrape the metrics endpointMonitoring →
Weaviate exposes Prometheus metrics, and there is a ready-made Grafana setup to start from.
Alert on the leading indicators
Alert on heap usage against the limit, disk usage against the read-only threshold, and shards reporting read-only. Request failures come last; by then users have noticed.
Collect the logs somewhere searchableLogging →
Weaviate logs JSON by default, so the fields are queryable without parsing message text. Alert on the action field rather than on wording, which changes between releases.
Watch the cluster node data for shards that are not readyCluster status →
It is the fastest way to tell a slow cluster from a partly broken one.
Be able to go back
Replication protects you from losing a node. It does not protect you from a bad import, a mistaken deletion, or a schema change you want to undo, because all three replicate perfectly.
Configure a backup backend and a scheduleBackups →
To storage that is not the cluster's own disk.
Restore one
An untested backup is a hypothesis. Restore into a scratch cluster and query it, at least once, before you need it to work.
Check what a backup covers
Against what you would need to rebuild, including collection configuration and inactive tenants.
Keep persistence on a volume that outlives the containerPersistence →
A container restart should not be a data-loss event.
Stay current
Running a version that is a year old means carrying every bug that has since been fixed, and paying for the upgrade later anyway, in one larger jump.
Read the release notes before upgradingRelease notes →
And the migration guide when there is one.
Check the known issues page when something is strangeKnown issues →
A surprising number of one-off mysteries are documented there with a version number attached.
Keep clients within the server's supported rangeSupport policy →
Client and server versions move independently, and a large gap between them is a common source of confusing errors.
More ways to run and extend Weaviate
Everything above applies to a cluster you run yourself. If running it is not the part you want to own, or you want more than the database:
Runs the cluster for you. Start with a Free cluster and point your existing client at it with the same connection code.
Weaviate on DigitalOcean, or Enterprise support for clusters you keep in your own environment. Pricing is public.
Generates vectors for your collections inside Weaviate Cloud, so there is no embedding model to run, host, or pay for separately.
Answers questions over your collections in natural language, choosing the searches, filters, and aggregations itself.
Long-term memory for AI agents, built on Weaviate: agents remember across sessions without you designing the storage.
Further resources
Questions and feedback
Have a question or feedback? Here's how to reach us.