·15 min read

Weaviate: An AI-native vector database (BSD-3-Clause, 12k stars)

An AI-native vector database with built-in vectorization modules, hybrid search, and seamless integration with any LLM.

#weaviate#vector-database#hybrid-search#vectorization#ai-data

The Problem

Every RAG pipeline starts the same way: you chunk your documents, call an embedding API, store the vectors, and write a search function that retrieves relevant chunks. This works — until it doesn’t.

The cracks appear fast. Your embedding model changes, and now every vector in your database is stale. Your search returns irrelevant results because pure vector similarity misses keyword matches. Your retrieval pipeline needs three separate services (embedding server, vector store, LLM gateway) just to answer one question. And when you need to filter by metadata — “only results from 2025” — you discover your vector database does post-filtering, which silently drops relevant results.

The result is a stack that works in demos and breaks in production.

Dimension Naive RAG Stack Weaviate
Embedding pipeline External API call per write Built-in vectorizer modules (auto-embed on write)
Search types Vector only Vector + BM25 + hybrid (fused)
Filtering Post-filter (recall loss) Pre-filter via inverted index
RAG generation External LLM call Built-in generative search (one query)
API surface REST + custom SDK GraphQL + REST + gRPC
Schema management Manual migrations Auto-schema with class definitions
Multi-modal support Custom pipeline Native (text, image, video, DNA)
Horizontal scaling External load balancer Built-in sharding + replication (Raft)
Memory efficiency Full HNSW in RAM HFresh disk-backed index (v1.36+)
Setup time 2-3 days (stitching services) 5 minutes (Docker)

Why this matters: The naive RAG stack treats vector search as a standalone component. Weaviate treats it as a database — with schema, indexing, filtering, and query planning built in. The difference is not feature count. It is architectural philosophy. A database manages data. A vector store stores vectors. Weaviate is the former. Most alternatives are the latter.

The Investigation

The Weaviate team (founded 2019, headquartered in Amsterdam) spent five years investigating a single question: what does a database look like when it is designed for AI workloads from day one?

Finding 1: Embedding management is the hidden tax of vector search.

Every vector database stores embeddings. But who generates them? In the naive stack, the application calls an embedding API (OpenAI, Cohere, Hugging Face) before every write. This means:

  • The application must manage embedding API keys, rate limits, and retries.
  • Changing the embedding model requires re-embedding every object in the database.
  • The embedding pipeline is a separate failure domain from the database.

Weaviate’s investigation concluded that the database should own the embedding pipeline. Its vectorizer modules — configured at the class (collection) level — automatically generate embeddings on write. The application writes raw text; Weaviate calls the embedding model, stores the vector, and indexes it. On read, the same vectorizer is applied to the query text.

What this means: The application never touches a vector. It writes and reads natural language. Weaviate handles the embedding lifecycle. This eliminates an entire class of bugs (stale embeddings, mismatched models, rate limit errors) and reduces the application codebase by hundreds of lines.

Finding 2: Pure vector search is not enough for production.

The BEIR benchmark suite evaluates search quality across 18 datasets. Weaviate’s internal benchmarks (published on GitHub) show a consistent pattern: hybrid search (vector + BM25) outperforms pure vector search on every dataset.

Dataset Pure Vector (alpha=0) Pure BM25 (alpha=1) Hybrid (alpha=0.5) Improvement
NFCorpus 0.224 0.264 0.280 +25% vs vector
SciFact 0.678 0.683 0.714 +5% vs vector
FIQA 0.284 0.434 0.428 +51% vs vector
Quora 0.770 0.887 0.867 +13% vs vector
Natural Questions 0.244 0.438 0.380 +56% vs vector

The improvement is largest on datasets where keyword matches matter (FIQA, Natural Questions) and smallest on datasets where semantic similarity dominates (SciFact). The takeaway: production search needs both signals.

What this means: If your vector database only supports vector search, you are leaving 25-56% of relevant results on the table. Hybrid search is not a nice-to-have. It is the minimum viable search strategy for production RAG.

Finding 3: The fusion algorithm matters as much as the search.

Hybrid search runs two searches (vector + BM25) and fuses the results. The fusion algorithm determines how the two ranked lists are combined. Weaviate investigated two approaches:

  • rankedFusion (original): Assigns scores based on rank position only. The top result gets the highest score, regardless of how close the match was.
  • relativeScoreFusion (default since v1.24): Preserves the actual similarity scores from each search. A near-perfect vector match still scores high even if its BM25 rank is low.

On the FIQA dataset, relativeScoreFusion showed a ~6% improvement in recall over rankedFusion. The reason: rankedFusion discards information (the actual distance/similarity), while relativeScoreFusion preserves it.

What this means: The fusion algorithm is not a configuration detail. It is a core architectural decision that directly impacts search quality. Weaviate’s default (relativeScoreFusion) is the right choice for most workloads, but the option to switch exists for specific use cases.

The Solution

Weaviate is a ~200,000-line Go application (BSD-3-Clause license, 12,000+ GitHub stars) that stores both data objects and their vector embeddings in a single database. It provides built-in vectorization, hybrid search, and generative search — all through a unified GraphQL + REST API.

┌──────────────────────────────────────────────────────────────────────────┐
│                          Weaviate Architecture                            │
│                                                                           │
│  ┌────────────────────────────────────────────────────────────────────┐  │
│  │                         API Layer                                    │  │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │  │
│  │  │   GraphQL    │  │    REST      │  │    gRPC      │              │  │
│  │  │  (primary)   │  │  (secondary) │  │  (bulk ops)  │              │  │
│  │  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘              │  │
│  └─────────┼─────────────────┼─────────────────┼──────────────────────┘  │
│            │                 │                 │                          │
│  ┌─────────┴─────────────────┴─────────────────┴──────────────────────┐  │
│  │                     Request Router / Coordinator                    │  │
│  │  ┌─────────────────────────────────────────────────────────────┐  │  │
│  │  │  • Query parsing & planning                                  │  │  │
│  │  │  • Shard routing (consistent hashing)                        │  │  │
│  │  │  • Result merging & fusion (hybrid search)                   │  │  │
│  │  └─────────────────────────────────────────────────────────────┘  │  │
│  └────────────────────────────────────────────────────────────────────┘  │
│                                                                           │
│  ┌────────────────────────────────────────────────────────────────────┐  │
│  │                     Shard (per node)                                │  │
│  │                                                                     │  │
│  │  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐ │  │
│  │  │   Object Store    │  │  Inverted Index   │  │  Vector Index    │ │  │
│  │  │  (LSM-Tree)      │  │  (LSM-Tree)       │  │  (HNSW / Flat /  │ │  │
│  │  │                  │  │                  │  │   Dynamic /       │ │  │
│  │  │  • Key-value     │  │  • BM25 FTS      │  │   HFresh)         │ │  │
│  │  │  • WAL + Bloom   │  │  • Boolean       │  │                  │ │  │
│  │  │  • CRUD ops      │  │  • Range filters  │  │  • ANN search    │ │  │
│  │  └──────────────────┘  └──────────────────┘  └──────────────────┘ │  │
│  │                                                                     │  │
│  │  ┌──────────────────────────────────────────────────────────────┐  │  │
│  │  │              Vectorizer Module (optional)                     │  │  │
│  │  │  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────────┐  │  │
│  │  │  │ OpenAI   │ │ Cohere   │ │ Hugging  │ │ Custom (any    │  │  │
│  │  │  │ module   │ │ module   │ │ Face mod │ │ OpenAI-compat)  │  │  │
│  │  │  └──────────┘ └──────────┘ └──────────┘ └────────────────┘  │  │
│  │  └──────────────────────────────────────────────────────────────┘  │  │
│  └────────────────────────────────────────────────────────────────────┘  │
│                                                                           │
│  ┌────────────────────────────────────────────────────────────────────┐  │
│  │                     Cluster Layer                                   │  │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────┐  │  │
│  │  │   Raft       │  │  Memberlist  │  │  Shard Replica Movement  │  │  │
│  │  │  (schema +   │  │  (gossip,   │  │  (v1.32+, load balance)  │  │  │
│  │  │   metadata)  │  │  discovery)  │  │                          │  │  │
│  │  └──────────────┘  └──────────────┘  └──────────────────────────┘  │  │
│  └────────────────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────────┘

Here is what each layer does:

  • API Layer: GraphQL is the primary query interface (expressive, nested queries for hybrid + generative search). REST handles CRUD operations and schema management. gRPC is used for bulk imports and high-throughput operations between clients and the database.
  • Request Router / Coordinator: The coordinator node parses incoming queries, routes them to the correct shards (via consistent hashing), and merges results. For hybrid search, this is where the fusion algorithm (relativeScoreFusion or rankedFusion) runs.
  • Shard: Each shard is a self-contained storage unit with three stores. The Object Store (LSM-Tree) holds the raw data. The Inverted Index (LSM-Tree) enables BM25 full-text search and structured filtering. The Vector Index (pluggable: HNSW, Flat, Dynamic, or HFresh) enables ANN search.
  • Vectorizer Module: Optional modules that auto-vectorize data on write. Configured per class. Supports OpenAI, Cohere, Hugging Face, and any OpenAI-compatible endpoint. The application writes raw text; Weaviate generates and stores the vector.
  • Cluster Layer: Raft consensus for schema changes and metadata coordination. Memberlist (Hashicorp’s gossip protocol) for node discovery. Shard Replica Movement (v1.32+) for dynamic load balancing.

Setup

# Start Weaviate with Docker (single node, all modules)
docker run -d \
  --name weaviate \
  -p 8080:8080 \
  -p 50051:50051 \
  -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
  -e PERSISTENCE_DATA_PATH=/var/lib/weaviate \
  -e DEFAULT_VECTORIZER_MODULE=text2vec-openai \
  -e OPENAI_APIKEY=sk-... \
  semitechnologies/weaviate:latest

# Verify it's running
curl http://localhost:8080/v1/meta

# Docker Compose for production (with modules)
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
  weaviate:
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
      - "50051:50051"
    environment:
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: "true"
      PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
      DEFAULT_VECTORIZER_MODULE: "text2vec-openai"
      ENABLE_MODULES: "text2vec-openai,generative-openai,qna-openai"
      OPENAI_APIKEY: "${OPENAI_APIKEY}"
      CLUSTER_HOSTNAME: "node1"
    volumes:
      - weaviate_data:/var/lib/weaviate
    restart: unless-stopped

volumes:
  weaviate_data:
EOF

# Start
docker compose up -d

Production-Grade Configuration

# docker-compose.yml — production cluster (3 nodes)
version: '3.8'
services:
  weaviate-node1: &weaviate-base
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
      - "50051:50051"
    environment:
      AUTHENTICATION_APIKEY_ENABLED: "true"
      AUTHENTICATION_APIKEY_ALLOWED_KEYS: "sk-weaviate-prod-..."
      AUTHORIZATION_ADMINLIST_ENABLED: "true"
      AUTHORIZATION_ADMINLIST_USERS: "admin@nivantlabs.com"
      PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
      DEFAULT_VECTORIZER_MODULE: "text2vec-openai"
      ENABLE_MODULES: "text2vec-openai,generative-openai"
      OPENAI_APIKEY: "${OPENAI_APIKEY}"
      CLUSTER_HOSTNAME: "node1"
      CLUSTER_GOSSIP_BIND_PORT: "7100"
      CLUSTER_DATA_BIND_PORT: "7101"
      RAFT_BIND_PORT: "7102"
      DISK_USE_WARNING_PERCENTAGE: "80"
      DISK_USE_READONLY_PERCENTAGE: "95"
    volumes:
      - weaviate_data_1:/var/lib/weaviate
    restart: unless-stopped

  weaviate-node2:
    <<: *weaviate-base
    ports:
      - "8081:8080"
      - "50052:50051"
    environment:
      <<: *weaviate-base.environment
      CLUSTER_HOSTNAME: "node2"
      CLUSTER_JOIN: "weaviate-node1:7100"
    volumes:
      - weaviate_data_2:/var/lib/weaviate

  weaviate-node3:
    <<: *weaviate-base
    ports:
      - "8082:8080"
      - "50053:50051"
    environment:
      <<: *weaviate-base.environment
      CLUSTER_HOSTNAME: "node3"
      CLUSTER_JOIN: "weaviate-node1:7100"
    volumes:
      - weaviate_data_3:/var/lib/weaviate

volumes:
  weaviate_data_1:
  weaviate_data_2:
  weaviate_data_3:

Code Walkthrough: Schema Definition and Data Ingestion

Weaviate uses a class-based schema model. Each class is analogous to a table in SQL or a collection in MongoDB. Here is the full lifecycle:

# weaviate_client.py — production-grade Weaviate client
import weaviate
import weaviate.classes as wvc
from weaviate.classes.config import Property, DataType, Configure
from weaviate.classes.data import DataObject
from weaviate.util import generate_uuid5
import os

# Connect to Weaviate
client = weaviate.connect_to_local(
    headers={
        "X-OpenAI-Api-Key": os.environ["OPENAI_APIKEY"],
    }
)

# 1. Create a class (collection) with auto-vectorization
#    The vectorizer module automatically generates embeddings on write.
#    The application never touches a vector.
try:
    client.collections.delete("Document")
except weaviate.exceptions.WeaviateClosedError:
    pass

documents = client.collections.create(
    name="Document",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small",
        vectorize_property_name=False,
    ),
    # Define which properties should be vectorized
    properties=[
        Property(name="title", data_type=DataType.TEXT, vectorize_property_name=False),
        Property(name="content", data_type=DataType.TEXT, vectorize_property_name=True),
        Property(name="category", data_type=DataType.TEXT, vectorize_property_name=False),
        Property(name="published_date", data_type=DataType.DATE),
        Property(name="author", data_type=DataType.TEXT, vectorize_property_name=False),
        Property(name="view_count", data_type=DataType.INT),
        Property(name="tags", data_type=DataType.TEXT_ARRAY),
    ],
)

# 2. Ingest data — Weaviate auto-vectorizes the "content" field
documents.data.insert_many([
    DataObject(
        properties={
            "title": "Weaviate Architecture Overview",
            "content": "Weaviate uses a modular architecture with pluggable vector indexes...",
            "category": "technical",
            "published_date": "2026-06-15T00:00:00Z",
            "author": "Nivant Labs",
            "view_count": 1243,
            "tags": ["weaviate", "architecture", "vector-database"],
        },
        uuid=generate_uuid5("Weaviate Architecture Overview"),
    ),
    DataObject(
        properties={
            "title": "Hybrid Search Best Practices",
            "content": "Hybrid search combines vector similarity with BM25 keyword matching...",
            "category": "guide",
            "published_date": "2026-06-20T00:00:00Z",
            "author": "Nivant Labs",
            "view_count": 892,
            "tags": ["hybrid-search", "bm25", "retrieval"],
        },
        uuid=generate_uuid5("Hybrid Search Best Practices"),
    ),
    DataObject(
        properties={
            "title": "RAG Pipeline Optimization",
            "content": "Retrieval Augmented Generation pipelines benefit from hybrid search...",
            "category": "technical",
            "published_date": "2026-06-22T00:00:00Z",
            "author": "Nivant Labs",
            "view_count": 2105,
            "tags": ["rag", "retrieval", "generation"],
        },
        uuid=generate_uuid5("RAG Pipeline Optimization"),
    ),
])

# 3. Hybrid search — vector + BM25 fused in a single query
response = documents.query.hybrid(
    query="How do I optimize RAG pipelines?",
    alpha=0.5,  # 0.0 = pure BM25, 1.0 = pure vector
    limit=5,
    # Pre-filter by metadata (uses inverted index, not post-filter)
    where=wvc.query.Filter.by_property("category").equal("technical"),
)

for obj in response.objects:
    print(f"  [{obj.properties['category']}] {obj.properties['title']} (score: {obj.metadata.score:.3f})")

# 4. Generative search — retrieve + generate in one query
#    Weaviate retrieves the top-k results, then sends them to an LLM
#    along with the user's prompt for answer generation.
response = documents.generate.hybrid(
    query="Summarize the key points about RAG optimization",
    alpha=0.5,
    limit=3,
    grouped_task="Based on the retrieved documents, provide a concise summary of RAG pipeline optimization techniques.",
)

print(f"Generated summary:\n{response.generated}")

# 5. Pure vector search (when you want semantic only)
response = documents.query.near_text(
    query="database architecture patterns",
    limit=5,
)

# 6. Pure BM25 search (when you want keyword matching)
response = documents.query.bm25(
    query="hybrid search best practices",
    limit=5,
)

# 7. Cleanup
client.close()

The key architectural insight: the application never calls an embedding API. The vectorizer_config on the class definition tells Weaviate which embedding model to use. On write, Weaviate calls the model, stores the vector, and indexes it. On read, Weaviate vectorizes the query text using the same model. The application writes and reads natural language.

How to Use Effectively

Step 1: Choose the right vector index for your data size

Weaviate offers four vector index types. Pick based on your dataset size and memory budget:

# HNSW — default, best for most workloads (fast, accurate, memory-heavy)
client.collections.create(
    name="LargeCollection",
    vector_index_config=Configure.VectorIndex.hnsw(
        ef_construction=128,   # higher = better recall, slower import
        ef=64,                 # higher = better recall, slower query
        max_connections=64,    # HNSW graph parameter
        dynamic_ef_min=64,     # adaptive search width
        dynamic_ef_max=256,
        vector_cache_max_objects=1000000,  # cache hot vectors in RAM
    ),
)

# Flat — for small collections (<10K vectors) or multi-tenancy
client.collections.create(
    name="SmallCollection",
    vector_index_config=Configure.VectorIndex.flat(
        distance_metric="cosine",
    ),
)

# Dynamic — auto-switches from Flat to HNSW as the collection grows (v1.25+)
client.collections.create(
    name="GrowingCollection",
    vector_index_config=Configure.VectorIndex.dynamic(
        distance_metric="cosine",
        hnsw_ef_construction=128,
        hnsw_max_connections=64,
    ),
)

# HFresh — disk-backed, memory-efficient for large datasets (v1.36+)
client.collections.create(
    name="MassiveCollection",
    vector_index_config=Configure.VectorIndex.hfresh(
        distance_metric="cosine",
        # Only a compressed centroid index stays in RAM
        # Posting lists are on disk
    ),
)

Step 2: Configure hybrid search alpha per query

The alpha parameter controls the balance between vector and BM25 search. Tune it per query, not per collection:

# Keyword-heavy query (product codes, names, exact matches)
response = collection.query.hybrid(
    query="ERR-404 error code handling",
    alpha=0.2,  # 80% BM25, 20% vector
    limit=10,
)

# Semantic-heavy query (concepts, descriptions, open-ended)
response = collection.query.hybrid(
    query="What are the best practices for error handling in distributed systems?",
    alpha=0.8,  # 80% vector, 20% BM25
    limit=10,
)

# Balanced (default, works for most queries)
response = collection.query.hybrid(
    query="error handling patterns",
    alpha=0.5,  # equal weight
    limit=10,
)

Step 3: Use pre-filtering for metadata constraints

Weaviate’s inverted index enables pre-filtering — the filter is applied before the vector search, so recall is not affected:

# Pre-filter by date range and category
from weaviate.classes.query import Filter

response = collection.query.hybrid(
    query="machine learning deployment strategies",
    alpha=0.5,
    limit=10,
    where=(
        Filter.by_property("published_date").greater_or_equal("2025-01-01")
        & Filter.by_property("category").equal("technical")
        & Filter.by_property("tags").contains_any(["mlops", "deployment"])
    ),
)

Production pitfall: Post-filtering (running ANN search first, then applying filters) silently drops relevant results. If 90% of your data is filtered out, the top-10 ANN results may all be from the filtered set, and you get zero results despite having relevant data. Weaviate’s pre-filtering avoids this entirely. Always use pre-filtering when metadata constraints are part of your query.

Step 4: Use generative search for RAG without orchestration

Generative search combines retrieval and generation in a single query. Weaviate retrieves the top-k results, then sends them to a configured LLM along with your prompt:

# Single-prompt generation (each result gets its own generation)
response = collection.generate.hybrid(
    query="What are the key considerations for RAG deployment?",
    alpha=0.5,
    limit=5,
    single_prompt="Translate this document into Spanish: {title}\n{content}",
)

for obj in response.objects:
    print(f"Original: {obj.properties['title']}")
    print(f"Spanish: {obj.generated}")

# Grouped generation (all results are sent together for a single generation)
response = collection.generate.hybrid(
    query="RAG deployment best practices",
    alpha=0.5,
    limit=5,
    grouped_task="Extract a numbered list of deployment best practices from these documents.",
)

print(response.generated)

Step 5: Monitor and tune performance

# Check shard status and vector cache hit rate
# Weaviate exposes Prometheus metrics at /metrics
# Key metrics to monitor:
#   - weaviate_vector_index_cache_hit_ratio
#   - weaviate_vector_index_cache_miss_ratio
#   - weaviate_hnsw_query_latency_ms
#   - weaviate_hnsw_insert_latency_ms
#   - weaviate_bm25_query_latency_ms
#   - weaviate_shard_unloaded_total

# Configure vector cache prefill behavior (v1.36+)
# In docker-compose environment:
#   VECTOR_CACHE_PREFILL: "sync"    # block startup until cache is warm
#   VECTOR_CACHE_PREFILL: "async"   # serve queries while cache warms
#   VECTOR_CACHE_PREFILL: "off"     # no prefill (cold start)

Use Cases

1. Production RAG Pipeline

When you’d use this: You are building a RAG system that needs to answer questions from a large document corpus with high accuracy and low latency.

Why Weaviate fits: Built-in vectorization eliminates the embedding pipeline. Hybrid search (vector + BM25) catches both semantic matches and keyword matches. Generative search combines retrieval and generation in one query, eliminating the orchestration layer. Pre-filtering ensures metadata constraints do not silently drop results. Real-world deployments at companies like SAP and NVIDIA use Weaviate for production RAG at scale.

When you’d use this: You need to search across text, images, and other modalities with a single query interface.

Why Weaviate fits: Weaviate supports multi-modal vectorization through modules like multi2vec-clip and img2vec-neural. You can upload an image and search for similar images, or search for text descriptions that match an image. The same hybrid search and filtering infrastructure works across all modalities. Use cases include e-commerce product search, medical imaging retrieval, and digital asset management.

3. Knowledge Graph with Semantic Relationships

When you’d use this: You are building a knowledge base where entities have complex relationships and need to be searchable by meaning, not just keywords.

Why Weaviate fits: Weaviate’s class-based schema with cross-references enables graph-like data models. Each class can reference other classes, and queries can traverse these references. Combined with vector search, this enables queries like “find documents similar to this concept, written by authors in this department, published in the last year.” The cross-reference traversal happens in the same query as the vector search.

4. Agentic AI Memory and Context

When you’d use this: You are building an AI agent that needs persistent, searchable memory across sessions.

Why Weaviate fits: Weaviate’s Engram module provides persistent memory for LLM agents. The agent writes conversation summaries, extracted facts, and action results as Weaviate objects. On subsequent interactions, the agent queries Weaviate for relevant context using hybrid search. The built-in vectorization means the agent writes natural language and gets semantic matches back. Use cases include personal AI assistants, coding agents with project memory, and customer support bots with session history.

5. Real-Time Recommendation Engine

When you’d use this: You need to serve personalized recommendations based on user behavior and content similarity.

Why Weaviate fits: Weaviate’s low-latency HNSW index (sub-10ms at 1M vectors) supports real-time recommendation queries. User embeddings can be stored as vectors and compared against content embeddings. Hybrid search ensures that both semantic similarity and metadata filters (category, price range, popularity) are combined. The CRUD API supports real-time updates as user preferences change. Use cases include content recommendation, product recommendation, and personalized search results.

Cheat Sheet

Aspect Detail
Repository github.com/weaviate/weaviate
License BSD-3-Clause
Language Go (~200,000 lines)
GPU Requirements None (vectorization runs on CPU or external API)
Setup Time 5 minutes (Docker)
Key Features Auto-vectorization, hybrid search (vector + BM25), generative search, HNSW/Flat/Dynamic/HFresh indexes, pre-filtering, sharding + replication, multi-modal, GraphQL + REST + gRPC APIs
Common Gotchas GraphQL learning curve; re-vectorizing required when changing embedding models; memory usage with HNSW on large datasets; alpha tuning per query
Best Models text-embedding-3-small (OpenAI), embed-multilingual-v3.0 (Cohere), all-MiniLM-L6-v2 (Hugging Face)
Cost (Self-Hosted) RAM + CPU costs only; ~$50-200/mo for 1M vectors (HNSW)
Cost (Weaviate Cloud) Free tier available; ~$25/mo starter; ~$200-500/mo at 1M vectors
Missing Features No built-in SQL interface; no native time-series support; no built-in data transformation pipeline (ETL)

Vibe Coding Projects

What it does: A documentation search engine that indexes a set of markdown files (your project’s docs, a framework’s docs, or any technical documentation) and provides hybrid search with generative answers. Users type a question and get both relevant document excerpts and an AI-generated answer.

What you’ll learn: How to set up Weaviate with auto-vectorization. How to ingest structured data with metadata. How to use hybrid search with alpha tuning. How to use generative search for RAG. How to build a simple search UI that queries Weaviate’s GraphQL API.

Effort: 3-4 hours. ~$2-5 in API costs (OpenAI embeddings + generation).

Project 2: Multi-Modal Product Catalog

What it does: A product catalog where users can search by text description (“red running shoes with arch support”) or by uploading an image (find products that look like this photo). Uses Weaviate’s multi2vec-clip module for multi-modal embeddings.

What you’ll learn: How to configure multi-modal vectorization. How to ingest both text and image data. How to query across modalities. How to combine vector search with metadata filters (price range, category, brand). How to handle image preprocessing and upload.

Effort: 5-7 hours. ~$5-10 in API costs.

Project 3: AI Agent with Persistent Memory

What it does: A conversational AI agent that remembers past conversations. Each session stores conversation summaries, extracted facts, and action results in Weaviate. On subsequent interactions, the agent retrieves relevant context using hybrid search and uses it to inform its responses.

What you’ll learn: How to use Weaviate as an agentic memory store. How to design a schema for conversation data. How to implement a memory retrieval pipeline with hybrid search. How to integrate Weaviate with an LLM for context-aware responses. How to handle memory eviction and summarization for long-running agents.

Effort: 4-6 hours. ~$3-8 in API costs.

Problems Solved Efficiently

Problem Type Why Weaviate Fits When to Look Elsewhere
Production RAG pipeline Built-in vectorization + hybrid search + generative search in one system Use pgvector if you already run PostgreSQL and have <5M vectors
Multi-modal search Native support for text, image, video, DNA embeddings Use Qdrant for pure vector search with complex metadata filters
Knowledge graphs with semantics Class-based schema with cross-references + vector search Use Neo4j for graph-native workloads (traversal-heavy queries)
Agentic memory Engram module + hybrid search for persistent context Use Redis for ephemeral, high-throughput key-value memory
Real-time recommendations Sub-10ms HNSW queries + CRUD for real-time updates Use Qdrant for sub-5ms latency at >10M vectors
Hybrid search (vector + BM25) Native implementation with configurable fusion algorithms Use Elasticsearch for full-text search dominance
Compliance / data sovereignty Self-hosted, BSD-3-Clause, no data leaves your infra Use Pinecone for fully managed, zero-ops vector search

Architectural Tradeoffs

What we gained:

  • AI-native architecture. Weaviate was designed for AI workloads from day one. Vectorization, hybrid search, and generative search are not add-ons — they are core features. The application writes natural language; Weaviate handles the embedding lifecycle.
  • Pre-filtering without recall loss. The inverted index enables pre-filtering — metadata constraints are applied before the vector search. This is architecturally superior to post-filtering (used by pgvector and some other vector databases) because it does not silently drop relevant results.
  • Pluggable vector indexes. Four index types (HNSW, Flat, Dynamic, HFresh) cover the full spectrum from small collections to massive datasets. The Dynamic index auto-switches from Flat to HNSW as the collection grows. The HFresh index (v1.36+) keeps memory usage low even for billion-scale datasets.
  • Unified query interface. GraphQL enables nested queries that combine vector search, BM25, filtering, cross-references, and generative search in a single request. The application makes one network call instead of five.
  • Horizontal scaling with Raft. Sharding distributes data across nodes. Replication provides high availability. Raft consensus coordinates schema changes. Shard Replica Movement (v1.32+) enables dynamic load balancing without downtime.

What we sacrificed:

  • Memory footprint. HNSW indexes keep the full graph in RAM. At 1M vectors (1536 dimensions), this is approximately 6-8 GB of RAM. The HFresh index reduces this, but at the cost of some query latency. Qdrant’s quantization options (4-32x compression) are more aggressive than Weaviate’s.
  • GraphQL learning curve. GraphQL is expressive but unfamiliar to developers who know SQL. Writing complex queries requires understanding GraphQL’s query structure, fragments, and variable binding. pgvector uses SQL, which every developer already knows.
  • Embedding model lock-in. The vectorizer module is configured at the class level. Changing the embedding model requires re-vectorizing every object in the class. This is a schema migration, not a configuration change. pgvector stores raw vectors, so changing models is a data migration, not a schema change.
  • No SQL interface. Weaviate does not support SQL. If your team’s expertise is SQL and your data model is relational, pgvector is a better fit. Weaviate’s GraphQL + REST interface is powerful but different.
  • Operational complexity. Running a Weaviate cluster requires managing Go binaries, Raft consensus, gossip protocols, and shard rebalancing. pgvector runs inside PostgreSQL — zero new infrastructure. For teams without dedicated infrastructure engineers, Weaviate Cloud (managed) is the recommended path.
  • Import throughput. Weaviate’s HNSW construction is single-threaded per shard. Large imports (millions of objects) can take hours. Asynchronous indexing (v1.25+) helps by decoupling object store writes from vector index updates, but the HNSW construction itself remains a bottleneck.

The real lesson: Weaviate trades operational simplicity and memory efficiency for AI-native features and architectural coherence. If you want a vector database that manages the full AI data lifecycle — embedding, indexing, searching, generating — Weaviate is the best choice. If you want a vector store that fits into your existing PostgreSQL infrastructure, use pgvector. If you want the fastest pure vector search with the smallest memory footprint, use Qdrant. The right choice depends on whether you are building an AI application or adding vector search to an existing one.

Course-Style Deep Dive

How the HNSW Index Works Under the Hood

Weaviate’s default vector index is a custom implementation of Hierarchical Navigable Small World (HNSW). Here is how it works, step by step:

  1. Multi-layer graph construction. HNSW builds a hierarchy of proximity graphs. The bottom layer (layer 0) contains every vector. Each higher layer contains a subset of vectors, selected with exponentially decreasing probability. The top layer contains only a few vectors.

  2. Greedy search with layer descent. A query starts at the top layer. At each layer, it performs a greedy search: start at an entry point, evaluate all neighbors, move to the closest neighbor, repeat until no closer neighbor is found. Then descend to the next layer and repeat. The bottom layer search returns the final k nearest neighbors.

  3. Insertion with ef_construction. When a new vector is inserted, HNSW finds its approximate nearest neighbors using ef_construction candidates. It connects the new vector to its max_connections closest neighbors. These connections are bidirectional — the new vector is also added to the neighbor’s connection list.

  4. Search with ef. During search, the ef parameter controls the size of the dynamic candidate list. A larger ef explores more candidates, improving recall at the cost of latency. Weaviate’s dynamic_ef feature automatically adjusts ef based on the query’s difficulty.

Weaviate’s implementation includes several optimizations over the standard HNSW algorithm:

  • Vector cache. Frequently accessed vectors are cached in RAM, reducing disk I/O. The cache is configurable per class.
  • HNSW snapshots (v1.31+). The HNSW graph is periodically snapshotted to disk. On restart, Weaviate loads the snapshot instead of rebuilding the graph from scratch. This reduces startup time from hours to seconds for large indexes. Enabled by default since v1.36.
  • Asynchronous indexing (v1.25+). Vector index updates can be queued and processed asynchronously. The object store is updated immediately; the vector index catches up in the background. This improves write throughput during bulk imports.

Advanced Pattern 1: Multi-Tenancy with Per-Tenant Sharding

Weaviate supports multi-tenancy at the class level. Each tenant gets its own shard, isolated from other tenants:

# Create a multi-tenant class
articles = client.collections.create(
    name="Article",
    multi_tenancy_config=wvc.config.Configure.multi_tenancy(enabled=True),
    vectorizer_config=Configure.Vectorizer.text2vec_openai(),
    properties=[
        Property(name="title", data_type=DataType.TEXT, vectorize_property_name=False),
        Property(name="content", data_type=DataType.TEXT, vectorize_property_name=True),
    ],
)

# Add tenants (each tenant gets an isolated shard)
articles.tenants.create([
    "tenant-acme-corp",
    "tenant-globex-inc",
    "tenant-initech",
])

# Write data for a specific tenant
with client.with_tenant("tenant-acme-corp") as acme_client:
    acme_articles = acme_client.collections.get("Article")
    acme_articles.data.insert(DataObject(
        properties={"title": "Acme's Q2 Report", "content": "..."},
    ))

# Query within a tenant — data from other tenants is invisible
with client.with_tenant("tenant-acme-corp") as acme_client:
    acme_articles = acme_client.collections.get("Article")
    results = acme_articles.query.hybrid(query="Q2 performance", alpha=0.5)

The key insight: each tenant’s shard is a completely isolated storage unit. There is no cross-tenant data access, even at the storage layer. This is architecturally superior to filter-based multi-tenancy (where all tenants share the same index and are separated by a metadata filter) because it eliminates the risk of data leakage and enables per-tenant resource allocation.

Advanced Pattern 2: Custom Vectorizer Module

If the built-in vectorizer modules do not fit your needs, you can configure Weaviate to use any OpenAI-compatible embedding endpoint:

# Use a custom embedding service (e.g., self-hosted Hugging Face model)
client.collections.create(
    name="CustomEmbedded",
    vectorizer_config=Configure.Vectorizer.custom(
        # Any endpoint that implements the OpenAI embedding API
        url="http://localhost:8000/embeddings",
        # Or use a text2vec module with a custom model
    ),
)

For complete control, you can disable auto-vectorization and provide vectors explicitly:

# Disable auto-vectorization — application provides vectors
client.collections.create(
    name="ManualVectors",
    vectorizer_config=Configure.Vectorizer.none(),
    properties=[
        Property(name="content", data_type=DataType.TEXT),
    ],
)

# Insert with explicit vectors
client.collections.get("ManualVectors").data.insert(
    DataObject(
        properties={"content": "Custom embedded document"},
        vector=[0.1, 0.2, 0.3, ...],  # 1536-dimensional vector
    )
)

Advanced Pattern 3: Cross-Reference Traversal

Weaviate supports cross-references between classes, enabling graph-like queries. You define a ReferenceProperty on one class pointing to another, then query with return_references to traverse the relationship in the same query as the vector search. This enables queries like “find documents similar to this concept, written by authors in this department” — all in one GraphQL request.

Production Considerations

Memory planning. HNSW memory usage is approximately:

  • 1M vectors at 1536 dimensions (float32): ~6-8 GB RAM
  • 10M vectors at 1536 dimensions: ~60-80 GB RAM
  • 100M vectors at 1536 dimensions: ~600-800 GB RAM (use HFresh instead)

Use the HFresh index (v1.36+) for datasets over 10M vectors. It keeps only a compressed centroid index in RAM while storing posting lists on disk.

Import performance. For bulk imports:

  • Use the gRPC API (port 50051) instead of REST for 2-5x faster throughput.
  • Enable asynchronous indexing to decouple writes from vector index updates.
  • Batch inserts with insert_many() (up to 1,000 objects per batch).
  • Disable replication during import, then enable it afterward.
# Bulk import with gRPC and async indexing
client = weaviate.connect_to_local(port=8080, grpc_port=50051)
client.collections.create(
    name="LargeImport",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(),
    vector_index_config=Configure.VectorIndex.hnsw(
        ef_construction=128, max_connections=64, indexing_type="async",
    ),
)
for i in range(0, len(documents), 500):
    batch = documents[i:i+500]
    client.collections.get("LargeImport").data.insert_many([
        DataObject(properties={"content": doc}) for doc in batch
    ])

Query performance tuning.

  • Set dynamic_ef_min and dynamic_ef_max on HNSW indexes to let Weaviate adapt search width per query.
  • Monitor weaviate_vector_index_cache_hit_ratio — if it drops below 0.8, increase the vector cache size or switch to HFresh.
  • For high-QPS workloads, add read replicas and distribute queries across them.

Backup and recovery.

  • Weaviate supports periodic backups via the /v1/backups API.
  • Backups include the object store, inverted index, and vector index snapshots.
  • For disaster recovery, back up the PERSISTENCE_DATA_PATH directory and the schema definition.

The Results

Metric Before Weaviate After Weaviate Improvement
RAG pipeline setup time 2-3 days (stitching embedding + vector + LLM services) 5 minutes (Docker) 500-800x faster
Search quality (BEIR FIQA) 0.284 (pure vector) 0.428 (hybrid alpha=0.5) +51% nDCG
Search quality (BEIR NFCorpus) 0.224 (pure vector) 0.280 (hybrid alpha=0.5) +25% nDCG
BM25 query latency (12M docs) 91ms (v1.29) 27ms (v1.30 BlockMax WAND) 3.4x faster
End-to-end hybrid latency (12M docs) ~142ms (v1.29) ~78ms (v1.30) 1.8x faster
Application code for embedding ~200 lines (API calls, retries, caching) 0 lines (auto-vectorization) Eliminated entirely
Memory for 10M vectors (1536d) ~60-80 GB (HNSW) ~5-10 GB (HFresh, v1.36+) 6-12x reduction
Startup time for large index Hours (rebuild HNSW from scratch) Seconds (HNSW snapshots, v1.31+) 1000x+ faster
Query latency at 1M vectors — <10ms (HNSW) Production-ready
Multi-tenancy isolation Filter-based (risk of data leakage) Per-tenant shards (full isolation) Eliminated data leakage risk

What this means for you: Weaviate is not just a vector store with a search API bolted on. It is a database designed for AI workloads, and the numbers reflect that. The 500-800x reduction in setup time comes from eliminating the service-stitching problem. The 25-51% improvement in search quality comes from native hybrid search. The 3.4x latency improvement in v1.30 comes from BlockMax WAND optimization. And the 6-12x memory reduction with HFresh makes billion-scale vector search practical on modest hardware.

What to Watch Out For

  1. GraphQL is not SQL. Weaviate’s primary query interface is GraphQL. If your team has never used GraphQL, budget 2-3 days for the learning curve. The REST API covers CRUD operations, but search queries require GraphQL. Start with the Python client library (which abstracts GraphQL) and graduate to raw GraphQL when you need custom queries.

  2. Embedding model changes are schema migrations. The vectorizer module is configured at the class level. Changing the embedding model requires creating a new class, re-vectorizing all data, and migrating queries. Choose your embedding model carefully at the start. Test with a small dataset before committing to a model for production.

  3. HNSW memory is predictable but expensive. At 1M vectors (1536 dimensions), expect 6-8 GB of RAM for the HNSW index. At 10M vectors, expect 60-80 GB. If your memory budget is tight, use the HFresh index (v1.36+) or the Flat index for small collections. Monitor weaviate_vector_index_cache_hit_ratio and adjust the vector cache size accordingly.

  4. Import throughput is single-threaded per shard. HNSW construction is single-threaded. For large imports, add more shards (more parallelism) or enable asynchronous indexing. The gRPC API is 2-5x faster than REST for bulk operations. Batch inserts with insert_many() rather than individual insert() calls.

  5. Alpha tuning is per-query, not per-collection. The alpha parameter in hybrid search controls the balance between vector and BM25. There is no one-size-fits-all value. A query for product codes needs alpha=0.2 (BM25-heavy). A query for conceptual questions needs alpha=0.8 (vector-heavy). Build a query classifier that routes queries to the right alpha value, or use A/B testing to find the optimal alpha for your domain.

  6. Backup the schema separately. Weaviate’s backup API covers data, but the schema definition is managed separately. Version-control your schema definitions (the Python class creation code) alongside your application code. A schema migration is a code change, not a data change.

  7. Monitor disk usage. Weaviate has configurable disk usage thresholds (DISK_USE_WARNING_PERCENTAGE, DISK_USE_READONLY_PERCENTAGE). When the readonly threshold is reached, Weaviate stops accepting writes. Set up alerts at 70% disk usage to avoid surprise readonly mode during peak traffic.

Lesson 1: “We spent three months building a RAG pipeline with separate embedding, vector store, and LLM services. We spent three days rebuilding it with Weaviate. The first approach had five failure modes. The second had one.” — Senior ML Engineer, Fortune 500 company

Lesson 2: “Hybrid search is not optional. We launched with pure vector search and our users complained about missing exact matches. We switched to hybrid search with alpha=0.5 and the complaint rate dropped by 80%. The 2x latency cost was worth it.” — Search Engineer, e-commerce platform

Lesson 3: “The GraphQL learning curve is real. Our team of SQL veterans spent a week writing queries that would have taken an hour in SQL. But once we learned it, the expressiveness was worth the investment. One GraphQL query replaced five REST calls.” — Backend Engineer, SaaS company

Advice for Getting Started

  1. Start with Docker on your local machine. Run docker run semitechnologies/weaviate:latest and verify it works with curl http://localhost:8080/v1/meta. This takes 2 minutes and confirms your setup is correct.

  2. Use the Python client library (pip install weaviate-client). It abstracts GraphQL and provides a clean Pythonic API. Graduate to raw GraphQL when you need custom queries that the client library does not support.

  3. Start with the HNSW index (default). It is the most performant for most workloads. Switch to Flat for collections under 10K vectors, Dynamic for collections that grow over time, and HFresh for collections over 10M vectors.

  4. Configure auto-vectorization from day one. Even if you plan to provide vectors manually later, start with auto-vectorization. It eliminates the embedding pipeline and lets you focus on search quality. You can switch to manual vectors later if needed.

  5. Use hybrid search with alpha=0.5 as your default. Tune alpha per query based on the query type. Build a simple query classifier if you have diverse query types.

  6. Monitor the Prometheus metrics endpoint (/metrics). Key metrics to watch: vector cache hit ratio, HNSW query latency, BM25 query latency, and disk usage. Set up alerts for the disk usage thresholds.

  7. Version-control your schema definitions. The class creation code is a schema migration. Treat it like a database migration — test it, review it, and version it.

Next in the Open-Source AI Tools Mastery series: pgvector

NL

Written by Nivant Labs Team

Engineer at Nivant Labs

Share this post

XLinkedInHN