Hybrid RAG Retrieval Pipeline

Overview

RAG (Retrieval-Augmented Generation) systems are only as good as their retrieval layer. This project focused on building a retrieval pipeline that goes beyond simple vector similarity to incorporate multiple complementary search strategies.

The Three Retrievers

Dense Vector Search

Documents are chunked (512 tokens, 50 token overlap) and embedded using OpenAI’s embedding model. Stored in ChromaDB for efficient approximate nearest neighbor search. Good for semantic similarity but misses exact terminology matches.

Sparse Keyword Search (BM25)

Classic term-frequency based retrieval running in parallel with vector search. Catches queries where specific terms matter — product codes, supplier names, technical specifications — that embedding models may not distinguish well.

Knowledge Graph Traversal

At ingestion time, an LLM extracts entities (suppliers, products, regions, facilities) and relationships from each document. These are stored in a NetworkX graph. At query time, entities are extracted from the query and the graph is traversed (up to 2 hops) to find related context.

Fusion Strategy

def reciprocal_rank_fusion(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc in enumerate(results):
            doc_id = doc.id
            if doc_id not in scores:
                scores[doc_id] = 0
            scores[doc_id] += 1 / (rank + k)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

Reciprocal rank fusion merges results without requiring score normalization across different retrieval methods. The k parameter controls how much weight is given to top-ranked results.

Evaluation

Retriever	Relevance@5	Relevance@10
Vector only	67%	72%
Vector + BM25	74%	79%
Vector + BM25 + KG	86%	89%

The knowledge graph layer provided the largest single improvement, particularly for queries requiring information that spans multiple documents.