adaptive-rag-engine

Adaptive RAG Engine

---

Overview

Adaptive RAG Engine is a modular Retrieval-Augmented Generation framework designed to improve retrieval quality through adaptive query routing, hybrid retrieval, reciprocal rank fusion (RRF), cross-encoder reranking, confidence estimation, and optimized context construction before passing relevant information to a Large Language Model (LLM).

Unlike conventional RAG systems that rely on a single retrieval strategy, Adaptive RAG dynamically selects Dense, Sparse, or Hybrid Retrieval based on query analysis to improve retrieval precision while maintaining low latency.

---

Key Features

• Adaptive Query Analysis
• Intelligent Query Routing
• Dense Retrieval (SentenceTransformers)
• Sparse Retrieval (BM25)
• Hybrid Retrieval
• Reciprocal Rank Fusion (RRF)
• Cross-Encoder Re-ranking
• Confidence Estimation
• Context Optimization
• LangChain Retriever
• FastAPI REST API
• FAISS Vector Database
• Google Gemini Integration
• Modular Production Architecture
• Docker Ready
• PyPI Ready

---

System Architecture

flowchart TD

A[User Query]

A --> B[FastAPI REST API]

B --> C[Adaptive RAG Pipeline]

C --> D[Query Analyzer]

D --> E[Adaptive Query Router]

E -->|Semantic Query| F[Dense Retriever]

E -->|Keyword Query| G[BM25 Retriever]

E -->|Mixed Query| H[Hybrid Retrieval]

F --> I[SentenceTransformer Embeddings]

I --> J[FAISS Vector Store]

G --> K[BM25 Index]

J --> H

K --> H

H --> L[Reciprocal Rank Fusion]

L --> M[Cross Encoder Reranker]

M --> N[Confidence Estimator]

N --> O[Context Optimizer]

O --> P[LangChain Retriever Adapter]

P --> Q[Prompt Builder]

Q --> R[Gemini LLM]

R --> S[Generated Answer]

S --> T[FastAPI Response]

Retrieval Pipeline

User Query

↓

FastAPI REST API

↓

Query Analyzer

↓

Adaptive Query Router

↓

Dense Retriever (MiniLM)

BM25 Retriever

↓

Hybrid Retriever

↓

RRF Fusion

↓

Cross Encoder Reranker

↓

Confidence Estimator

↓

Context Optimizer

↓

LangChain Retriever

↓

Prompt Builder

↓

Gemini

↓

Generated Response

---

Technology Stack

---

Project Structure

app/
src/
tests/
docs/
data/
logs/

---

Installation

git clone ...
cd adaptive-rag-engine

pip install -r requirements.txt

---

Environment Variables

GEMINI_API_KEY=

GEMINI_MODEL=

HF_TOKEN=

---

Running

uvicorn app.main:app --reload

---

API

POST /rag/ask

Example

{
  "question":"Explain Retrieval Augmented Generation."
}

---

Docker

docker build -t adaptive-rag-engine .

docker run -p 8000:8000 adaptive-rag-engine

---

PyPI

pip install adaptive-rag-engine

---

Future Scope

• Redis Cache
• Streaming Responses
• Multi-modal Retrieval
• Kubernetes Deployment
• Multi-Agent RAG
• GraphRAG
• Knowledge Graph Integration

---

License

MIT License