adv-rag-eval

Adversarial RAG Evaluator (adv-rag-eval)

A fully local, zero-cost, privacy-first Python library to evaluate Retrieval-Augmented Generation (RAG) pipelines for hallucinations.

Instead of paying for expensive cloud APIs (like GPT-4) to judge your AI outputs, adv-rag-eval downloads a proprietary, fine-tuned Llama-3-8B model directly to your machine. It runs completely offline using llama-cpp-python, ensuring your enterprise data never leaves your local environment.

Features

• 100% Local & Private: No API keys, no data sent to OpenAI or Anthropic.
• Edge-Optimized: Compressed to a 4.9GB GGUF file. Runs smoothly on consumer laptops (RTX 3050, Mac M-series, or even pure CPU).
• Auto-Provisioning: Automatically downloads and caches the inference engine from Hugging Face on the very first run.
• Surgical Detection: Specifically fine-tuned on synthetic adversarial datasets to catch the three most dangerous RAG failure modes.

---

Installation (No C++ Build Tools Required!)

Because this library relies on a highly optimized C++ inference engine, standard installation might throw compiler errors on Windows. To bypass this, install the pre-compiled engine for your specific hardware first:

For standard laptops (CPU Only):

pip install llama-cpp-python --no-cache-dir --only-binary llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
pip install adv-rag-eval

**For Windows/Linux with Nvidia GPUs (CUDA 12.2):**
```bash
pip install llama-cpp-python --no-cache-dir --only-binary llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu122
pip install adv-rag-eval

**For Mac (Metal / Apple Silicon):**
```bash
CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python
pip install adv-rag-eval