GitHub - EmilResearch/RAGless: RAGless is a semantic retrieval system that answers questions about your documentation, without using an LLM at runtime.

RAGless — Deterministic Retrieval-Only Q&A System

RAGless is a retrieval-only question-answering system with zero LLM calls at runtime. Source documents are converted into self-contained informational blocks, indexed into a local vector database (Qdrant), and queried via asymmetric Gemini embeddings.

Zero hallucinations at runtime. Minimal latency. Near-zero cost per query.

See it in action

Overview

The project consists of three independent scripts:

Key Features

• No LLM at runtime — The chatbot relies purely on vector retrieval. No expensive API calls during user interaction.
• Robust Q-Q matching via answer_id aggregation — If multiple question variants for the same answer match the query, their scores are summed. This makes the result far more stable than classic "top-1" retrieval.
• Asymmetric Gemini embeddings — RETRIEVAL_DOCUMENT at ingestion time and RETRIEVAL_QUERY at retrieval time, as recommended by Google.
• Embedded Qdrant — No Docker server, no cloud service. Data is stored locally on disk (./qdrant_data).
• Smart chunking — Documents are chunked only if they exceed a token threshold, measured with the model's real tokenizer.
• Optional Judge verification — prepare_data.py can enable a second LLM pass to discard blocks not supported by the source text (--judge).
• Missed query logging — Below-threshold queries are automatically logged to missed_queries.log for later analysis.
• Guaranteed idempotency — Every run of ingest_to_qdrant.py recreates the collection from scratch, preventing hidden duplicates.

Requirements

• Python 3.10+
• Gemini API Key (free tier with generous limits)
• Python dependencies (see Installation section)

Installation

1. Clone or download the repository and navigate to the project folder.

2. Create a virtual environment (recommended):

   python -m venv venv
   source venv/bin/activate  # Linux/macOS
   # or
   venv\Scripts\activate   # Windows

3. Install dependencies:

   pip install litellm qdrant-client pypdf python-dotenv tqdm

4. Configure your API key:

   cp .env.example .env
   # Edit .env and insert your GEMINI_API_KEY

Project Structure

.
├── source/                    # Folder with source documents (.pdf, .txt, .md)
├── config.py                  # Centralized configuration (models, thresholds, paths)
├── prepare_data.py            # Script 1: Q&A block extraction
├── ingest_to_qdrant.py        # Script 2: embedding and indexing
├── chatbot.py                 # Script 3: CLI chatbot
├── data.json                  # Output of prepare_data.py (validated blocks)
├── qdrant_data/               # Local vector database (auto-created)
├── failed_chunks/             # Chunks that failed to produce valid JSON (debug)
└── missed_queries.log         # Log of below-threshold queries

Usage

1. Prepare the data (prepare_data.py)

Place your documents in the source/ folder (supports .pdf, .txt, .md), then run:

With optional Judge verification (slower but more accurate):

python prepare_data.py --judge

What it does:

• Reads each file and counts tokens.
• If the document is short (≤ 10,000 tokens), sends it whole to the LLM; otherwise splits it into chunks.
• Extracts JSON blocks with answer, questions, category, source_quote.
• Validates blocks and saves them to data.json.

2. Index into the vector database (ingest_to_qdrant.py)

python ingest_to_qdrant.py

What it does:

• Loads data.json.
• "Explodes" each block into as many rows as its question variants.
• Generates embeddings in batches via LiteLLM + Gemini.
• Recreates the Qdrant collection and inserts vectors with deterministic UUID5s.

3. Launch the chatbot (chatbot.py)

Available options:

python chatbot.py --threshold 0.75   # Change the minimum aggregated score threshold
python chatbot.py --debug            # Show internal scores and aggregation table

Interaction:

You> How does check-in work?
[INFO] Found 2 relevant answers, showing top 1:
──────────────────────────────────────────────────────────────────────
--- Answer 1 (Pertinence: 1.85) ---
Check-in is available from 3:00 PM to 8:00 PM. If you arrive after 8:00 PM,
please contact reception in advance...
──────────────────────────────────────────────────────────────────────
Source: source/regulations.txt

Type exit, quit, or :q to leave.

How Retrieval Works

The core of the chatbot is aggregation by answer_id:

1. The user query is embedded with task_type=RETRIEVAL_QUERY.
2. Qdrant returns the TOP_K_RETRIEVAL most similar points (questions).
3. Scores of points pointing to the same answer_id are summed.
4. A candidate is shown only if:
   • the aggregated score exceeds DEFAULT_THRESHOLD OR
   • the best single hit exceeds SINGLE_HIT_THRESHOLD
   • AND the best single hit is > 0.68 (minimum quality)

This mechanism makes the system robust: even if no single question variant is a perfect match, the sum of multiple weak matches on the same answer can make it emerge correctly.

RAGless vs. Classic Generative RAG

Philosophy

Classic RAG generates answers at runtime by retrieving context and prompting an LLM to synthesize a response. RAGless eliminates the generative step entirely. Answers are pre-generated during ingestion and retrieved verbatim at runtime.

Advantages of RAGless

Trade-offs and Limitations

Cost & Performance Comparison

The Hallucination Trade-off

RAGless eliminates hallucinations at runtime, where they are most dangerous because they are uncontrollable. The risk during ingestion still exists, but it is mitigated — and crucially — it happens offline, in a controlled environment, with the possibility of human review before the knowledge base goes to production.

RAGless shifts the hallucination risk from runtime to ingestion. Generation happens during prepare_data.py, which is why the optional --judge pass and deterministic UUIDs for idempotency were added.

The trade-off is: you give up real-time flexibility in exchange for offline verifiability. For high-risk domains, I prefer hallucinations I can catch in a log over ones I cannot predict.

Advanced Configuration

All tunable constants are in config.py:

Troubleshooting

Technical Notes

• LiteLLM is used as a unified proxy to call Gemini for both completions and embeddings.
• Qdrant Client in path= mode stores everything in local files: no server process is required.
• Qdrant point IDs are deterministic UUID5s (uuid5(NAMESPACE_DNS, answer_id:question_text)), so re-running ingestion does not create logical duplicates.

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPLv3).

See LICENSE for details.