Retrieval Augmented Generation (RAG)¶

This work is licensed under a Creative Commons Attribution 4.0 International License.

What is RAG?¶

Retrieval-Augmented Generation (RAG) is a technique that enhances AI language models by giving them access to external knowledge sources. Instead of relying solely on what a model learned during training, RAG allows AI to retrieve relevant information from your own documents, databases, or knowledge bases and use that information to generate more accurate, up-to-date, and contextually relevant responses.

RAG vs. Traditional AI Chat¶

Real-World Examples of RAG¶

You may already be using RAG without realizing it:

• NotebookLM: Google's research tool uses RAG to answer questions based only on your uploaded sources
• ChatGPT with file uploads: When you upload PDFs, ChatGPT retrieves relevant sections to answer your questions
• Claude Projects: Custom knowledge bases that Claude references during conversations
• OpenWebUI: Self-hosted interface with Knowledge collections for local, private RAG
• Enterprise search tools: Company wikis and knowledge bases that use AI to find and synthesize information
• Legal research platforms: AI tools that search case law and statutes to answer legal questions

How RAG Works¶

RAG operates through a pipeline that connects your documents to the AI model. Understanding this pipeline helps you use RAG tools more effectively and troubleshoot when responses aren't what you expect.

The RAG Pipeline¶

flowchart LR
    A[Your Documents] --> B[Chunking]
    B --> C[Embedding]
    C --> D[(Vector Database)]
    E[User Query] --> F[Query Embedding]
    F --> G[Similarity Search]
    D --> G
    G --> H[Retrieved Context]
    H --> I[LLM Generation]
    E --> I
    I --> J[Response with Citations]

    style A fill:#e1f5ff
    style D fill:#fff3e0
    style J fill:#c8e6c9

Step-by-step explanation:

1. Document Ingestion: Your documents (PDFs, web pages, databases) are loaded into the system
2. Chunking: Documents are split into smaller, manageable pieces (chunks)
3. Embedding: Each chunk is converted into a numerical representation (vector) that captures its meaning
4. Storage: Vectors are stored in a specialized database optimized for similarity search
5. Query Processing: When you ask a question, your query is also converted to a vector
6. Retrieval: The system finds chunks whose vectors are most similar to your query
7. Generation: The LLM generates a response using both your question and the retrieved context
8. Citation: The response includes references to the source documents

Embeddings¶

Text passages (or other data, like images) are transformed into numerical representations known as "embeddings." These embeddings capture semantic meaning, so similar concepts end up being close together in vector space.

Vector Databases¶

A vector database stores embeddings efficiently, using specialized data structures to handle large-scale, high-dimensional searches. Unlike traditional databases that match exact values, vector databases find the most semantically similar items to your query.

Table: Popular RAG Vector Database Software

Using RAG Without Code¶

You do not need to be a programmer to benefit from RAG. Several platforms provide RAG capabilities through user-friendly interfaces.

Consumer-Friendly RAG Tools¶

RAG with OpenWebUI¶

OpenWebUI is an open-source, self-hosted web interface for running AI models locally. It provides built-in RAG capabilities that allow you to chat with your own documents while keeping all data on your own computer or server---ideal for sensitive research data or when you want complete control over your AI infrastructure.

How RAG Works in OpenWebUI¶

OpenWebUI offers two complementary ways to use RAG:

1. Quick Document Chat: Upload files directly in a conversation using the + button or drag-and-drop
2. Knowledge Collections: Create reusable document libraries that persist across conversations

flowchart LR
    A[Your Documents] --> B{Upload Method}
    B -->|Single Chat| C[Direct Upload]
    B -->|Persistent| D[Knowledge Collection]
    C --> E[# Reference in Chat]
    D --> E
    E --> F[RAG-Enhanced Response]

    style A fill:#e1f5ff
    style D fill:#fff3e0
    style F fill:#c8e6c9

Step-by-Step: Creating a Knowledge Collection¶

Knowledge collections are the recommended approach for documents you will reference repeatedly, such as course materials, research literature, or project documentation.

Step 1: Access the Knowledge Section

1. Log into your OpenWebUI instance
2. Navigate to Workspace in the left sidebar
3. Select Knowledge

Step 2: Create a New Knowledge Base

1. Click + Create Knowledge (or the + button)
2. Configure your knowledge base:
   • Name: Give it a descriptive name (e.g., "Climate Policy Papers 2024")
   • Description: Briefly describe the contents
   • Access: Choose Private for personal use or Public to share with others on your instance
3. Click Create

Step 3: Add Documents

1. Open your newly created knowledge base
2. Add documents by:
   • Drag and drop files directly into the window
   • Click Add Content and select files from your computer
   • Paste a URL to import web content
3. Supported formats include: PDF, TXT, Markdown, DOCX, CSV, and more
4. Wait for processing to complete (you will see a progress indicator)

Step 4: Use Your Knowledge in Conversations

1. Start a new chat
2. Type # to see available knowledge collections
3. Select your knowledge base from the dropdown
4. Ask questions---OpenWebUI will retrieve relevant context from your documents

Quick Document Upload (Per-Conversation)¶

For one-time document queries, you can upload files directly into a conversation:

1. Click the + button in the chat input area
2. Select Upload Files (or drag files directly into the chat)
3. Choose your document(s)
4. Once processed, reference them with # followed by the filename
5. Ask your questions

Web Content in RAG¶

OpenWebUI can also retrieve content from web pages:

1. In the chat input, type # followed by a URL
2. Example: #https://www.nature.com/articles/s41586-024-00001-1
3. OpenWebUI will fetch, parse, and index the page content

OpenWebUI RAG Best Practices¶

Document Preparation

• Use text-based PDFs whenever possible (not scanned images)
• Descriptive filenames help you identify documents when using # references
• For long documents, consider whether the entire document is relevant or if excerpts would be better
• Markdown files often index more cleanly than complex PDFs

Chunking Configuration (Administrators)

If you manage an OpenWebUI instance, you can tune RAG performance in Admin Panel > Settings > Documents:

• Chunk Size: Larger chunks preserve more context but may dilute relevance
• Chunk Overlap: Higher overlap prevents important information from being split across chunks
• Markdown Header Splitting: Enable this for structured documents to keep sections together

Query Strategies

• Be specific: "What does Section 3.2 say about sampling methods?" works better than "Tell me about methods"
• Ask for citations: "Include the source for each claim"
• For complex questions, break them into parts and ask sequentially
• If results seem incomplete, try rephrasing---different wording retrieves different chunks

Comparison: OpenWebUI vs. Other RAG Tools¶

OpenAI's Vector Store and File Search¶

OpenAI Platform allows developers and advanced users to upload data to a vector store and enable file search in custom assistants. This provides more control over the RAG process compared to simple file uploads.

Building RAG Applications¶

For researchers and developers who want more control, building custom RAG applications enables:

• Processing large document collections
• Customizing retrieval strategies
• Integrating with existing systems
• Maintaining full data privacy

RAG Frameworks¶

Several frameworks simplify building RAG applications:

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import Chroma
from langchain.chains import RetrievalQA

# 1. Load your documents
loader = PyPDFLoader("research_paper.pdf")
documents = loader.load()

# 2. Split into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
)
chunks = text_splitter.split_documents(documents)

# 3. Create embeddings and store in vector database
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(chunks, embeddings)

# 4. Create a retrieval chain
llm = ChatOpenAI(model="gpt-4")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

# 5. Ask questions
response = qa_chain.invoke("What are the main findings?")
print(response)

RAG for Academic Research¶

RAG is particularly valuable for academic work where accuracy, citations, and working with large document collections are essential.

Use Cases in Research¶

Literature Reviews

• Upload dozens or hundreds of papers to a RAG system
• Ask synthesis questions: "How has methodology evolved in this field?"
• Identify gaps: "What aspects of this topic are under-researched?"
• Find contradictions: "Where do researchers disagree?"

Grant Writing

• Create a knowledge base of successful grant proposals
• Query for specific section examples: "Show me aims page structures"
• Synthesize background literature for significance sections

Data Analysis Support

• Upload codebooks, documentation, and methodology papers
• Ask questions about variable definitions and procedures
• Get help interpreting statistical results in context

Teaching Preparation

• Build a knowledge base of textbook chapters and supplementary materials
• Generate quiz questions based on specific content
• Create study guides that reference source materials

Best Practices for Academic RAG¶

Limitations and Considerations¶

RAG is powerful but not perfect. Understanding its limitations helps you use it appropriately.

Technical Limitations¶

• Chunk boundaries: Important information split across chunks may not be retrieved together
• Retrieval quality: The system may not always find the most relevant passages
• Context window limits: Very long retrieved passages may be truncated
• Embedding quality: Some specialized vocabulary may not embed well

Practical Considerations¶

• Garbage in, garbage out: Poor quality documents produce poor quality responses
• Not a replacement for reading: RAG helps you navigate documents, not avoid reading them
• Citation verification required: AI may misattribute or misquote sources
• Domain expertise still matters: You need expertise to evaluate response quality

Assessment Questions¶

Further Resources¶

Documentation and Guides¶

• LangChain RAG Tutorial
• LlamaIndex Getting Started
• Pinecone RAG Guide

Academic Papers¶

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks - The original RAG paper (Lewis et al., 2020)
• A Survey on Retrieval-Augmented Text Generation - Comprehensive survey of RAG techniques

Related Workshop Content¶

• NotebookLM - Consumer-friendly RAG for research
• Text Mining - Techniques for processing large text collections
• Code Execution - Setting up environments for custom RAG development
• Agentic AI - How RAG combines with autonomous AI agents