O’Reilly Media · 2026

Hands-On RAG
for Production

Design, develop, and deploy production-ready RAG applications.

Retrieval-augmented generation is easy to prototype and hard to ship. This book is about closing that gap — turning a working demo into a system that holds up under real traffic, real data, and real users.

Hands-On RAG for Production — book cover

Praise

What practitioners are saying

The first time I saw a RAG system fail in production, it was because someone had naively chunked their documents on fixed character boundaries and split a legal clause in half. The system retrieved the first chunk but not the second, so the model answered with the opposite of what the contract said.
Sharon Zhou, PhDFrom the foreword · Corporate VP of Engineering & AI, AMD
Hands-On RAG for Production doesn’t skip the unglamorous parts. Ofer and Forrest give document parsing, tables, and ingestion the serious treatment they deserve — which is exactly where most real-world RAG systems live or die.
Jerry LiuCEO, LlamaIndex
We are entering an era where software is no longer just a tool we use, but an intelligence we collaborate with. This book correctly identifies that the future of the ‘Corporate Brain’ relies on the unification of fragmented institutional knowledge through robust RAG pipelines. For teams building these living, breathing engines of insight, this book serves as the definitive guide to mastering the complex interplay between vector stores, agents, and evaluators.
Bob van LuijtCofounder and CEO, Weaviate
RAG has quietly become the default architecture for grounding LLMs in enterprise data, but the engineering discipline around it is still catching up. Mendelevitch and Bao lay out a rigorous, end-to-end framework — from ingestion and retrieval design through production evaluation and agentic extensions — that most teams are still figuring out by trial and error.
Jimmy LinACM and ACL Fellow; Chair Professor, University of Waterloo
If you are a developer, architect, or a leader tasked with bringing LLMs into the real world, this is your manual.
Jim DowlingFrom the foreword · CEO of Hopsworks
While anyone can hack together a toy RAG demo in an afternoon, the gap between a prototype and a production-grade system is vast. Hands-On RAG for Production masterfully explains the intricacies of achieving accuracy, low latency, and scale, while providing the crucial ‘build vs. buy’ framework every architect needs. If you’re moving beyond the sandbox, this is a must read.
Tallat M. ShafaatFounder & CEO, Vectara
Hands-On RAG for Production makes the leap from vector search to GraphRAG feel practical, intuitive, and immediately useful. The Knowledge Graph chapter gives readers the foundation they need to build more precise, explainable, and trustworthy AI systems.
Stephen ChinAuthor of GraphRAG: The Definitive Guide; VP of Developer Relations, Neo4j
Hands-On RAG for Production is the first book I’ve seen that treats RAG, agents, and evaluation with true production rigor. Every ML or platform engineer shipping LLM applications at scale should read this.
Shailja GuptaProduct Manager, AI Platforms
If you’ve built a demo RAG app and now need reliability, metrics, and tooling that scale, this book closes the gap with concrete patterns and hands-on examples.
Diva DugarSoftware Engineer

The book

From a RAG demo to a system you can run in production

The market is full of RAG pipelines, frameworks, and vector databases, which makes it hard to know what to actually use. This book walks through every phase of building a RAG application — data ingestion, embeddings, and vector search through to advanced techniques like agentic RAG, multimodal RAG, and GraphRAG — and is honest about the trade-offs at each step.

The focus is production: managing hallucinations, keeping latency and cost in check, securing your data, and producing answers you can explain. It also helps you make the call that matters most — whether to build a RAG system yourself or stand on a managed platform.

Build the base stack

Parsing, chunking, embeddings, vector search, and generation — the components every RAG system is made of.

Scale retrieval quality

Reranking, guardrails, and hallucination detection to keep answers grounded as the corpus grows.

Ship to production

Cut latency and cost with caching, protect sensitive data with redaction, and deploy with confidence.

Evaluate honestly

LLM-as-a-judge and the metrics that actually tell you whether your system is getting better.

Build vs. platform

A clear-eyed look at when a managed RAG platform earns its keep and when to roll your own.

Go beyond text

Agentic RAG, multimodal RAG over images and audio, and knowledge-graph RAG for harder questions.

Inside

Table of contents

Every chapter but the last ships with runnable notebooks. Follow a chapter straight through to its code.

  1. 01Introduction to Retrieval-Augmented Generation (RAG)1 notebook →
  2. 02The Base RAG Stack7 notebooks →
  3. 03Scaling Your RAG Stack5 notebooks →
  4. 04Deploying RAG to Production2 notebooks →
  5. 05The RAG Platform3 notebooks →
  6. 06Evaluating Your RAG Application3 notebooks →
  7. 07From RAG to AI Agents5 notebooks →
  8. 08Multimodal RAG5 notebooks →
  9. 09Knowledge-Enhanced RAG4 notebooks →
  10. 10The Future of RAGNo code

Run it yourself

Every example, as a notebook

The book ships with a companion repository of Jupyter notebooks — one set per chapter, runnable locally or in Google Colab. They use LangChain, LlamaIndex, and CrewAI alongside pgvector, SQLite-vec, Neo4j, and models from OpenAI and Anthropic, so you can follow along and adapt the code to your own data.

View on GitHub →

Who wrote it

About the authors

OM

Ofer Mendelevitch

Ofer Mendelevitch is an AI and ML leader specializing in building production systems with large language models (LLMs), retrieval-augmented generation (RAG), and agentic workflows. He is the author of Practical Data Science with Hadoop (Addison-Wesley).

FB

Forrest Sheng Bao

Forrest Sheng Bao is a cofounder of PAIR Systems, Inc. (also known as pairsys.ai). Previously he was a cohead of machine learning at Vectara, where he was a major contributor to the HHEM 2 hallucination detector for LLMs. Prior to Vectara, he was an assistant professor at Iowa State University, where he won an NSF CAREER Award for research on LLM evaluation. Forrest holds a PhD in computer science with a minor in electrical engineering from Texas Tech University.

Get the book

Pick your store

Available in print and digital. The full text is on O’Reilly; print copies ship from the retailers below.

ISBN 979-8-3416-2171-8 · O’Reilly Media