Small chunks give precise embeddings but lack context; big chunks have complete context but diluted embeddings. Hierarchical Chunking builds multi-level indexes (2048→512→128 tokens) with an Auto-Merge algorithm: leaf nodes match precisely, and when hit density exceeds a threshold the parent node is returned to the LLM instead. HiChunk shows a 12.7% evidence recall improvement; LlamaIndex and Haystack have it built in.
The first observation of 'What course articles do you have?' was contaminated by an old cache entry. A real cache miss retrieved all four university maps but spent 51.169 seconds across three Writer and Critic passes; after catalog-specific retrieval and review fixes, one uncached production observation passed q21 in 26.821 seconds.
Ask AI first extracts intent, complexity, and 1–4 search terms. It then routes across metadata, BM25, Vectorize, and RRF; a retry adds Critic gaps and disables the first-pass-only BM25 short circuit.
Ask AI splits one question across the UI, `/api/chat`, Planner, Research, Writer, Validation, Critic, and Related stages. Answer text, displayed sources, and related-reading cards come from separate paths with separate gates.
Ask AI keeps golden contracts, offline fixtures, live SSE output, and production observations separate. A passing fixture proves harness reproducibility; public sources can measure expected-source recall, but they do not expose hidden ranked chunks or establish model-graded faithfulness.
Ask AI finding a post does not mean the UI should display it as a source. An answer must pass deterministic Markdown and URL validation, then the Critic's relevance, intent, and grounding checks; if either gate fails, source cards are withheld.
Writer sees the first 8 candidates for a factual query or 12 for a recommendation by default. Citations must use an exact `source_url` from that set, and weak or empty retrieval triggers an instruction to abstain rather than fill gaps from model knowledge.
Agent Memory is a Cloudflare private beta service for letting agents remember users, teams, projects, and task context across conversations. It fits facts, events, instructions, and tasks; RAG documents, product data, files, and audit logs should still live in AI Search, Vectorize, D1, or R2.
Vectorize is Cloudflare's vector database. AI Search is the right starting point for a managed RAG pipeline; Vectorize is the better fit when you need control over chunking, embeddings, metadata filters, hybrid retrieval, reindexing, and fallback behavior.
Keenable.ai positions itself as search infrastructure for AI agents: a 100B+ document index, Search/Fetch APIs, MCP/CLI entry points, 100K free monthly requests, and keyless public endpoints. It is worth tracking, but the 100B+ index, latency, and quality claims are still mostly company-provided; NEEDLE is open, but needs external reruns and human review.
Same 'knowledge graph + retrieval' label, three different bets: Microsoft GraphRAG v3.1.2 pays indexing cost for global summarization, LightRAG cuts cost with dual-level retrieval and incremental updates, HippoRAG 2 turns RAG into growing associative memory via PPR — this guide splits the trade-offs by component with four query modes, indexing pipelines, and a selection matrix.
Anthropic Contextual Retrieval uses an LLM to prefix each chunk with 50-100 tokens, cutting failure rate from 5.7% to 1.9% with rerank at ~$1.02/1M tokens; Late Chunking encodes the full 32K-window document first then mean-pools by chunk boundaries for zero extra LLM cost — the trade-off is window, latency, and update shape.
Jina Reader turns a known URL into LLM-friendly Markdown; production use still requires explicit rendering, scope, token-budget, validation, and fallback decisions.
Parallel Web Systems separates Search, Extract, and Task APIs into web-access layers with different latency and cost profiles, while Basis maps citations, excerpts, and confidence to output fields.
The repository has a 20-query Traditional Chinese/English golden dataset, but no document-level qrels, retrieval runs, raw latency data, or executable benchmark script. Reporting Recall@k, MRR, or nDCG as measured results would therefore be dishonest; this article defines the contract needed to run them reproducibly.
R2R packages document ingestion, hybrid search, knowledge graphs, RAG, Agents, and access controls behind a REST API; it fits teams that already own their product frontend and backend and need a retrieval service.
Reliable citation is not appending URLs to an answer. Separate URLs, content copies, and source independence, then connect atomic claims to quote spans and snapshots through a rerunnable claim-source matrix.
Chapter 16 connects representation learning to systems: contrastive objectives shape an embedding space, semantic retrieval finds neighbors in it, and RAG passes retrieved context to a generator.
Units 13–14 connect models to external knowledge; A3 requires data collection, QA annotation, indexing, and ablations under CPU and latency constraints.
CS124 is the first course in Stanford's NLP branch. Its textbook is Jurafsky's own Speech and Language Processing, free online, and all nine assignment repos are public. But a banner sits on the course homepage: it will not be taught at all in AY 2026–27. And the chapter numbers the syllabus points at no longer match the August 2026 textbook.
A web retrieval benchmark must evaluate complete tasks, not HTTP 200s: 30 fixed cases across five failure strata and three live channels, measuring answers, citations, freshness, latency, cost, and unnecessary escalation. This article delivers the harness and gates, but no fabricated ranking while the three live channels remain unconfigured.
An agent should not open a browser for every web task: route first to Search or Fetch, then escalate on explicit signals such as status codes, weak content, JavaScript shells, authentication, or challenge pages, with retry, budget, cache, deduplication, and provenance constraints at every step.
Four certifications genuinely test RAG and retrieval evaluation: AWS AIF-C01 (chapters 2 and 3 total 52%, covering RAG, vector stores, and FM evaluation metrics), AWS AIP-C01 (11 of the 27 skill points in its 31% Domain 1 sit in vector storage and RAG), NVIDIA NCP-AAI (Knowledge Integration 10% plus Evaluation and Tuning 13%), and Microsoft AI-500 ('multi-agent RAG architecture' inside its 30–35% Develop area). Google PMLE contributes exactly one LLM-as-a-judge objective, and Claude CCDV-F — the developer certification people most readily assume covers RAG — has no retrieval objective across its eight domains, with Eval at just 2.6%. Includes a same-vendor foundational-vs-professional comparison, a four-vendor terminology map, non-transferable objectives, and a practice project.
Standard RAG gives a wrong answer when it retrieves the wrong chunk, and nothing in the system will notice. Agentic RAG adds a self-check, at the cost of the evaluator paradox: the ceiling on self-correction is whatever the evaluating LLM can judge about relevance.
Traditional RAG is a fixed pipeline of 'retrieve then answer.' Agentic RAG splits retrieval into three decision layers: when to retrieve (FLARE uses token probabilities; Adaptive-RAG uses a complexity classifier), what to retrieve (HyDE / RAG-Fusion / decomposition / Step-back), and how to fuse (RRF k=60 then cross-encoder rerank then compression -- Anthropic measured a -67% failure rate reduction). Key counter-intuitive insight: unnecessary retrieval hurts quality -- 'deciding not to retrieve' is a first-class capability.
Cosine similarity and relevance systematically diverge across an entire class of scenarios: negation (most IR models score at or below random on NevIR), exact identifiers, numeric thresholds, and logical combinations (SoTA models achieve recall@100 < 20 on LIMIT) -- some of these hit the theoretical ceiling of the single-vector paradigm, and switching to a larger model will not help. Recommended remedy order: hybrid BM25 -> reranker (Anthropic measured -67%) -> upstream metadata routing -> domain fine-tuning / multi-vector.
Traditional Chinese RAG retrieval failures are a three-layer stack: embedding granularity defects (BGE/GTE from 0.1B to 7B all mis-rank on simple queries like 'fried chicken'), Simplified Chinese / English corpus dominance causing local vocabulary drift ('premium', 'exclusion clause' alignment is unreliable), and MTEB Chinese benchmarks being Simplified Chinese making model selection signals misleading. The fix is architectural: OpenCC normalization -> hybrid + jieba segmentation -> reranker -> local fine-tuning last -- and the prerequisite for all of it is building a Traditional Chinese eval set first.
PageIndex skips chunking, embedding, and vector storage entirely. Instead it relies on LLM reasoning over a tree-structured table of contents the LLM itself wrote, reporting 98.7% on FinanceBench in its own vendor-run evaluation. It solves a different problem than vector RAG — finding the right section in a well-structured long document.
Traditional RAG splits documents into small chunks for retrieval, but this causes information fragmentation. LongRAG leverages 100K+ token long-context models to retrieve larger document segments (entire sections or even whole documents), reducing fragmentation while maintaining retrieval efficiency.
RAG has evolved far beyond simple 'search + generate' into a technology ecosystem spanning ten generations — and since 2025 into an Agentic/Reasoning era. This article is a systematic navigation guide: from Naive RAG to Multi-Agent/LongRAG across ten generations, the post-ten Agentic Era (Search-R1/RL search, MCP, GraphRAG 3.x, vision-native retrieval), retrieval strategies, chunking, embedding, reranking, evaluation frameworks, observability, and cost optimization. Each topic has a dedicated deep-dive article.
Chunks too large and retrieval loses precision; too small and you lose context; hit a table and retrieval falls apart entirely. Chunking is the most underrated part of RAG — pick the wrong strategy and no amount of downstream optimization will save you.
Bi-Encoders are too coarse, Cross-Encoders are too slow — ColBERT's Late Interaction finds the sweet spot: token-level comparison between query and document, but with document vectors that can be precomputed.
Filters too strict and getting zero results? CRAG automatically relaxes them and retries — far better than letting the LLM hallucinate an answer from general knowledge.
Vector search similarity scores don't equal relevance. Cross-Encoders use pairwise comparison to reorder results and push the truly relevant documents to the top.
BM25, vector search, HyDE, and Multi-Query each produce separate result sets -- how do you merge them sensibly? RRF uses ranks instead of scores, sidestepping the fundamental problem that scores from different systems are incomparable.
BM25 only recognizes words that appear in the query. SPLADE infers related terms and adds them to the search, gaining partial semantic capability while preserving the precision of keyword search.