A correct RAG answer can still disclose someone else’s data
Authorize every retrieval before data reaches the model. Cache hits and post-revocation responses must also respect current access to their source data.
#rag
170 postsChunking and Indexing: How Big to Cut, What to Search With, and How to Know You Chose Right
Building an index means three decisions: the chunking unit, whether to search with keywords, vectors or both, and which embedding model to use. All three are semi-permanent, since changing any of them means rebuilding. The research is cautious: semantic chunking isn't consistently better, splitting by sentences or by document structure is usually enough, and overlap has small, setup-dependent effects. In one Traditional Chinese study, the correct passage ranked first 57% of the time with vector search alone, keyword search 76%, and the two combined 81%. Don't choose by feel: compare two setups on order 11's test questions, changing one thing at a time.
Evaluation and Launch: How to Know Your AI Knowledge Base Works, and Keep It Improving
Evaluate in two layers: did retrieval find the source passage, and is the answer grounded in it and correct. Looking only at answers can fool you: one study found that when the context wasn't enough to answer, models still got 35% to 62% right using their own knowledge, which breaks on questions only your organization can answer. Use order 11's test questions for both layers, and evaluate permission-crossing and unanswerable questions too; calibrate LLM judges against human samples. After launch, turn escalated and complained-about conversations into failure cases, hand them back to document owners, and add them to the test set.
Structure and Metadata: Which Fields to Record Besides the Text
Metadata serves three purposes: returned for citation, used in search, and used to filter before retrieval; exact conditions like model, year or region belong in fields, and one study that wrote the company and year in front of every chunk raised top-five hits from 33% to 55%. Take filter values from source systems as fixed options, store dates as numbers, and don't let an LLM generate them freely.
Parsing and Cleaning: What Quietly Breaks When PDFs, Tables and Scans Become Text
When documents become text, what breaks usually doesn't throw an error: multi-column paragraphs joined in the wrong order, merged table cells flattened, headers and footers mixed into the body, OCR typos in scans, images and strikethroughs gone. High OCR accuracy doesn't mean good RAG either: in one study, documents with strikethroughs and color marks scored 82.9% on OCR but only 52.8% on question answering. Chunking matters more than switching tools: from the same conversion output, chunking by heading hierarchy with the heading path attached raised table questions from 66.6% to 99.8%. So always spot-check parsed output against the originals for these failure types.
Permissions: How to Make Sure the AI Doesn't Hand Department A's Documents to Department B
Filter permissions at retrieval time: each chunk carries the groups allowed to see it, each query carries the user's groups, and anything that doesn't match never becomes a candidate, rather than letting the model read it and then telling it not to say it. AI doesn't create new permissions, but it makes documents that were shared too widely findable with a single question, so clean up oversharing before launch. When permissions change, the index doesn't necessarily follow: some connectors update permissions only on full crawls, up to a day late. Filtering has a cost too: with strict conditions, some vector databases return too few results. Before launch, add a permission-crossing question to your test set.
Security: Can Documents in Your Knowledge Base Leak Personal Data or Give Orders to the AI?
Documents in a knowledge base may carry personal data or hidden instructions for the AI, and the latter has been demonstrated on Slack AI and Microsoft 365 Copilot. Telling the model to ignore them in the system prompt does little; it's safer to add personal-data and instruction gates at ingestion and limit what the model can do. No detection tool catches everything, so every layer only reduces the risk.
Inventorying Sources: Which Documents Belong in an AI Knowledge Base, and Which Version to Trust
Importing the whole shared drive is the most common way to start an AI knowledge base, and the most common mistake. When old and current versions are retrieved together, the model blends them: in one study, even with the current information retrieved, adding a single outdated passage cut the overall score by more than 24%. First build a source inventory recording each source's owner, update frequency, access restrictions and handling decision; collect only one workflow's well-governed sources in the first batch; and keep only the current version of each document, labeling any old version you must keep with its version and expiry date. After importing, check the file counts, because some connectors import nothing and report no error when they lack permissions.
Updates and Deletion: How an AI Knowledge Base Keeps Up When Documents Change or Disappear
When documents change or are deleted, the index doesn't keep up on its own. A sturdier incremental sync uses a content hash to find which documents changed and redoes only those; deletion has to reach every place a document is stored, including the full-text index and the vector store. Many tools don't handle deletion by default, and deletes and updates can take a while to take effect. Why clean up thoroughly: one preprint put a single outdated document into the context, and two small models flipped three to four in ten medical questions they had answered correctly; when told to answer from the documents, that rose to between two-thirds and three-quarters.
What's Different When AI Reads Your Knowledge Base: Write the Test Questions First
A wiki that people understand can still trip up an AI: what it sees is a few chunks cut out of the pages, without the navigation, heading hierarchy or surrounding text. So before building RAG, write a set of test questions: each with its answer and its source passage, plus a few deliberately unanswerable ones. Azure's RAG guide does this in parallel with content analysis; one study found that evaluating only with AI-generated questions overstated retrieval performance by 29 to 37 percentage points, so mix in real users' questions. If your knowledge base is under about 500 pages, you may not need RAG at all: put the whole thing in the prompt.
Building a Knowledge Base: Four Questions for Personal Notes, Team Wikis and AI Knowledge Bases
Personal notes, a team wiki and a knowledge base for AI differ in who reads them: your future self, your colleagues, a model. All three have to answer what to keep, how to organize it, how it gets found and who maintains it. Research on personal collections finds that most 'finding' is re-finding things you have already seen; research on enterprise wikis finds that once the person who tidied or wrote the content leaves, it starts to decay.
Contrastive Learning and Semantic Retrieval: How to Learn Vectors You Can Compare Without Labels | Stanford CS229 Lecture 13
This lecture unpacks the SimCLR loss into a B×B similarity matrix: each row is one softmax classification problem whose answer is the diagonal entry. The diagonal gets pulled up and everything else gets pushed down, so the model learns that similar things have nearby vectors without any labels. The second half uses the same vectors for semantic retrieval, then connects that to RAG.
Liquid AI LFM: On-Device Reasoning and Agent Workflows
LFM2.5-2.6B is an on-device hybrid reasoning model; checkpoint and OpenRouter context limits differ, with license and data terms to check before deployment.
pplx-embed-v2-late: An Open Late-Interaction Model That Searches PDF Pages Without OCR
On October 7, Perplexity open-sourced two ColBERT-style multimodal embedding models (0.6B and 9B, MIT license) that keep one 128-dim vector per token, score with MaxSim, and search PDF page images directly. The 9B scores 92.4% on MADQA (self-reported, with a Gemini 3.5 Flash agent); the cost is an index that grows with document length.
pplx-embed: Perplexity's Embedding Model Family and How to Choose Between Its Three Lines
pplx-embed is the embedding family Perplexity has been releasing since February 2026, in three lines: v1 single-vector (0.6B/4B, API at $0.004/$0.03 per 1M tokens), context embeddings (v1 plus a 9B v2 preview), and v2-late multi-vector (0.6B/9B, searches PDF page images directly). They solve different problems; v2 does not replace v1.
AWS Machine Learning Engineer Associate (MLA-C02): 45 of Its 107 Skills Are New
MLA-C02 replaced the English MLA-C01 on September 29, 2026. The four domains weigh 28 / 24 / 24 / 24, and each domain name now carries AI or FM: the official comparison lists 45 added skills covering vector databases, embeddings, RAG, selecting and fine-tuning foundation models, and deploying and monitoring agents, with 7 removed. It is currently an English-only beta (exam code ME1-C02, 170 minutes, 85 questions, $75). The standard version per the exam guide is 65 questions, pass at 720, valid 3 years, with Japanese, Korean, and Simplified Chinese arriving at general availability; there is no Traditional Chinese version.
Microsoft AI-200 (Azure AI Cloud Developer): AI Is in the Name, but It Tests the Backend That Keeps AI Apps Running
AI-200 (Azure AI Cloud Developer Associate) succeeds AZ-204, retired July 31, 2026. Its four areas weigh 20–25 / 25–30 / 20–25 / 20–25: containers, data services, messaging and serverless, security and monitoring. Nothing in the objectives covers model deployment, prompts, or agents; the closest it gets to AI is vector search on Cosmos DB, PostgreSQL pgvector, and Managed Redis. Official specs: $165 in the US, $83 in Taiwan, 120 minutes, pass at 700, 13 languages including Traditional Chinese, one-year validity, and no practice assessment yet.
AI Agent GitHub Digest — 2026-10-07
**Autoloom** (256★) bakes the Aegis engineering-governance method pack into a coding agent's execution loop, running on DeepSeek's own open-source DeepSeek Harness. **pi-pocket** (136★) is a mobile/web multiplayer shell for Pi, a coding agent from Earendil, built on the just-released Pi Durable harness. **brewery-ai** (193★, formerly Homebrew) walks you through an entire fine-tuning run in one conversation, under a license that's MIT plus a commercial-revenue threshold. **jev-doc-search** (99★) does long-document search with no vector database and no embeddings — it leans on TypeSafe's probabilistic decision model Jev paired with PageIndex's tree-shaped index. Claude Code 2.1.291 fixed two regressions that could drop messages or permission-prompt answers.
ML/LLM System Design and Coding Interviews: Four Architectures, Three Python Problems, and a STAR Skeleton
Four system design questions that come up often (enterprise RAG, an LLM serving stack, a customer-support agent, online A/B testing with a data flywheel), three Python problems (dynamic batching, sampling, backoff retries plus a token bucket), and a STAR skeleton for behavioral rounds. Each design question has a speakable answer and a self-check list, and all code was actually run.
RAG Interview Prep: From the Three-Stage Pipeline to Agentic RAG, and How to Tell Nine Variants Apart
Answer RAG interview questions in four steps: the three-stage pipeline, where each stage fails, which of nine variants patches which hole, and the 6 kinds of problems RAG cannot solve. Self-RAG, CRAG and Adaptive-RAG add judgment; Graph RAG and KG-RAG change the index; CAG skips retrieval; Agentic RAG and Reasoning RAG hand decisions to the model.
AI Agent GitHub Digest — 2026-10-02
PageIndex (38.4k★, +1,097 today) swaps vector indexes for a reasoning-based table of contents. iFixAi (18.3k★, +340) audits whether an agent actually did its job in under 120 seconds. Octop (6.2k★, +285) is Tencent Cloud's open-source, local-first multi-agent assistant platform. BMAD-METHOD (53.7k★) rewrites agile development into a spec-driven workflow for coding agents. Agno v3.1.0 ships RBAC but needs a stop-the-world migration for its filesystem; Pydantic AI v2.52.0 patches a web_fetch security bug and folds its harness into the main repo.
Retrieval-Augmented Generation (RAG) — Chunk the Text, Embed It, Put the Closest Pieces Back in the Prompt | NCCU Yen-Lung Tsai Generative AI Lecture 8
L06 said a prompt is two things: correct information and clear instructions. RAG lets the computer fetch the information part on its own. Split your documents into chunks, turn chunks and questions into feature vectors with the same model fθ, find the closest few chunks, and drop them into a template: 'Answer {question} based on {retrieved_chunks}.' The code comes in two notebooks: Demo06a builds a vector database with LangChain and FAISS and zips it as faiss_db.zip; Demo06b loads it back, connects an LLM, and wraps it in Gradio. The week-8 assignment is to do the same with your own data.
Building a Cat-Facts RAG Two Ways, with LangChain and by Hand | NTHU Hung-Yu Kao Natural Language Processing Assignment 4 / Tutorial
Each of the two RAG TA sessions builds one version. The first installs Ollama on Colab to run llama3.2:1b and wires up a minimal RAG with LangChain's Chroma, MMR, and retrieval chain. The second uses LangChain only for data prep and writes the rest by hand: chunking, text and vector stores, hybrid BM25 + cosine retrieval merged with RRF, then generation with Llama-3.2-1B-Instruct. HW4 applies the first session's skeleton to 150 cat facts and 150 GPT-5-generated QA pairs. The generator must be Llama3.2-1b and the embedding model jina-embeddings-v2-base-en, and you report recall@1, recall@5, and exact match. Code is 45% of the grade and the report 55%; the report analyzes how prompts, data format, document order, and counterfactual information change the results.
NTHU NLP RAG, Part 2: Connecting the Retriever to a Reader, from ORQA and REALM to Self-RAG | NTHU Hung-Yu Kao Natural Language Processing Topic Guide
The second half of W11_RAG.pdf starts at the "From Retrievers to QA" slide and turns a retriever plus a reader into a full QA system. It begins with ORQA and REALM (2019–2020), where BERT is the reader, then covers the first paper named RAG and REPLUG, which keeps the LLM frozen. A single table then sorts seven recent fixes into three groups: rewrite the query (Query Rewriting, HyDE), make the generator robust to noise (RetRobust, RAFT, RAAT), and decide when to retrieve (FLARE, Self-RAG). It ends with noise types, four abilities an LLM needs inside RAG, and generative retrieval (GR) with reliable response generation (RRG). On the recording side, the W11 Thursday lecture stops at the RAG paper; I could not find a recording that covers the later slides.
Reading NTHU Kao's NLP: RAG (Part 1) — Hallucination and Retrievers, from BM25 to DPR and GTR | NTHU Hung-Yu Kao Natural Language Processing Topic Guide
An LLM will confidently answer that Oppenheimer was born in 1967 (the year he died). RAG retrieves first and generates second. The first 60 pages of Hung-Yu Kao's Fall 2025 RAG deck are all about finding the right material. Sparse vectors (bag-of-words, TF-IDF, BM25) are cheap and dependable; dense vectors catch paraphrases. BERT's [CLS] isn't a good sentence vector as-is, hence Sentence-BERT pooling and bi-encoders. A cross-encoder is accurate but needs nearly 50 million passes for 10,000 sentences, while a bi-encoder needs 20,000. SimCSE uses dropout as data augmentation, DPR beats BM25 with only 1,000 training examples, and GTR shows that scaling up a dual encoder improves out-of-domain retrieval.
RAG: Looking Things Up Before Answering When the Model Cannot Remember or Is Out of Date | NTU Yun-Nung Chen Applied Deep Learning Lecture 8
A language model stores knowledge in its parameters: rare facts are poorly remembered, events after training are unknown, internal company documents were never seen, and answers cannot be checked. RAG lets it take the exam open-book: retrieve relevant passages for the question first, then hand them to the model with the question. This lecture covers how retrieval is done (chunking, sparse and dense retrieval, bi-encoders and cross-encoders, training a retriever without labeled pairs), what can be added before and after retrieval (query rewriting, reranking, multiple rounds of retrieval), the lines along which RAG developed (what to retrieve, how to use it, when to retrieve), and finally which situations suit prompting, RAG, or fine-tuning.
Agentic LLMs, or Letting the Model Look Things Up, Call Functions, and Run Its Own Loop | Stanford CME295 Lecture 7
Lecture 7 of CME295 (2025) patches three LLM gaps: RAG fixes knowledge frozen at training time with a two-stage retrieve-then-rerank pipeline; tool calling fixes the inability to act by having a backend execute the function call the model writes; agents chain those calls with ReAct's observe-plan-act loop. The 2026 edition renames it AI Agents and adds context compaction, harness optimization, coding agents, and skills, the biggest rewrite in the course.
Few-Shot OpenQA with DSPy | Stanford CS224U Assignment 2
CS224U's second assignment, hw_openqa.ipynb, asks you to answer questions that come with no passage, using only a frozen language model and a frozen ColBERT retriever. The Spring 2023 version was written for DSP; in January 2024 the repo switched to DSPy and pinned dspy-ai==2.4.13. Before you start you need an OpenAI API key, a ColBERTv2 checkpoint of about 406 MB, and a 600 MB prebuilt index. The notebook's first setup call, dspy.OpenAI, no longer exists in DSPy 3.4.
From Classical IR and IR Metrics to Neural IR | Stanford CS224U Topic Guide
The Spring 2023 edition of CS224U spends a whole unit on retrieval, because OpenQA gives you only the question and you have to find the evidence yourself, while large language models fabricate sources. The slides by Potts and Omar Khattab go from TF-IDF and BM25 through Success@K, MRR, and average precision to four neural IR designs (cross-encoder, DPR, ColBERT, SPLADE) and how each trades expressiveness against scale, and they end by asking you to count latency and cost as metrics too.
The Bug Photo That Wouldn't Show Up: Three Rounds of Fixing Image Search
An AI assistant platform's image search needed three rounds of fixes: pushing node_type filtering into the ES query to stop text chunks from hogging top-k slots, making filename matching deterministic instead of relying on the LLM to pass an optional parameter, and switching from Postgres icontains to ES match for CJK-aware partial matching.
10% of Conversations Were Making Things Up: Debugging a Silent Retriever Failure
About 10% of conversations on our AI assistant platform randomly lost knowledge base tools — the agent hallucinated answers from training data instead. Root cause: a bare except Exception swallowed Elasticsearch connection failures during retriever initialization, silently skipping tool registration. Fix: retry + surface failures to system prompt + structured metadata tracking.
Three Fixes for One Setting: When top_k Lies Across Four Layers
A user set retrieval top_k to 15, but the monitoring dashboard showed 5 and the streaming UI flashed 5 before jumping to 15. The same top_k value existed at four layers — LLM tool arguments, runtime, trace DB, and streaming payload — each requiring its own override. Three sequential fixes, each revealing the next layer was also wrong.
Commercial Document Parsing APIs Compared: Specialized Parsers, General VLMs, and the Big Three Clouds
Three routes to commercial document parsing: specialized parsers (Cohere Parse at $1.50/k pages, LlamaParse Agentic Plus at 90.2% on ParseBench), Big Three cloud prebuilts (Azure/Google/AWS for structured field extraction), and general-purpose VLMs (Fable 5.1 scores 78.92 on ParseBench and crushes specialized parsers on charts, but costs 3–16× more and hallucinates). At 100K pages/month, plain OCR runs ~$150 across providers; add tables and AWS jumps to $1,500, Claude Sonnet 5 to $900. The first question isn't 'which is most accurate' — it's 'do you need transcription or comprehension?'
Groundlane Series Part 6: The Document Toolkit — effort Dial, Field-Aware Chunking, and Confidence Routing
document_parse gains an effort parameter (fast/standard/deep) unifying anydoc WASM, OCR.space, and Docling VLM into a single dial; document_chunk's fieldAware mode extracts field names from table headers and metadata to solve RAG attribute conflation; document_smart_parse now returns a confidence score so agents decide whether to upgrade.
Groundlane Series Part 7: Selector Healing, Retrieval Test, and Quality Benchmarks
Groundlane v0.1.0 adds four quality mechanisms: web_extract selector healing (three-layer deterministic fallback, no LLM), corpus_retrieval_test (RAG recall verification inspired by RAGFlow), a document benchmark CLI (character-level F1), and a search benchmark framework (ground truth corpus with multi-provider comparison). Tool count goes from 54 to 55.
Hand-Build Your First RAG: From Chunking to Agent-Decided Retrieval | Stanford CS329Z Lecture 3
Lecture 03 starts from the project question and works through RAG's motivation, chunk boundaries, approximate vector search, BM25 / dense / hybrid / rerankers, and letting the agent decide when to retrieve. The assigned Lewis et al. RAG paper and ColBERT add the original recipe and late interaction.
Make Agents Remember What Matters: MemGPT, Letta, Mem0, and Generative Agents | Stanford CS329Z Lecture 7
Lecture 07 sorts agent memory into working, episodic, semantic and procedural types, compares skill libraries, MemGPT-style tiers, reflective consolidation and memory maintenance, and covers memory benchmarks plus forgetting, privacy and over-personalization. The readings run from MemGPT's virtual memory to Letta, Mem0 and Generative Agents.
For a Coding Agent, Build a Good IDE First | Stanford CS329Z Lecture 17
Per the schedule, Lecture 17 covers how coding agents work end to end: the SWE-agent, Claude Code and OpenHands architectures, scaffolds as design decisions, SWE-bench and the 4-tuple framework in practice, and the future of software development. The readings cover SWE-agent's ACI, OpenHands's event-stream sandbox, Claude Code practice, long-running harnesses and SWE-bench.
Compound AI Systems: How Models, Data, and Tools Work Together | Stanford CS329Z Lecture 1
Start with the opening slides on the agent loop, components, and engineering challenges, then examine BAIR's compound systems, Ng's work patterns, and Si's execution-feedback experiments. Distinguish the lecture, required reading, and additional readings.
Marker: Datalab's Open-Source Pipeline Parser — Faster, CPU-Ready, More Accurate
Marker (Datalab open source, Apache 2.0 license, v2.0.0 released 2026-07-20, 39.5k stars) is a pipeline-style document parsing standard library that outputs Markdown and JSON, supports optional LLM boost (`--use_llm`, default `gemini-3.5-flash`), custom formatting logic, table/formula/inline-math/link/reference/code formatting, image extraction and preservation, header/footer removal, and runs on pure CPU, GPU, or MPS (`Apple Silicon`). Unlike [Docling](/posts/tech/2026-09-06-docling-document-parsing) (structured JSON core, dedicated XML exports, pure MIT), Marker centers on Markdown/JSON under `Apache 2.0` with a separate model-weight license (`AI Pubs Open Rail-M`, $5M commercial threshold); unlike [MinerU](/posts/tech/2026-09-05-mineru-ocr-doc-parsing) (custom agreement with MAU/revenue thresholds + attribution obligations), Marker offers a simpler licensing story (`Apache 2.0` code) but requires accepting a separate model-weight license for weights. The series framework ([three-layer model](/posts/ai/2026-08-06-document-parsing-three-layers)) positions all three (`MinerU`, `Docling`, `Marker`) as pipeline-based parsing-layer options with distinct licensing, output, and speed trade-offs.
Docling: IBM's Open-Source, MIT-Licensed Document Parsing Standard Library Built on Structured JSON
Docling (IBM Research Zurich, now governed by the Linux Foundation AI & Data, MIT license, v2.100.0 released 2026-06-09) is an open-source document parsing standard library with structured JSON (DoclingDocument) as its core output. It supports PDF, DOCX, PPTX, XLSX, HTML, EPUB, Apple Pages, video (MP4/AVI/MOV with ASR transcription and keyframes), audio (WAV/MP3), email (EML/MSG), ODF, and XBRL financial reports, with a swappable-stage pipeline parser (pure CPU or GPU-accelerated), VlmPipeline option (GraniteDocling 258M VLM), MCP server and API server (docling-serve), and native integrations with LangChain, LlamaIndex, Crew AI, and Haystack.
MinerU：從 PDF 到 Markdown 的開源文件解析引擎，為 RAG 與 LLM 訓練而生
MinerU（OpenDataLab / opendatalab/MinerU）是一個開源文件解析引擎，把 PDF、圖像、DOCX、PPTX、XLSX 轉成結構化 Markdown 與 JSON，內建公式識別、表格提取與 109 語言 OCR，起源於 InternLM 前訓練階段，適合 RAG 前處理與知識庫建構。
Agentic Parsing: Letting Agents Decide How to Parse Documents
Traditional document parsing runs a fixed pipeline regardless of input, but contracts, financial reports, and technical manuals each need different strategies. Agentic Parsing lets LLM agents observe a document and dynamically choose tools — AgenticOCR parses only the regions that matter (70%+ visual token savings), and ParseBench shows even the best method scores only 84.9% across 2,000 enterprise pages. No silver bullet.
ColPali: Skip OCR, Retrieve Documents Directly from Images
ColPali renders each PDF page as an image, generates patch-level multi-vector embeddings with a vision-language model, and retrieves via MaxSim late interaction. On table-heavy financial PDFs, recall jumps from 62% to 84% — no OCR, no chunking. The tradeoff: ~100× storage, GPU required, no BM25.
Hierarchical Chunking + Auto-Merge: Small Chunks Search Well, Big Chunks Read Well
Small chunks give precise embeddings but lack context; big chunks have complete context but diluted embeddings. Hierarchical Chunking builds multi-level indexes (2048→512→128 tokens) with an Auto-Merge algorithm: leaf nodes match precisely, and when hit density exceeds a threshold the parent node is returned to the LLM instead. HiChunk shows a 12.7% evidence recall improvement; LlamaIndex and Haystack have it built in.
Multi-hop Retrieval: When Answers Are Scattered Across Documents
Standard RAG retrieves one set of documents per query, but real questions often need reasoning across 2-4 documents. IRCoT pioneered interleaved retrieval-reasoning, PAR²-RAG beats IRCoT by 23.5% accuracy on four benchmarks, and CompactRAG compresses LLM calls down to just two.
Self-RAG: Teaching the Model to Decide When to Retrieve
Self-RAG trains four reflection tokens (Retrieve / IsREL / IsSUP / IsUSE) into an LLM, letting it decide on-the-fly whether to retrieve, whether results are relevant, and whether its own output is grounded. ICLR 2024 Oral (top 1%), the 7B model beats ChatGPT and Llama2-chat + RAG on multiple QA benchmarks. The catch: it requires fine-tuning—no API-only models.
Table Serialization: How Format Choice Shapes RAG Retrieval for Tabular Data
Markdown-KV format achieves 60.7% LLM comprehension accuracy vs 44.3% for CSV — a 16-point gap from format alone. But retrieval and comprehension have different optimal formats: metadata prepend + row-wise key-value is the current best combination for table RAG.
AI Agent GitHub Digest — 2026-09-02
openclaw/openclaw, a self-hosted personal assistant, has climbed to 388k stars by wiring WhatsApp, Telegram, Slack and other chat channels into one Gateway. The same week, NVIDIA shipped SkillSpector, which scans Claude Code, Codex, and MCP skills for 71 vulnerability patterns — research it cites found 26.1% of skills contain vulnerabilities and 5.2% show likely malicious intent. Also today: stablyai/orca turns parallel multi-agent coding into a full IDE, and VectifyAI/PageIndex challenges the assumption that RAG needs a vector database with a reasoning-based tree index. claude-code v2.1.257 adds a Containment Escape security rule, and agno v3.0.5 stops swallowing embedding failures silently and starts reporting them honestly.
Why Ask AI Could Not List Its Course Maps: A Catalog Retrieval and Convergence Incident
The first observation of 'What course articles do you have?' was contaminated by an old cache entry. A real cache miss retrieved all four university maps but spent 51.169 seconds across three Writer and Critic passes; after catalog-specific retrieval and review fixes, one uncached production observation passed q21 in 26.821 seconds.
How Ask AI Finds Posts: Planner, Hybrid Retrieval, and Retry
Ask AI first extracts intent, complexity, and 1–4 search terms. It then routes across metadata, BM25, Vectorize, and RRF; a retry adds Critic gaps and disables the first-pass-only BM25 short circuit.
How Ask AI Indexes Posts: Chunks, D1 FTS5, and Vectorize
Ask AI indexing runs in two production stages: source-hash changes update D1, post chunks, and FTS5 first; embedding checkpoints and a delete queue then let Vectorize catch up asynchronously. The two stores do not share one transaction.
How a Question Moves Through Ask AI: UI, API, Agents, and Source Cards
Ask AI splits one question across the UI, `/api/chat`, Planner, Research, Writer, Validation, Critic, and Related stages. Answer text, displayed sources, and related-reading cards come from separate paths with separate gates.
Evaluating Ask AI Retrieval: Golden Contracts, Fixtures, Live Runs, and Evidence Boundaries
Ask AI keeps golden contracts, offline fixtures, live SSE output, and production observations separate. A passing fixture proves harness reproducibility; public sources can measure expected-source recall, but they do not expose hidden ranked chunks or establish model-graded faithfulness.
Debugging Ask AI in Production: SSE, Traces, Cache, Checkpoints, and Shadow Runs
One Ask AI request leaves five different evidence surfaces: public SSE, Langfuse traces, D1 logs, semantic cache, and a hidden shadow run. They expose different data, and no single surface reconstructs the complete retrieval context.
When Ask AI May Show Sources: Validation, Critic Review, Degradation, and the Source Gate
Ask AI finding a post does not mean the UI should display it as a source. An answer must pass deterministic Markdown and URL validation, then the Critic's relevance, intent, and grounding checks; if either gate fails, source cards are withheld.
How Ask AI Turns Evidence into an Answer: Writer Context and Citation Contracts
Writer sees the first 8 candidates for a factual query or 12 for a recommendation by default. Citations must use an exact `source_url` from that set, and weak or empty retrieval triggers an instruction to abstain rather than fill gaps from model knowledge.
How to Use Cloudflare Agent Memory: Keep Agent Memory Separate from RAG Documents
Agent Memory is a Cloudflare private beta service for letting agents remember users, teams, projects, and task context across conversations. It fits facts, events, instructions, and tasks; RAG documents, product data, files, and audit logs should still live in AI Search, Vectorize, D1, or R2.
Where to Store Cloudflare AI App Data: D1, R2, and Durable Objects
An AI app should not put conversations, artifacts, memory, retrieval documents, locks, and eval traces into one store. D1 fits queryable product data, R2 fits large files and artifacts, Durable Objects fit named coordination and per-session state, and Agent Memory / AI Search / Vectorize handle memory and retrieval.
Cloudflare AI Stack Guide: Building AI, RAG, and Agents on Workers
The Cloudflare AI Stack series covers the infrastructure around AI apps: where models run, how gateway control works, how RAG is built, how agents keep running, how memory is governed, and how browser, sandbox, secrets, data, and observability fit into a product.
How to Use Cloudflare Vectorize: Taking Control of RAG Retrieval
Vectorize is Cloudflare's vector database. AI Search is the right starting point for a managed RAG pipeline; Vectorize is the better fit when you need control over chunking, embeddings, metadata filters, hybrid retrieval, reindexing, and fallback behavior.
How to Use Cloudflare AI Search: Data Sources, Hybrid Retrieval, and Workers Bindings
Formerly AutoRAG, the managed search primitive: drop files into built-in storage or attach R2 and websites, auto-index with Markdown conversion plus vector and BM25, retrieve with hybrid, RRF, and reranking, and query from Workers via namespace or instance bindings, REST, or MCP.
Why Did 'I Want a Beginner AI Course' Return Zero Results? Debugging Chinese Tokenization and a Broken RAG Data Path
Entering '我想找入門的ai課程' in Ask AI showed zero searched posts and triggered a refusal, while Related Reading recommended exactly the right article. The first fix addressed Chinese tokenization, the LIKE fallback, and the Vectorize data path. A second pass added short Han-and-number tokens, post metadata retrieval, and a rule that exposes sources only after both Validation and Critic pass.
Embeddings: How Models Turn Words Into Computable Vectors
Models don't understand text — they only understand numbers. Embeddings map each token to a vector of several hundred dimensions, where semantically similar words end up close together in vector space. This is the shared foundation behind search, RAG, and classification.
Fine-tuning vs RAG: When to Teach the Model vs When to Look Things Up
Data changes often and you need citations → RAG. Need consistent style or want to run on a small device → fine-tuning. In practice, many production systems use both: fine-tune a small model that speaks your domain language, then use RAG to supply up-to-date facts.
When Search Returns Only 10 Results: Fixing CJK Recall in Cloudflare D1 FTS5 Hybrid Search
Querying “認證” returned only ~10 hits while 149 files (509 occurrences) matched; 41 posts and 76 chunks were found via LIKE, but D1 chunks_fts had 0 rows and unicode61/trigram both returned 0 for 2-char CJK terms. Fix: LIKE fallback with char-level OR first, then trigram migration + pnpm sync, then pagination beyond the hard limit of 12.
Agentic / Reasoning RAG: From Search-R1's RL Multi-Turn Search to Deep Research and MCP's Reasoning × Retrieval Paradigm
In 2025 RAG stopped being 'retrieve once, generate once.' Search-R1 trains models to search autonomously in multiple turns with RL, REX-RAG/AlignRAG add policy and alignment branches, OpenAI Deep Research productizes the loop, and MCP generalizes retrieval into unified tool invocation. This post unpacks the design philosophy, trade-offs against ten generations, and when to adopt the new paradigm.
Which Graph RAG to Choose: GraphRAG v3.1.2 vs LightRAG vs HippoRAG 2 — Design, Cost, and Selection
Same 'knowledge graph + retrieval' label, three different bets: Microsoft GraphRAG v3.1.2 pays indexing cost for global summarization, LightRAG cuts cost with dual-level retrieval and incremental updates, HippoRAG 2 turns RAG into growing associative memory via PPR — this guide splits the trade-offs by component with four query modes, indexing pipelines, and a selection matrix.
Late Chunking vs Contextual Retrieval: Encode-First Zero-Cost Context vs LLM-Prefix Precision and Cost
Anthropic Contextual Retrieval uses an LLM to prefix each chunk with 50-100 tokens, cutting failure rate from 5.7% to 1.9% with rerank at ~$1.02/1M tokens; Late Chunking encodes the full 32K-window document first then mean-pools by chunk boundaries for zero extra LLM cost — the trade-off is window, latency, and update shape.
2024 AI Conference Review: Natural Language Processing
NLP conferences redefined themselves under LLM dominance in 2024. ACL made open science its annual theme, and four of its seven Best Papers probed fundamental limits of language models. EMNLP turned toward multilingual and cross-cultural work, with Best Papers spanning speech representations and gradient interpretability. ACL and EMNLP received more than 10,000 submissions combined, but the deeper anxiety was what remains of NLP when LLMs can perform nearly every traditional NLP task.
Cohere — The RAG-Native Outlier: How Command, Embed, Rerank, and Aya Fit Together
Cohere is the only family that ships generation, retrieval, reranking, and multilingual as distinct products. Command A runs 256K context on two GPUs at 111B, Embed v4 does mixed image-text retrieval, Rerank v4 handles 32K semi-structured data, and Aya covers 101 languages — a four-piece stack built for RAG. This post breaks down each pillar's positioning, licensing, and selection guide.
Amazon Bedrock Deep Dive: Putting Model APIs Inside the AWS Governance Boundary
Amazon Bedrock is more than a reseller for multiple model APIs. It brings model invocation, IAM, Regions, Knowledge Bases, Guardrails, and CloudWatch into one AWS control plane. It fits teams already on AWS that value governance overhead more than the lowest token price.
Chroma Vector Database: From Local RAG to Distributed Retrieval
Chroma manages embeddings, documents, and metadata through collections; it embeds into Python locally, uses HNSW on a single node, and separates compute from storage with object storage, SSD caches, and SPANN in distributed deployments.
Cognee Complete Guide: Turning Documents into Graph Memory for Agents
Cognee is a data-to-memory pipeline: a relational store preserves sources and provenance, a vector store finds semantically similar content, and a graph store represents entity relationships, exposed through remember, recall, improve, and forget.
Information Retrieval: The Indexing and Ranking Layer Beneath RAG | Stanford CS124 Week 4
Week 4 teaches search: turn the query and the documents into vectors, weight them with tf-idf, rank by cosine similarity, speed it up with an inverted index, evaluate with precision, recall and MAP, and finally connect to dense retrieval and RAG. This post works the slides' four-document example in full.
Six Components of RAG and Language Agents | Stanford CS224N Lecture 10
Lecture 10 moves from question answering and RAG into language agents, then decomposes them into reasoning and planning, memory, tools, data, and evaluation. An agent is an inspectable loop between a model and external state.
WikiChat's Seven-Stage Defense Against Hallucination | Stanford CS224V Lecture 5
The [WikiChat paper](https://aclanthology.org/2023.findings-emnlp.157/) expands RAG into query formulation, retrieval, filtering, generation, claim extraction, renewed retrieval and verification, and removal of unsupported content—and evaluates retrieval separately from factuality.
STORM, Co-STORM, and Knowledge Curation | Stanford CS224V Lecture 2
STORM uses perspective-guided questions, simulated interviews, and outlines to broaden research; Co-STORM keeps a person in the loop so discovering unknown questions and co-editing become part of the system.
Dify as a Low-Code Agent Platform: From a Working Workflow to a Published AI App
Dify puts models, Knowledge, visual Workflows, Agents, Plugins, and application APIs in one workspace; this guide builds a minimal Workflow that can be tested, published, and called through the API, then explains when an Agent is actually warranted.
Haystack Deep Dive: Testable RAG with Components and Pipelines
Haystack turns indexing, retrieval, generation, and evaluation into replaceable Components connected by directed-multigraph Pipelines; it fits Python teams that want RAG flows to be tested, versioned, and deployed as code.
Jina Reader Guide: Turn Web Pages into Agent-Readable Markdown
Jina Reader turns a known URL into LLM-friendly Markdown; production use still requires explicit rendering, scope, token-budget, validation, and fallback decisions.
LanceDB Deep Dive: Embedding Vector Search in Arrow Data Workflows
LanceDB stores vectors, metadata, and multimodal source data in the Lance columnar format. Its OSS edition embeds in Python, TypeScript, or Rust processes; distributed Enterprise becomes relevant when the data or service outgrows one machine.
Milvus Vector Database Deep Dive: Segments, Indexes, and Distributed Operations
Milvus separates real-time ingestion, historical queries, index building, and persistence into independently scalable components. It fits large, continuously updated retrieval services, but smaller projects often pay too much operational complexity for that architecture.
pgvector Deep Dive: Bringing Vector Search Back into PostgreSQL
pgvector is a PostgreSQL extension, not a standalone vector database. It adds exact and approximate vector search to the same data model, transactions, and operations stack, while leaving index tuning and horizontal scaling as PostgreSQL concerns.
Private Corpus Search Boundaries: Decide Where Data May Go First
The first private-corpus decision is not which vector database to buy. Define data classes, trust zones, policy enforcement points, and freshness SLAs so every index, model, and observability system receives only the minimum data it is allowed to process.
Private-Corpus Retrieval Eval: Turning a Traditional Chinese Query Set into a Reproducible Benchmark
The repository has a 20-query Traditional Chinese/English golden dataset, but no document-level qrels, retrieval runs, raw latency data, or executable benchmark script. Reporting Recall@k, MRR, or nDCG as measured results would therefore be dishonest; this article defines the contract needed to run them reproducibly.
From Source to Index: Sync and Incremental Updates for Private Corpora
Private-corpus sync is not periodic refetching. It requires stable canonical IDs, source versions plus checksums for change detection, idempotent upserts, and tombstones that propagate deletion through every index.
R2R Deep Dive: Ingestion, Hybrid Search, and RAG behind an API
R2R packages document ingestion, hybrid search, knowledge graphs, RAG, Agents, and access controls behind a REST API; it fits teams that already own their product frontend and backend and need a retrieval service.
Choosing a RAG Framework: LlamaIndex, Haystack, RAGFlow, Dify, and R2R Operate at Different Layers
LlamaIndex and Haystack are code-first frameworks; RAGFlow and Dify are managed application platforms; R2R packages retrieval as an API service. Choose how much control your team needs over ingestion, retrieval, and operations before choosing a tool.
RAGFlow Deep Dive: From Document Parsing and Chunk Review to Cited Answers
RAGFlow puts document parsing, human chunk review, retrieval tests, chat, and citations in one platform; it fits layout-heavy PDFs and tables, but carries more deployment weight and platform state than a Python library.
Turning Retrieval, RAG, and Advanced Architectures into a System | Berkeley CS288 Topic Guide
Units 13–14 connect models to external knowledge; A3 requires data collection, QA annotation, indexing, and ablations under CPU and latency constraints.
LlamaIndex Is Not a RAG Framework Anymore, and Old Tutorials Won't Tell You
LlamaIndex (51,775 GitHub stars, MIT, verified 2026-08-21) has moved its center of gravity from indexing to Workflows: the standalone llama-index-workflows package pulls 2.81M weekly PyPI downloads, more than the 1.97M of the llama-index umbrella package itself. This post covers the core abstractions, the trade-off against hand-rolling a pipeline, and a hands-on test of its defaults on Traditional Chinese text — at the same chunk_size=1024, English fits 4,645 characters and Traditional Chinese only 1,332. Plus one fact you need before choosing: the TypeScript port is archived and unmaintained.
Qdrant Complete Guide: Collections, Hybrid Search, and Self-Hosted Operations
Qdrant is not just a place to store embeddings: define the vector schema, index frequently filtered payload fields, then add dense+sparse queries, tenant boundaries, snapshots, and monitoring to build an operable retrieval service.
Agentic AI: Formal Methods Against Hallucination | Stanford CS224V Course Overview
CS224V is Stanford's Agentic AI course: it teaches how to constrain LLM agents with formal semantics, knowledge graphs and formal methods so they fabricate less, with two assignments and then a project that carries most of the grade. The fourteen lecture decks, the reading list, the assignment handouts and the open-source systems the course uses (STORM, WikiChat, SUQL, Genie Worksheets) are public; there are no public recordings, the posted slides are deliberately incomplete, and the credentials the assignments need go only to enrolled students.
Engineering AI Agents: Building an Agent Harness from One Chat-Completion Call, No Frameworks | Stanford CS329Z Course Overview
CS329Z is Stanford's course on engineering AI agents: you first build RAG, tool use and the agent loop by hand without frameworks, then see what frameworks like DSPy abstract away; the two assignments are building an agent harness and designing an agent evaluation. The syllabus, the assigned readings, the first eleven slide decks, and the starter code, data and public tests for Assignment 1 are available; the recordings require an enrolled Canvas login.
LLM Application Design Interview Guide: From RAG to Agent Architecture
LLM Application Design is the hottest new interview topic in 2025-2026. Key focus areas: RAG pipeline chunking/retrieval/reranking design, agent tool-use and planning loops, context window management strategies, guardrails and safety design, and LLM application evaluation methods. Interviewers especially value whether you've hit real-world pitfalls.
AI Agent Arxiv Digest — 2026-08-19
QUMem uses episode segmentation plus a three-stage agent pipeline to infer user state, beating the strongest baseline by 4.6 pp overall success rate on KnowU-Bench; LENS retrieves without pre-built indexes, achieving 84.8% evidence recall vs ReAct's 50.4% with zero degradation when indexes go stale; Intent-Guided Decoding arbitrates between retrieved content and model memory at decode time, yielding up to 65.4 pp accuracy gains on factual-conflict benchmarks
AWS GenAI Developer Professional (AIP-C01) Prep Guide: Exam Scope, Resources, and a Four-Step Plan
AIP-C01 tests integrating foundation models into production AWS applications: RAG, agents, security, cost, and evaluation. This guide follows the five official domains with a prerequisite self-check, four-step preparation plan, resource choices, and hands-on checkpoints, plus a ten-week schedule, an AI study prompt, and exam-day guidance. Start with a diagnostic, then close gaps through one RAG-and-agent project.
RAG and Retrieval Evaluation Across Four Exams — and One Everyone Assumes Tests It, Which Doesn't
Four certifications genuinely test RAG and retrieval evaluation: AWS AIF-C01 (chapters 2 and 3 total 52%, covering RAG, vector stores, and FM evaluation metrics), AWS AIP-C01 (11 of the 27 skill points in its 31% Domain 1 sit in vector storage and RAG), NVIDIA NCP-AAI (Knowledge Integration 10% plus Evaluation and Tuning 13%), and Microsoft AI-500 ('multi-agent RAG architecture' inside its 30–35% Develop area). Google PMLE contributes exactly one LLM-as-a-judge objective, and Claude CCDV-F — the developer certification people most readily assume covers RAG — has no retrieval objective across its eight domains, with Eval at just 2.6%. Includes a same-vendor foundational-vs-professional comparison, a four-vendor terminology map, non-transferable objectives, and a practice project.
Agents, Prompts, and RAG: What's Left After the Lecture Is the Hard Part | Stanford CS230 Lecture 8
A BCG experiment found a jagged frontier: inside it, AI substantially improved consultants' work; outside it, AI made results worse — and people fell asleep at the wheel. The lecture also takes a strong position: avoid fine-tuning wherever possible, because by the time you're done tuning, the next model already beats your fine-tuned version.
Drawing the Lines: Agent, Workflow, RAG, and MCP
The line between workflow and agent is who decides the steps — the developer at design time, or the model at run time. By that definition most LLM systems in production today are workflows. Plus a usable test for choosing between RAG and an agent.
Three Shapes of RAG and the Evaluator Paradox
Standard RAG gives a wrong answer when it retrieves the wrong chunk, and nothing in the system will notice. Agentic RAG adds a self-check, at the cost of the evaluator paradox: the ceiling on self-correction is whatever the evaluating LLM can judge about relevance.
The Parsing Layer: When Structure Must Be Inferred — and Licensing Is the Real Selection Axis
Scans and complex layouts leave you no choice but to infer structure with a model. But the technical gap between MinerU, Marker, and Docling is far smaller than the licensing gap — MinerU needs a separate license past $20M monthly revenue, Marker's model weights need payment past a funding threshold, and only Docling is cleanly MIT. Read the LICENSE before the benchmark.
The Three-Layer Ladder of Document Parsing: Pick the Layer Before the Tool
The most common mistake in feeding documents to an LLM isn't picking the wrong tool — it's picking the wrong layer. Structure already in the file goes to the conversion layer (milliseconds); text without structure goes to extraction; only inferred structure needs parsing. anydoc's 4.7ms against Docling's 513.6ms is a 109× gap, and most people jump straight to the most expensive layer.
How to Rigorously Compare Before and After Agent Changes: From Golden Sets to Statistical Testing
Even with temperature=0, LLM outputs can still fluctuate by up to 15% in practice. To rigorously compare agent changes, you need a frozen golden set, at least 3 runs per query averaged out, LLM-as-judge blind evaluation (pairwise preference flip rate reaches 35%), and paired statistical tests -- not just running each version once and going by feel.
How Agents Decide Whether to Retrieve, What to Retrieve, and How to Merge: Three Decision Layers of Agentic RAG
Traditional RAG is a fixed pipeline of 'retrieve then answer.' Agentic RAG splits retrieval into three decision layers: when to retrieve (FLARE uses token probabilities; Adaptive-RAG uses a complexity classifier), what to retrieve (HyDE / RAG-Fusion / decomposition / Step-back), and how to fuse (RRF k=60 then cross-encoder rerank then compression -- Anthropic measured a -67% failure rate reduction). Key counter-intuitive insight: unnecessary retrieval hurts quality -- 'deciding not to retrieve' is a first-class capability.
Semantic Similarity ≠ Retrieval Relevance: Scenarios, Detection, and Remedies for Systematic Embedding Retrieval Failures
Cosine similarity and relevance systematically diverge across an entire class of scenarios: negation (most IR models score at or below random on NevIR), exact identifiers, numeric thresholds, and logical combinations (SoTA models achieve recall@100 < 20 on LIMIT) -- some of these hit the theoretical ceiling of the single-vector paradigm, and switching to a larger model will not help. Recommended remedy order: hybrid BM25 -> reranker (Anthropic measured -67%) -> upstream metadata routing -> domain fine-tuning / multi-vector.
A More Expensive Embedding Won't Save Your Traditional Chinese RAG: Three Layers of Failure and the Fix Order
Traditional Chinese RAG retrieval failures are a three-layer stack: embedding granularity defects (BGE/GTE from 0.1B to 7B all mis-rank on simple queries like 'fried chicken'), Simplified Chinese / English corpus dominance causing local vocabulary drift ('premium', 'exclusion clause' alignment is unreliable), and MTEB Chinese benchmarks being Simplified Chinese making model selection signals misleading. The fix is architectural: OpenCC normalization -> hybrid + jieba segmentation -> reranker -> local fine-tuning last -- and the prerequisite for all of it is building a Traditional Chinese eval set first.
Auto-Embedding on File Upload Is a Bad Default: A Survey of Adaptive / Agentic RAG and Agentic Parsing
Making 'chunk and embed every uploaded file automatically' the default behavior means making a decision for the LLM that it could have made itself. From Self-RAG (2310.11511) and Adaptive-RAG (2403.14403) to AgenticOCR (2602.24134), the academic trajectory is pushing three layers of decision-making -- whether to retrieve, whether to parse, and how to chunk -- from the ingestion pipeline back to the agent at conversation time.
LLM Agent Tool Descriptions Determine Tool Selection: Three Bug Fixes
Rewriting tool descriptions from soft suggestions to hard rules (whitelist + consequence explanation) eliminated the LLM's incorrect tool selection; adding skip_signal=True fixed vector store double-indexing.
Claude for Financial Services: Dissecting Anthropic's Multi-Agent Reference Implementation
Anthropic open-sourced 12 financial-industry Agents and 11 MCP connectors. The real takeaway isn't the Agents themselves but the layered design of 'one prompt, two runtimes' and 'pure-file extensibility.'
DeepSeek-OCR: The 10x Compression Experiment That Turns Long Context into Images
DeepSeek-OCR's paper is titled Contexts Optical Compression -- OCR is just the means; what it actually validates is that 'rendering text as images and feeding them to a VLM' achieves 10x compression at 97% accuracy. This is a qualitative shift for long-context LLM and RAG token costs.
Local Deep Research Walkthrough: A Privacy-First Deep Research Agent
Local Deep Research is a privacy-first deep research agent built on LangChain + LangGraph, integrating 20+ search engines and 30+ research strategies. Its flagship langgraph_agent_strategy takes the LLM-autonomous tool-calling approach, offering a fundamentally different paradigm from fixed-pipeline RAG graphs.
PageIndex: RAG Without Vectors — Turning Long Documents Into a Book With a Table of Contents
PageIndex skips chunking, embedding, and vector storage entirely. Instead it relies on LLM reasoning over a tree-structured table of contents the LLM itself wrote, reporting 98.7% on FinanceBench in its own vendor-run evaluation. It solves a different problem than vector RAG — finding the right section in a well-structured long document.
Building a Legal Contract RAG in 36 Hours: Weaviate Query Agent + ColQwen Architecture Breakdown
Using Weaviate Query Agent + ColQwen multi-vector model, a single prompt built a production-grade legal contract search system in 36 hours -- this post breaks down its architecture logic, technology choices, and what you actually need to watch out for.
knowledge-pipeline: A Six-Layer Pipeline for RAG Quality Control
A six-layer deterministic pipeline that handles everything from URL ingestion to vector embedding automatically, filtering out garbage before it enters your RAG system through an eight-dimension scoring system.
MarkItDown: Convert Any File to Markdown Before Feeding It to an LLM
A lightweight open-source tool from Microsoft that converts PDF, Office, images, audio, and more into Markdown — purpose-built for LLM pipelines.
quidproquo Blog Improvement Roadmap: Content, Technical Debt, RAG Design, and Harness Infrastructure
Using my own 30+ RAG/Agent posts to audit the blog itself, I identified a prioritized improvement list spanning content quality, site tech, RAG design fixes, harness infrastructure, and AI agent applications — no phases, just priorities.
The Full Picture of Cloudflare Workers AI Binding: It's More Than Just run()
env.AI is not just run(). It also exposes toMarkdown (document-to-Markdown conversion), autorag (managed RAG), gateway (external provider proxy), and models (metadata lookup). Understanding these four method groups is what unlocks Cloudflare as a full AI platform inside Workers.
Three Modes of LLM Knowledge Bases: Knowledge Vault, Experience Vault, and Blog
Andrej Karpathy proposed a framework for compiling personal knowledge wikis with LLMs — collect raw data, have the LLM compile it into .md wiki pages, run Q&A against the wiki, and file outputs back. This post compares three practical approaches: Karpathy's knowledge vault model, the community's experience vault model, and quidproquo's blog model.
AI-Ready Content: The Complete Guide to Making Your Website an AI-Readable Data Source
In 2025-2026, websites need to be readable not just by humans but by AI. From llms.txt and Schema Markup to GEO and RAG ingestion pipelines, this post maps out the complete technical landscape for turning your website into an AI-consumable data source.
"Recommend the next route" and "Recommend something similar" are not the same thing — Intent Disambiguation in RAG Recommendation Systems
In a climbing RAG system, 'recommend the next route' (progression) and 'recommend a similar route' (similarity) were conflated by a single hasSimilarRouteIntent() function, causing recommendation quality to collapse. The fix is a two-stage intent classification with a Regex Fast Path + LLM Fallback.
RAG Multi-Entity Queries: When the User Lists Five Routes and the System Only Sees the First
The RAG system's extractRouteReference() used a for...return pattern that grabbed only the first match — so when a user provided five completed routes, only one was used. The fix evolves through three layers: rule-based multi-entity extraction, user profile aggregation, and embedding centroid.
When Vector Search Matches by Name Instead of Grade: Attribute Conflation in RAG Systems
Query: 'I just sent Beauty in the Mirror 5.11b — recommend routes of similar difficulty.' The results came back full of routes with similar-sounding names, not similar grades. Root cause: dense embeddings compress multiple attributes into a single vector, and the rarity of the route name drowns out the grade signal. The fix: three layers of defense — metadata pre-filtering, query rewriting, and score fusion.
LangGraph: Managing Agent Workflows with Graph Structures
LangGraph models LLM workflows as directed graphs, solving the pain points of multi-turn iteration, conditional branching, and parallel execution that are difficult to handle with linear pipelines.
Context Engineering: Why Your AI Agent's Problem Is Information, Not the Model
Context Engineering is the core concept that replaced Prompt Engineering in 2025: the focus shifted from 'how to ask' to 'what information to provide.' Delivering the right information at the right time into the context window is more effective than upgrading to a stronger model. This post covers the definition, four key strategies, practical techniques, and common failure modes.
Agent Memory Systems: From RAG to Read-Write Memory Evolution
RAG is read-only. Agent Memory lets AI not only read but also write and persist information. Three memory types: Procedural (behavior patterns), Episodic (temporal events), and Semantic (factual knowledge) form a complete cognitive memory system.
Multi-Agent RAG: Distributed Retrieval Architecture with Specialized Agent Collaboration
A single RAG Agent handling all queries hits knowledge boundaries and performance bottlenecks. Multi-Agent RAG dispatches retrieval tasks to multiple specialized Agents, each with its own knowledge base and retrieval strategy, coordinated by a central Orchestrator that merges results.
LongRAG: Rethinking RAG Chunking Strategy with Long-Context Models
Traditional RAG splits documents into small chunks for retrieval, but this causes information fragmentation. LongRAG leverages 100K+ token long-context models to retrieve larger document segments (entire sections or even whole documents), reducing fragmentation while maintaining retrieval efficiency.
Speculative RAG: Small Models Draft in Parallel, Large Model Verifies at Once
Speculative RAG uses small specialist models to generate multiple answer drafts from different document subsets in parallel, then a large model verifies and selects the best answer in one pass. The paper reports +12.97 points accuracy and -50.83% latency on PubHealth — but that is the best cell in the table; other benchmarks gain far less.
The Complete Guide to RAG System Patterns: A Ten-Generation Evolution from Naive to Multi-Agent with Practical Navigation
RAG has evolved far beyond simple 'search + generate' into a technology ecosystem spanning ten generations — and since 2025 into an Agentic/Reasoning era. This article is a systematic navigation guide: from Naive RAG to Multi-Agent/LongRAG across ten generations, the post-ten Agentic Era (Search-R1/RL search, MCP, GraphRAG 3.x, vision-native retrieval), retrieval strategies, chunking, embedding, reranking, evaluation frameworks, observability, and cost optimization. Each topic has a dedicated deep-dive article.
Agentic RAG: Letting the LLM Decide When to Search Again
For complex multi-hop questions, a single RAG search isn't enough. Agentic RAG lets the LLM evaluate whether retrieved results are sufficient — if not, it rewrites the query and searches again, forming a ReAct loop.
BGE-M3: Why This Embedding Model Works Well for Traditional Chinese RAG
Your choice of embedding model directly determines RAG search quality. BGE-M3's multilingual training, 1024-dimensional vectors, and matching Reranker make it a practical pick for Traditional Chinese RAG.
Chunking Strategies: How You Split Text Determines Whether RAG Can Find the Answer
Chunks too large and retrieval loses precision; too small and you lose context; hit a table and retrieval falls apart entirely. Chunking is the most underrated part of RAG — pick the wrong strategy and no amount of downstream optimization will save you.
ColBERT: The Third Way in Vector Search
Bi-Encoders are too coarse, Cross-Encoders are too slow — ColBERT's Late Interaction finds the sweet spot: token-level comparison between query and document, but with document vectors that can be precomputed.
Contextual Retrieval: Giving Every Chunk Its "What This Is About" Context
When you split a document into chunks, each chunk loses its place in the original document. Contextual Retrieval solves the isolated-chunk problem by generating a per-chunk context from the whole document and prepending it at index time.
CRAG: Automatically Relaxing Filters When Retrieval Comes Up Empty
Filters too strict and getting zero results? CRAG automatically relaxes them and retries — far better than letting the LLM hallucinate an answer from general knowledge.
Cross-Encoder Reranking: Surfacing the Most Relevant Documents
Vector search similarity scores don't equal relevance. Cross-Encoders use pairwise comparison to reorder results and push the truly relevant documents to the top.
GraphRAG: Structuring Knowledge as a Graph for Relationship-Based Reasoning
Vector search finds similarity; graph search traverses relationships. When a question requires reasoning across multiple entities — crag → route → sender → grade distribution — GraphRAG outperforms standard RAG.
Hybrid Search: Using BM25 + Vector Search to Cover Each Other's Blind Spots
Vector search handles semantics; BM25 handles keywords. Combining them with RRF is what lets you handle both fuzzy queries and exact terms at the same time.
HyDE: Boosting Vector Search Recall with Hypothetical Answers
Have an LLM generate an 'ideal answer' first, then embed that hypothetical document for search — it outperforms searching with the raw query.
RAG Personalization: Learning User Preferences from Conversations
After each conversation, asynchronously extract likely user preferences and skill level, then automatically personalize search parameters on the next query — no manual setup required.
MMR + Popularity Weighting: Recommendations That Are Both Relevant and Diverse
Ranking purely by relevance leaves you with five documents all describing the same route. MMR strikes a balance between relevance and diversity, and layering in popularity weighting makes results even more useful.
Modular RAG Pipeline: Designing RAG as a Composable DAG
RAG doesn't have to be a rigid three-step process. It's a set of steps that can be dynamically enabled, skipped, or reordered. Pipeline as Code lets the system adapt its behavior without redeployment.
Multi-Query Expansion: Search One Question from Multiple Angles
A single vector search on a complex query often misses relevant documents. Let the LLM rewrite the query into 3-5 sub-queries, run them in parallel, and recall improves significantly.
Multimodal RAG: Bringing Images into the Knowledge Base
Climbing routes carry a ton of visual information (topos, wall photos) that text-only RAG misses entirely. Multimodal RAG makes images searchable and understandable.
Three Generations of RAG: From Naive to Modular
Naive RAG works but has real problems. Advanced RAG patches those problems. Modular RAG rearchitects the whole system to be composable and configurable. Understanding all three generations is the key to understanding why modern RAG systems look the way they do.
Plan-and-Execute: A RAG Pattern That Plans Before It Acts
For complex queries, have the LLM map out what information is needed and in how many steps — then execute that plan. More systematic than thinking on the fly.
Query Classification: Teaching Your RAG System How to Answer Each Question
Not every question needs full RAG. Classify queries with an LLM first, then route to the right execution path — saving cost and improving accuracy.
RAG A/B Testing: A Scientific Approach to Comparing Pipeline Configurations
"Adding a Cross-Encoder feels better" is not a scientific evaluation. A/B testing tells you whether a change actually works, how much it helps, and which query types benefit.
RAG Cold Start: Building a Useful System When You Have No Data
A RAG system needs data to answer questions, but data only accumulates as the system gets used. Cold-start strategy is what bridges the gap from empty to useful.
RAG Cost Optimization: Minimizing the Cost of Every Query
RAG system costs come from LLM tokens, Embedding APIs, and vector search. Every stage has room for cost reduction, but you need to verify that optimizations don't sacrifice too much quality.
RAG Evaluation Frameworks and Tool Selection: Promptfoo, RAGAS, DeepEval, and TruLens
No industry standard mandates one RAG evaluation tool. Measure retrieval, generation, and operations separately, then choose Promptfoo, RAGAS, DeepEval, or TruLens for the actual stack.
RAG Common Failure Modes: 10 Problems and Their Solutions
When a RAG system breaks, 90% of the time it's one of these 10 failure modes. Identify which one first, then apply the matching fix — far more effective than optimizing blindly.
RAG Guardrails: Adding a Defense Layer to Inputs and Outputs
The attacks RAG systems face go beyond the technical level — Prompt Injection and Jailbreak are real threats. Both inputs and outputs need independent protection layers.
RAG Observability Tool Landscape: Choices in 2026
Rolling your own traces is good enough, but open-source tools save you a lot of work. Langfuse, Phoenix, and LangSmith each have their niche — the right choice depends on your trade-offs around self-hosting, open source, and integration complexity.
RAG Observability: Per-Node Tracing to Turn the Black Box Transparent
The hardest part of a RAG system isn't building it — it's figuring out why a particular answer went wrong. Pipeline Tracing records every step's decisions and data so debugging has a clear trail to follow.
RAG Prompt Engineering: How to Design System Prompts and Context
Search found the right documents, but the LLM's answers are still poor — often the problem lies in prompt design. System prompt structure, context formatting, and instruction placement all affect output quality.
RAG Streaming: Using SSE to Display LLM Responses as They Generate
LLM generation takes 3-5 seconds, and waiting for the full response before displaying it makes for a terrible experience. SSE pushes tokens as they're generated, reducing time-to-first-character from 5 seconds to under 1 second.
RAG Quota System: Controlling LLM Costs with Dual Limits
Limiting request count alone is not enough — a single long query can consume ten times the tokens of a normal one. Dual quotas (request count + token count) are what truly control costs.
RAG vs Fine-tuning: It's Not Either/Or
RAG and Fine-tuning solve different problems. RAG gives the model new knowledge; Fine-tuning changes the model's behavior and style. In most cases you use both, not pick one.
RRF: How to Merge Multi-Source Results in RAG Systems
BM25, vector search, HyDE, and Multi-Query each produce separate result sets -- how do you merge them sensibly? RRF uses ranks instead of scores, sidestepping the fundamental problem that scores from different systems are incomparable.
Self-Reflection + LLM-as-Judge: Having AI Evaluate Its Own Answers
Use another LLM to evaluate answer accuracy and quality — if the score is too low, regenerate, and automatically add appropriate disclaimers.
Semantic Caching: Run the RAG Pipeline Only Once for Semantically Similar Queries
Caching doesn't have to match exact query strings -- semantically similar questions can hit the cache too, skipping the entire RAG pipeline execution.
SPLADE: Smarter Sparse Vector Search Beyond BM25
BM25 only recognizes words that appear in the query. SPLADE infers related terms and adds them to the search, gaining partial semantic capability while preserving the precision of keyword search.
Text-to-SQL Router: Precise Queries That Skip RAG
Questions like 'how many routes did I complete this year' will never be answered well by RAG semantic search — querying the database directly is far more accurate. Let the LLM identify intent, extract parameters, and execute predefined SQL templates.
Vector Database Selection: How to Choose Between Pinecone, Weaviate, Qdrant, and Vectorize
Vector database selection is more constrained by deployment platform than LLM selection. Determine your platform and scale requirements first, then evaluate features — don't just look at benchmarks.
Why Does a Climbing Community Need AI? NobodyClimb's Experiment and What We Learned
NobodyClimb uses RAG to tackle scattered climbing route information, ties quota limits to community engagement, and leverages Cloudflare Workers AI to bring inference costs close to zero.
NobodyClimb: Building a Climbing Community Platform Entirely on Cloudflare
A climbing community platform where the web app, mobile app, and AI Q&A all run on Cloudflare — no dedicated servers.
NobodyClimb AI Architecture: Building a 20-Node RAG Pipeline on Cloudflare Workers
A dynamically composable RAG pipeline built on Cloudflare Workers AI (gemma-3-12b-it + bge-m3): 14 base steps + 6 LangGraph-specific nodes, with three strategy graphs (Baseline / Agentic / Plan-Execute) selected at runtime.