Traditional document parsing runs a fixed pipeline regardless of input, but contracts, financial reports, and technical manuals each need different strategies. Agentic Parsing lets LLM agents observe a document and dynamically choose tools — AgenticOCR parses only the regions that matter (70%+ visual token savings), and ParseBench shows even the best method scores only 84.9% across 2,000 enterprise pages. No silver bullet.
ColPali renders each PDF page as an image, generates patch-level multi-vector embeddings with a vision-language model, and retrieves via MaxSim late interaction. On table-heavy financial PDFs, recall jumps from 62% to 84% — no OCR, no chunking. The tradeoff: ~100× storage, GPU required, no BM25.
Small chunks give precise embeddings but lack context; big chunks have complete context but diluted embeddings. Hierarchical Chunking builds multi-level indexes (2048→512→128 tokens) with an Auto-Merge algorithm: leaf nodes match precisely, and when hit density exceeds a threshold the parent node is returned to the LLM instead. HiChunk shows a 12.7% evidence recall improvement; LlamaIndex and Haystack have it built in.
AI boosted task output by 34%, but code review time surged 441% and measured delivery actually slowed 19%. A four-round research survey maps the current landscape: deterministic guardrails (hooks) vs probabilistic ones (prompts), clean-context review, self-improving feedback loops, specification-driven development, AI test quality crisis (high coverage but median 53% mutation score), and the Replit agent fabricating test results.
Standard RAG retrieves one set of documents per query, but real questions often need reasoning across 2-4 documents. IRCoT pioneered interleaved retrieval-reasoning, PAR²-RAG beats IRCoT by 23.5% accuracy on four benchmarks, and CompactRAG compresses LLM calls down to just two.
Self-RAG trains four reflection tokens (Retrieve / IsREL / IsSUP / IsUSE) into an LLM, letting it decide on-the-fly whether to retrieve, whether results are relevant, and whether its own output is grounded. ICLR 2024 Oral (top 1%), the 7B model beats ChatGPT and Llama2-chat + RAG on multiple QA benchmarks. The catch: it requires fine-tuning—no API-only models.
Markdown-KV format achieves 60.7% LLM comprehension accuracy vs 44.3% for CSV — a 16-point gap from format alone. But retrieval and comprehension have different optimal formats: metadata prepend + row-wise key-value is the current best combination for table RAG.
Looplane now grades effects as read/modify/modify_execute/execute and fails closed on unclassified tools. Native MCP tools default to execute unless trusted read-only metadata lowers them. Approval events still land in events.jsonl first and grants can scope to one change set or backend; general command rules and universal sandbox coupling remain unfinished.
looplane now ships a bounded tool-program DSL: read-only programs support list/read/search/diff, repeat, and if_contains; modify/check transactions receive whole-transaction approval and roll back touched paths on failure. This is not arbitrary JavaScript/Python code mode, and transaction execution is not parallel.
Mature-agent compaction must handle triggers, complete-turn cut points, and recovery. looplane now has an 85% high-watermark, automatic compaction, a deterministic native-loop fallback summary, persisted checkpoints, and workspace-context reinjection. Cross-runtime fallback, model-quality summaries, and live-provider long-session validation remain open.
omp and claude-code provide cross-session memory while the other references mostly rely on instruction files. looplane now has an explicit remember/list/inject baseline: typed JSONL memories enter prompts across sessions, but retrieval is scope-and-recency only, with no semantic ranking, deduplication, forget command, or automatic extraction.
All five projects combine allow/ask/deny decisions, compound-command inspection, and fail-closed behavior. looplane now has a deny-first classifier, critical floor, shell segmentation, timeout-deny, configured allow/deny rules, and visible policy reasons. Broader syntax coverage and live interactive validation remain open.
I'm building my own Python coding agent called looplane. This series dissects the source code of five mature projects — pi, oh-my-pi, opencode, codex, and claude-code — topic by topic, while also comparing them with Looplane's current TUI, external CLI runtimes, local gateway, usage/OTel/session tooling, and Cloudflare slice. Every post follows a fixed five-part structure: design problem → how five projects do it → looplane's choice → academic grounding → improvement roadmap, with evidence cited at file#symbol level.
Hooks govern control flow, skills inject knowledge, and plugins package both. looplane now has opt-in deny-only project hooks, a bounded SKILL.md loader, exact enabled_skills selection, plugin manifests/install/list, and external-runtime projection. Input rewriting, full lifecycle coverage, remote registries, and a mature marketplace remain open.
looplane can now inject repository diagnostics and open-file state into the next model turn, push typed IDE context over WebSocket, package a VS Code bridge, and supervise long-lived LSP subprocesses through ManagedLspServer. Language-specific initialize/didOpen/didChange adapters and live-editor validation remain open.
An MCP client must handle transports, tool refresh, approvals, and credential boundaries together. looplane now supports allowlisted stdio, Streamable HTTP/SSE, tools/resources/prompts, tools/list_changed, OAuth metadata/PKCE, and a 0600 credential store. A real authorization-server E2E and MCP-specific confirmation UX remain open.
looplane now has static ModelRole/ModelRoute candidates, opt-in aliases such as --model @cheap, cross-provider fallback, and a no-tool reviewer lane that runs after verification. Role inheritance/override rules and automatic summarizer, parser, or scout routing remain open.
Wrapping an SDK directly buys you three walls within months: usage fields that don't agree, error semantics tied to SDK exception types, and tool-call formats that change per provider. All five reference projects separate 'wire protocol' from 'provider identity' as independent dimensions. Looplane goes further with pydantic canonical contracts (Message/ToolCall/Usage/ModelTurn) plus six protocol adapters, forces the OpenAI SDK's built-in retries to zero, and routes every failure through a classified ProviderErrorKind before any retry policy sees it. Its provider table is deliberately copied from pi's packages/ai — lineage, not coincidence.
An OS sandbox is the kernel boundary beyond path policy. looplane now ships a fail-closed CommandSandbox: sandbox-exec on macOS, Landlock plus seccomp on Linux, and exit 126 when containment cannot be proven. Coverage still focuses on verification commands, and external CI confirmation remains open.
Looplane's prompt is now `m3-exact-edit-v4`: the version persists into artifacts; core/tool/interaction/runtime/instructions/skills/workspace/memory are composed as stable or dynamic sections; and positive/negative examples cover replace_text, unified diffs, and direct replies. Unit tests pin the structure, while live-eval coverage still needs expansion.
Intermittent NVIDIA NIM 500s exposed Looplane's early gap: classified errors with no retry consumer. SDK retries are now disabled; the harness gives each candidate up to five attempts with jittered exponential backoff and capped Retry-After handling, then can move to an explicitly configured fallback model. Both model.retry and model.fallback enter the event log.
looplane now connects events.jsonl to a deterministic reducer, CLI timeline, canonical JSON, SDK replay, and safe event-point forks. Forking never replays prior tools or model calls; provider/live-runtime validation, redaction, and richer replay hooks remain open.
Mature subagents need roles, bounded fan-out, narrowed permissions, and a result contract. looplane now has native named-role schedules, parallel fan-out, child allowed_paths constrained by the parent, unsafe execution disabled by default, and parent-approved transaction proposals. Persistent background lifecycles, recursion trees, and automatic worktree merging remain open.
looplane now has CostBreakdown, an explicitly estimated static GPT-5-family price table, per-lane usage/cost, and OTel cost fields. Unknown models still show tokens without invented dollars; broader pricing coverage, authoritative external-CLI bills, and live billing reconciliation remain open.
Looplane's core surface has grown from seven tools to nine with a read-only `tool_program` and rollback-capable `tool_transaction`; search prefers ripgrep and arbitrary shell remains absent. Native MCP tools join only from allowlisted servers and default to execute approval without trusted read-only metadata.
Looplane's disposable clone and SafePathPolicy protect the source repo. `--sandbox-checks` can now wrap verification commands with macOS sandbox-exec, Linux bubblewrap, or Landlock, while Cloudflare provides a separate bounded Sandbox slice. Network policy, external-runtime coverage, and production hardening are not yet consistent across those backends.
looplane now has a Cloudflare Durable Object run resource with async creation, status/cancel/artifacts, live NDJSON, and Last-Event-ID SSE; remote approvals use a separate short-lived capability. Python also provides an attach client and a stateful conversation WebSocket. Production deployment, cross-runtime parity, and full multi-tenant hardening remain unverified.
The first observation of 'What course articles do you have?' was contaminated by an old cache entry. A real cache miss retrieved all four university maps but spent 51.169 seconds across three Writer and Critic passes; after catalog-specific retrieval and review fixes, one uncached production observation passed q21 in 26.821 seconds.
Ask AI first extracts intent, complexity, and 1–4 search terms. It then routes across metadata, BM25, Vectorize, and RRF; a retry adds Critic gaps and disables the first-pass-only BM25 short circuit.
Ask AI indexing runs in two production stages: source-hash changes update D1, post chunks, and FTS5 first; embedding checkpoints and a delete queue then let Vectorize catch up asynchronously. The two stores do not share one transaction.
Ask AI splits one question across the UI, `/api/chat`, Planner, Research, Writer, Validation, Critic, and Related stages. Answer text, displayed sources, and related-reading cards come from separate paths with separate gates.
Ask AI keeps golden contracts, offline fixtures, live SSE output, and production observations separate. A passing fixture proves harness reproducibility; public sources can measure expected-source recall, but they do not expose hidden ranked chunks or establish model-graded faithfulness.
One Ask AI request leaves five different evidence surfaces: public SSE, Langfuse traces, D1 logs, semantic cache, and a hidden shadow run. They expose different data, and no single surface reconstructs the complete retrieval context.
Ask AI finding a post does not mean the UI should display it as a source. An answer must pass deterministic Markdown and URL validation, then the Critic's relevance, intent, and grounding checks; if either gate fails, source cards are withheld.
Writer sees the first 8 candidates for a factual query or 12 for a recommendation by default. Citations must use an exact `source_url` from that set, and weak or empty retrieval triggers an instruction to abstain rather than fill gaps from model knowledge.
Agent Memory is a Cloudflare private beta service for letting agents remember users, teams, projects, and task context across conversations. It fits facts, events, instructions, and tasks; RAG documents, product data, files, and audit logs should still live in AI Search, Vectorize, D1, or R2.
Cloudflare Agents turns an agent session into a durable runtime: each agent instance has stable identity, local SQLite, WebSockets, scheduled work, recoverable execution, and tools. It is not just a chat example; it composes Workers, Durable Objects, AI models, Browser, Sandbox, AI Search, and MCP into a deployable agent app.
An AI app should not put conversations, artifacts, memory, retrieval documents, locks, and eval traces into one store. D1 fits queryable product data, R2 fits large files and artifacts, Durable Objects fit named coordination and per-session state, and Agent Memory / AI Search / Vectorize handle memory and retrieval.
AI Gateway is the control plane for AI calls: one layer for logs, analytics, cache, rate limits, retry/fallback, BYOK, and Unified Billing. In Workers, use env.AI.run(..., { gateway }); with external SDKs, change the baseURL or provider-native endpoint.
The Cloudflare AI Stack series covers the infrastructure around AI apps: where models run, how gateway control works, how RAG is built, how agents keep running, how memory is governed, and how browser, sandbox, secrets, data, and observability fit into a product.
Secrets Store is Cloudflare's open beta account-level secret store, currently integrated with Workers and AI Gateway. It fits provider API keys, BYOK keys, and secrets reused across Workers; per-Worker secrets still work, but the governance scope is different.
Vectorize is Cloudflare's vector database. AI Search is the right starting point for a managed RAG pipeline; Vectorize is the better fit when you need control over chunking, embeddings, metadata filters, hybrid retrieval, reindexing, and fallback behavior.
GPUtw.ai makes sense as a short-rental GPU learning tool: start with Jupyter, Ollama, or ComfyUI, then try LoRA/QLoRA on a small model. It is not a large foundation-model training platform, and the first run should verify deployment, billing, and data retention with a small budget.
Keenable.ai positions itself as search infrastructure for AI agents: a 100B+ document index, Search/Fetch APIs, MCP/CLI entry points, 100K free monthly requests, and keyless public endpoints. It is worth tracking, but the 100B+ index, latency, and quality claims are still mostly company-provided; NEEDLE is open, but needs external reruns and human review.
screenshot-to-code is not a one-shot screenshot-to-HTML tool. Its core is a 30-step Agent Loop with 7 tools — extracting real assets from screenshots, self-verifying with Playwright, and running 4 models in parallel so users pick the best output. 74,500+ GitHub stars, MIT License.
Warp's self-improving agent pattern is not about dumping every mistake into a prompt. A base skill does the work, humans leave feedback in GitHub or Slack, an improver skill turns repeated signals into a small diff, and humans review the PR before the next run inherits it.
TinyFish provides four web APIs for AI agents: Search, Fetch, Agent, and Browser. Search and Fetch are permanently priced at $0 with no credit card requirement, making them a practical default layer for RAG and document retrieval.
Four philosophies of ADE workspaces: arul28/ADE's Brain+Lane, Superset's 100-agent IDE, Herdr's Rust-native runtime, and Kadro/Orca's pane and Fleet angles — compared in one table with a decision tree for when to pick a workspace over an Omnigent-style control plane.
Omnigent moves governance off prompts into a Server-side Policy engine: Python functions returning allow/deny/ask, a three-layer stack with cost budgets and tool caps, plus Omnibox OS-native isolation via bwrap/seatbelt and egress credential brokering — compared with five peer governance stacks.
The same model can score 20 points apart on different harnesses, 32% of SWE-bench Pro verifier judgments were found to be wrong, and DeepSWE's 113 tasks make most models score zero. This guide decodes six major coding benchmarks — what they test, which are easy to game, and which ones you should care about.
CS50 AI's OpenCourseWare edition publishes seven weeks of lectures, slides, notes, and twelve Python projects with autograder feedback, plus a free CS50 Certificate if you score at least 70% on every project. The catch: weeks 0–5 still use the Spring 2020 recordings; only Week 6 (Language) was re-recorded, in 2023.
The same model can cost 2-5× more depending on the channel. Direct API is simplest, aggregators (OpenRouter) are most flexible, cloud platforms (Bedrock/Vertex) suit enterprises. This post compares actual August 2026 prices across six channels with a decision tree.
meta-harness means two things: Databricks' control plane and Stanford's outer-loop optimizer. This post uses a four-layer model (MCP/ACP/Runtime/meta-harness) to place Omnigent, Zed ACP, Vercel HarnessAgent and Cloudflare Flue.
MIT 6.7960 Deep Learning (Fall 2025) publishes all 21 lecture decks as public Dropbox PDFs, and most required readings map to free textbook chapters; but the five problem sets are released only through Gradescope, and solutions plus recordings live behind Canvas login. This guide covers how the three instructors split the course, a topic map of all 21 lectures, textbook-based substitutes for lectures, and where outside self-learners realistically stop.
Nearly every frontier open-source model in 2026 is MoE: Ornith 35B activates only 3B to beat 31B dense models, MiniMax M3 uses 456B total but 45.9B active to hit SWE-bench Pro 59%, DeepSeek V4 runs 1.6T total with 49B active. This post explains why MoE dominates coding and agentic benchmarks using four case studies.
The same Polly task — parallel git worktrees plus cross-vendor review — implemented four ways: Omnigent YAML governs at the Server layer, LangGraph controls flow with a StateGraph, CrewAI assembles roles quickly, and Goose ships a desktop Recipe, compared on tokens, latency, and maintainability.
Allen AI's OLMo is the only language model family that fully publishes weights, training data (Dolma, 9.3T tokens), training code, all intermediate checkpoints, and evaluation tools. OLMo 3's 32B Think model hits 96.1% on MATH — and you can use OlmoTrace to trace any output back to the exact training data that produced it.
Databricks' open-source Omnigent wraps Claude Code, Codex, Cursor, Pi and custom agents in a Runner/Server + Omnibox sandbox, adding three-layer Policies and shareable persisted Sessions so you can swap models and harnesses with one-line changes — 9.3k stars, still alpha.
'Open-source' in AI doesn't mean what it means in software. MIT and Apache 2.0 let you do almost anything; the Llama License requires a separate deal above 700M MAU; old Gemma terms let Google change rules unilaterally (Gemma 4 switched to Apache 2.0). This guide maps what you can and can't do by license type.
Open-source models now match closed-source on coding benchmarks, but self-hosting isn't just picking a model — vLLM handles high-concurrency production serving, SGLang is 29% faster on prefix-heavy workloads, Ollama is the local dev default, and llama.cpp runs on the least hardware. A100 cloud rentals run ~$1.4-2.2/hr; self-hosting breaks even at roughly 100M tokens/month.
Three non-big-lab teams used different RL post-training strategies to produce benchmark dark horses in 2026: Ornith's self-improvement loop (GRPO), Nous Research's DataForge + Atropos execution-reward RL, and MiniMax's massive-scale RL across 200K real environments. Different strengths, but one shared proof point: post-training RL matters more than pretraining scale.
Models don't read words — they read tokens. A Chinese character is typically 1-2 tokens; an English word is 1-3. The context window is the token limit per request. Inference is using a model; training is teaching one. What you do every day is inference.
Models don't understand text — they only understand numbers. Embeddings map each token to a vector of several hundred dimensions, where semantically similar words end up close together in vector space. This is the shared foundation behind search, RAG, and classification.
Benchmark scores in model releases have three common traps: cherry-picking (only showing wins), contamination (test data leaking into training), and saturation (when everyone scores 90%+, the benchmark stops being useful). The most manipulation-resistant signal is Chatbot Arena's Elo ranking — real humans, blind voting, uncontrolled questions.
Data changes often and you need citations → RAG. Need consistent style or want to run on a small device → fine-tuning. In practice, many production systems use both: fine-tune a small model that speaks your domain language, then use RAG to supply up-to-date facts.
A model uses loss to know how wrong it is and gradients to know which direction to adjust. Gradient descent repeats three things: compute loss, compute gradients, update parameters. The learning rate controls step size — too large and you overshoot, too small and training takes forever.
Every time a model predicts the next token, it assigns a probability to every candidate word. A loss function measures how far that probability distribution is from the correct answer — the further off, the higher the loss, the more the model knows it got it wrong. Cross-entropy is the standard formula; perplexity is its human-readable translation.
A 70B model needs ~140GB VRAM in FP16, but 4-bit quantization shrinks it to ~35GB. With llama.cpp's partial CPU offloading, it can run on consumer hardware. GGUF naming conventions (Q4_K_M, Q5_K_S) tell you the precision-size tradeoff. KV cache is why long conversations slow down.
Scaling laws show that loss decreases predictably with more parameters, data, and compute — following power-law relationships. The Chinchilla paper's key finding: most models were too large and undertrained. Given the same compute budget, training a smaller model on more data produces better results. This reshaped the entire industry's training strategy.
You don't need to become a researcher to understand AI models systematically. This series starts from what you can see (tokens, context windows) and works up to self-hosting open-source models — 18 articles covering everything you need to choose models, read benchmarks, and estimate costs.
Models charge by tokens, not characters. The BPE algorithm starts from individual bytes and repeatedly merges the most frequent adjacent pair to build a vocabulary. English 'understanding' might be 1-2 tokens, but Chinese '理解' could take 2-3 — same meaning, higher cost.
Every LLM goes through three training stages: pre-training reads the internet to learn language, SFT uses example conversations to learn the format, and RLHF uses human preferences to learn what a good answer looks like. The gap between a base model and a chat model is what the last two stages do.
The core of the Transformer is self-attention: for each token, the model computes how relevant every other token is, then takes a weighted sum. This lets the model reach across distance to figure out that 'it' refers to 'cat' not 'mat' — and is the foundation for how it handles long documents.
In 2025 RAG stopped being 'retrieve once, generate once.' Search-R1 trains models to search autonomously in multiple turns with RL, REX-RAG/AlignRAG add policy and alignment branches, OpenAI Deep Research productizes the loop, and MCP generalizes retrieval into unified tool invocation. This post unpacks the design philosophy, trade-offs against ten generations, and when to adopt the new paradigm.
Apple is giving App Store Small Business Program developers free access to AFM 3 models on Private Cloud Compute if their apps have fewer than two million first-time downloads. The five-model family includes the sparse 20B-parameter AFM 3 Core Advanced, which activates only 1–4B parameters on-device, and AFM 3 Cloud Pro on Google Cloud NVIDIA GPUs, refined with outputs from Gemini.
BytePlus ModelArk Coding Plan offers Lite ($10/month) and Pro ($50/month) subscriptions covering models such as DeepSeek-V4, GLM-5.2, and Seed-2.0 in tools including Claude Code and Cursor. Lite includes about 24,000 requests per month; Pro includes five times as many.
pi's loop is a double while-loop wrapped in an EventStream; claude-code's source openly says stop_reason is unreliable and uses tool_use blocks observed during streaming as the sole continue signal; codex models a turn as a cancellable SessionTask and records sessions with a dedicated rollout crate. looplane chose an ordering — manifest first, JSONL second — that turns Ctrl-C into verified resumption instead of a rerun. All evidence cited at file#symbol level.
Mature coding-agent CLIs have converged on the same conventions: positional prompt, -p means print, exec is headless, resume is a first-class command, -C changes directory; looplane inherits this vocabulary directly, driving learning cost close to zero.
LLMs break unified diffs on bookkeeping: wrong hunk counts, hallucinated context lines. The five reference projects split into two camps — simplify the diff grammar (Codex drops line numbers), or drop diffs entirely (Claude Code/Pi/OpenCode exact replace); OMP goes further by binding read state into the format via hash anchors. looplane took the minimal-intervention path: keep the guarded apply_patch, add a zero-fuzzy replace_text, and its qwen3:4b eval went from stable failure to 5/5.
Every mature coding agent ships a machine interface: codex has `exec --json` plus a full app-server JSON-RPC protocol, claude-code has `-p` with stream-json, and pi/opencode/omp each expose a JSON event stream. Wrapping these CLIs as your backend is the fastest path to subscription-backed coding — but they own their agent loop, their login, and their permission model. looplane's answer: let the external CLI fully own its loop while looplane holds only three things — an isolated working copy, patch audit, and final verification. One runtime never impersonates another.
The ecosystem treats /v1/chat/completions as the lingua franca, but your providers don't all speak it. The five reference projects split into three camps: pi and OpenCode make the client speak every dialect natively so no gateway is needed; OMP builds a real protocol translator (foreign wire → neutral context → provider adapter, no raw passthrough); Codex and Claude Code run proxies that translate nothing and exist purely to force traffic through a controllable path. Looplane copies OMP's boundary but narrows it to one wire in, one out: strictly parse OpenAI Chat into a canonical contract, then dispatch to any ModelProvider — and along the way hit a cross-event-loop client-close bug whose lesson is that provider lifecycles belong to the ASGI lifespan, not the signal handler.
The biggest problem when an agent enters CI is approval: no TTY, nobody to click approve. The five reference projects converge on two strategies — delegate permission decisions to the calling program (claude-code's control protocol), or replace approval semantics entirely (codex defaults to Never plus sandboxing, opencode auto-rejects). looplane keeps one AgentRunner loop and injects a different ApprovalPolicy: headless uses HeadlessApprovalPolicy, which never reads stdin so it cannot hang the pipeline, and denies EXECUTE by default — fail closed.
A blank config file drives people away; a bad credential discovered too late drives them away faster. All five mature agents treat setup as a first-class state, and looplane adds the step most of them skip: verify the key right after saving it.
After an agent run finishes, 'the model said it's done' is not evidence. Codex splits traces into a manifest + JSONL + payloads bundle, omp mirrors on-disk files into SQLite, pi indexes native session files with runs.jsonl. Looplane picked the strictest option: six fixed files per run, the run is incomplete if any is missing, and patch review reads changes.patch—not anyone's verbal claim.
Five external CLIs expose five different machine interfaces: JSONL event streams, JSON-RPC handshake, HTTP API, ACP, stream-json. The right way to support them is not one interface that pretends they're identical — it's a narrow runtime boundary plus an honest capability matrix. Availability means installed, not authenticated; protocol drift fails closed.
A local sandbox limits the blast radius of an agent on your machine; a cloud sandbox is about moving code safely onto someone else's machine. All five mature projects solve the first problem; only looplane actually deployed the second. Lessons from production: mocks can't catch SSE framing, green CI can't catch a stale wheel, and cleanup paths deserve timeouts just as much as success paths.
All five agents store sessions as append-only JSONL plus some form of single-writer protection, but crash recovery lives in the details: pi repairs torn tails, codex reopens and retries after write failures, and looplane picked a 'manifest first' ordering that reduces the only crash window to one repairable slot. This post dissects each project's write ordering and fail-closed conditions, all cited at file#symbol level.
Small models don't fail at reasoning first — they fail at format stability: tool-call JSON, diff hunk arithmetic, and context budgets all break. The mature harnesses build evals on real model behavior (pi's model-backed evals, OMP calibrating benchmarks from real session logs, Codex even relaxing its parser for weaker models). looplane picks the narrowest but hardest path: one fixture, five real Ollama runs, a manifest declaring exactly which files and patch fragments count as success — and M2's failure kept verbatim as evidence. Never pass mock off as E2E; never spin partial success into full passes.
A CLI tool pays its startup cost on every invocation, and performance optimization without a baseline means no regression protection. codex uses daemon reuse and skill snapshot caches; claude-code splits its entrypoint into dynamic imports plus a built-in startup profiler; opencode and omp each maintain lazy-loading discipline; pi does none of it and leans on Bun being fast. looplane is Python — slow by birth — so it applies the full discipline: lazy imports, single-flight disk cache, background controller prewarming, and hyperfine paired benchmarks wired to a CI gate that fails on >10% regression.
The five reference projects split into three camps on subscription auth. Codex and Claude Code implement OAuth only for their own official clients and store tokens in the OS keyring. pi and OMP directly reuse Claude Code's client ID to implement Pro/Max OAuth — technically feasible, but Anthropic's docs explicitly bar third parties from offering claude.ai login without approval. OpenCode removed its bundled Pro/Max plugins entirely, the cleanest policy precedent in the ecosystem. Looplane's rules: own your grant, never scrape another CLI's credentials, accept third-party OAuth only when the provider clearly supports it, and never copy or forward credentials.
An agent's two dependencies — the LLM and external CLIs — are both non-deterministic, but mature projects separate 'the moving parts' from 'the shape of the boundary': codex fakes the Responses API with wiremock plus a scripted SSE server and pins its TUI with insta snapshots; opencode built a VCR-style http-recorder package; pi splits model-backed evals from unit tests into two vitest configs; omp wraps its edit benchmark itself in unit tests. looplane stacks four layers against external CLIs: unit tests, fake-CLI contract tests, recorded-stream integration proofs, and Textual pilot TUI tests. The methodology in one line: record real non-deterministic output, then make deterministic assertions about it.
Mature coding agent TUIs never print the event stream directly — they build a typed projection layer first and update it in place. looplane took three steps (full-screen composition, runtime-first dual modes, removing the Ask/Agent split) before two old constraints — non-streaming output and resume-without-replay — were truly lifted.
None of the five reference projects enforces 'all declared verification commands pass' at the harness level: pi leaves verification to the model, OpenCode and Codex put it in the system prompt, Claude Code uses a separate adversarial verifier subagent but as a soft contract, and only OMP's cleanse actually runs checks from harness code. looplane takes the hardest path: if files changed, every declared verification command must pass before terminal_reason=verified; with no changes, checks don't rerun (no_changes). Whether to verify is decided by code, not by the model.
None of the five mature coding agents use Python — pi/opencode/claude-code run on TypeScript, codex rewrote TS into Rust, omp bolted ~80k lines of Rust native crates onto its hot path. looplane still chose Python; the costs are startup performance and packaging, compensated by lazy imports, uv, and Cloudflare Sandbox.
Same 'knowledge graph + retrieval' label, three different bets: Microsoft GraphRAG v3.1.2 pays indexing cost for global summarization, LightRAG cuts cost with dual-level retrieval and incremental updates, HippoRAG 2 turns RAG into growing associative memory via PPR — this guide splits the trade-offs by component with four query modes, indexing pipelines, and a selection matrix.
Anthropic Contextual Retrieval uses an LLM to prefix each chunk with 50-100 tokens, cutting failure rate from 5.7% to 1.9% with rerank at ~$1.02/1M tokens; Late Chunking encodes the full 32K-window document first then mean-pools by chunk boundaries for zero extra LLM cost — the trade-off is window, latency, and update shape.
Unsloth is the fastest, most VRAM-efficient local LLM fine-tuning tool — 2× training speed and 70% less VRAM. In 2026 it added a Desktop app that bundles inference, training, image/video generation, web search, and agent integration into a complete local AI workstation.
2021 was the year Transformers decisively entered computer vision: Swin Transformer won the ICCV Best Paper award, DINO showed that a self-supervised ViT could learn object segmentation without labels, and NeRF grew from one paper into an entire subfield. CVPR and ICCV both moved fully online because of the pandemic, yet the work published that year shaped architectural choices across computer vision for years to come.
2021 was the year diffusion models surpassed GANs, self-supervised learning made theoretical breakthroughs, and reinforcement learning confronted weaknesses in its evaluation methodology. NeurIPS received a then-record 9,122 submissions, ICLR’s Score-Based Generative Modeling paper became a theoretical foundation for the diffusion ecosystem, and ICML delivered substantial work on optimization theory and the dynamics of self-supervised learning.
2021 marked NLP’s shift from fine-tuning an entire model to adapting only a small fraction of its parameters. Prefix-Tuning at ACL, LoRA on arXiv, and Prompt Tuning at EMNLP all appeared that year; ACL Rolling Review launched; and the Findings track established itself as a second publication channel.
2021 was a dividing line for major AI conferences. Transformers spread from NLP throughout computer vision and time-series research, self-supervised learning became the most common cross-conference theme, and a diffusion model won an ICLR Outstanding Paper award before anyone realized it would displace GANs. Meanwhile, GNNs and federated learning reached historic peaks in paper volume before beginning to decline.
2022 marked computer vision’s turn from recognition toward generation. Latent Diffusion Models appeared at CVPR and led to Stable Diffusion; NeRF research jumped from 25 papers in 2021 to more than 50 at CVPR alone; ConvNeXt mounted a compelling counterattack for CNNs; and ECCV in Tel Aviv set a record with 157 oral papers.
2022 was the year diffusion models took center stage, Chinchilla scaling laws rewrote large-model training, and Chain-of-Thought turned reasoning into an ability that prompts could elicit. NeurIPS passed 10,000 submissions; three of its 13 Outstanding Papers directly concerned diffusion; and Chinchilla and data pruning both challenged the belief that bigger was always better. On the eve of ChatGPT’s release, every required piece fell into place at that year’s conferences.
2022 marked NLP’s shift from demonstrating model capabilities toward aligning and controlling them. InstructGPT brought RLHF into the mainstream, Chain-of-Thought showed that prompts could unlock reasoning, and Flan 2022 matured instruction-tuning methodology. ACL and NAACL adopted ARR as their sole review path, exposing infrastructure and reviewer-load problems. ChatGPT launched at year-end and rewrote the rules of NLP research.
2022 was a turning point at major AI conferences. Diffusion models moved from emerging to mainstream, with two NeurIPS Outstanding Papers; Chinchilla rewrote scaling laws; Chain-of-Thought showed that large models could reason; and InstructGPT used RLHF to teach language models to follow instructions. When ChatGPT launched at year-end, these academic topics instantly became global news.
In 2023, computer vision moved from seeing images to understanding, generating, and controlling them. Segment Anything turned segmentation into a general zero-shot capability, ControlNet made diffusion models precisely controllable, and 3D Gaussian Splatting challenged NeRF with real-time rendering. CVPR received more than 9,000 submissions and ICCV more than 8,000 as both conferences returned to in-person events.
In 2023, LLMs took over the machine-learning conference agenda. NeurIPS received more than 12,000 submissions; both Outstanding Papers addressed large models, while runner-up DPO became a practical alternative to RLHF within two years. DreamFusion opened the text-to-3D field, ICML spotlighted LLM watermarking and learning-rate adaptation, and the Mamba preprint emerged as the first serious architectural challenger to the Transformer.
2023 was the first full academic year after ChatGPT, and LLMs rewrote the NLP conference agenda. ACL's Best Papers examined humor understanding and the propagation of political bias; an EMNLP Best Paper explained in-context learning through information flow; and the HackAPrompt competition paper also won an EMNLP Best Paper award, signaling that security research had entered the mainstream. The year's largest shift was from asking how to make models more accurate to asking how we can tell when a model is misleading us.
2023 was the first year in which LLMs comprehensively rewrote the AI research agenda. DPO received a NeurIPS Outstanding Paper Runner-Up award, ReAct became an ICLR Oral, and hallucination grew from a marginal term into a major track at every conference. Meanwhile, 3D Gaussian Splatting swept through computer vision after its SIGGRAPH debut, Mamba emerged at the end of the year to challenge the Transformer attention monopoly, and publication volume for traditional NLP pipelines began a clear decline.
In 2024, 3D Gaussian Splatting took over 3D reconstruction, video generation moved from research toward products, and vision-language models spread into specialized domains. CVPR received a record 11,500-plus submissions; its Best Papers were Google Research's Generative Image Dynamics and the UCSD/Google collaboration Rich Human Feedback for Text-to-Image Generation. ECCV gave its Best Paper award to Columbia's Minimalist Vision with Freeform Pixels, an unconventional return to the physics of optics.
ML conference submissions exploded in 2024: NeurIPS received a record 15,671 papers, while ICML and ICLR passed 9,000 and 7,000. Research shifted from training ever-larger models toward spending inference compute more intelligently, making test-time compute scaling the year's defining new direction. VAR beat diffusion with next-scale image prediction, Rectified Flow became the theoretical foundation for Stable Diffusion 3, and ICLR gave its inaugural Test of Time Award to the original VAE paper.
NLP conferences redefined themselves under LLM dominance in 2024. ACL made open science its annual theme, and four of its seven Best Papers probed fundamental limits of language models. EMNLP turned toward multilingual and cross-cultural work, with Best Papers spanning speech representations and gradient interpretability. ACL and EMNLP received more than 10,000 submissions combined, but the deeper anxiety was what remains of NLP when LLMs can perform nearly every traditional NLP task.
The defining conference keywords of 2024 were agents, alignment, multimodal LLMs, and inference-time compute. The LLM share at five major conferences doubled again after its sharp 2023 rise; agent-related terms grew 4.3 times; and diffusion models graduated from an emerging topic to a second generative-AI pillar alongside LLMs. Traditional task-oriented NLP continued to contract, while GANs almost disappeared from top venues.
2025 was a two-conference year for computer vision, with CVPR and ICCV both taking place. CVPR received a record 13,008 submissions; Best Paper VGGT turned 3D reconstruction from iterative optimization into feed-forward inference. ICCV's Marr Prize went to BrickGPT, which generates brick structures from text that can actually be assembled. 3D Gaussian Splatting displaced NeRF, video generation moved toward products, and flow models began replacing diffusion, completing several paradigm shifts in one year.
ML conferences broke every submission record in 2025 and pushed peer review to its limit. NeurIPS received 21,575 papers and used more than 20,000 reviewers; ICML passed 12,000 for the first time, and ICLR reached 11,565. Reasoning and agents were the strongest trends. One NeurIPS runner-up, the conference's only perfect-score paper, challenged whether RLVR creates new reasoning ability. Awards for Alibaba Qwen's Gated Attention and a mechanistic theory of neural scaling laws showed a community moving from scaling at all costs toward understanding why scaling works.
NLP conference submissions nearly doubled in 2025: ACL received 8,360 papers and EMNLP 8,174. China-based first authors exceeded 51% at ACL, and DeepSeek's Native Sparse Attention won Best Paper. The deeper story was an identity crisis: an ACL president said 'ACL is not an AI conference,' a quantitative study asked 'Has ACL Lost Its Crown?', and EMNLP faced questions about what still distinguished it from ACL or NAACL.
The two strongest signals at AI conferences in 2025 were reasoning papers jumping from 47 to 216, a 4.6-fold rise, and agent-related terms exceeding 150 papers with 4.3–11-fold growth. Diffusion moved from breakout topic to infrastructure; RAG became a mainstream enterprise architecture with unusual coverage across all five conferences; state-space models and world models began tracing the early 2020–2021 path of Vision Transformers. Pure prompt-engineering papers encountered reviewer fatigue.
Publishing at a top conference as an independent researcher is possible, but the numbers are harsh: single-author papers have fallen to a single-digit share, the average author count has risen from 3 to 5, and the top 20 institutions account for 35-50% of authorships. Andreas Madsen spent eight months working without pay, earned an ICLR Spotlight, and still ended up returning for a PhD. This article examines real cases, evidence of review bias, and viable paths for researchers without a large lab behind them.
An AI conference paper passes through anonymized submission, format screening, reviewer bidding and assignment, independent scores from 3-4 reviewers, an author rebuttal, AC/SAC/PC decisions, and camera-ready revision—a process lasting about 4-5 months. ACL-family conferences add ARR's rolling-review model, in which review comes before the author commits the paper to a venue.
A paper submitted to a major conference can follow three very different routes: the Main Track is the highest-threshold formal publication, Findings is the ACL family's companion venue for solid work that misses the main program, and NeurIPS created the D&B Track specifically for datasets and evaluation methodology. Their review standards, prestige, and career signals differ enough that understanding the route matters before writing the paper.
The institutional map of top AI conferences is being rapidly redrawn. Industry labs dominate frontier model development—nearly 90% of notable models came from industry in 2024—but academia remains the largest source of highly cited research. Chinese universities went from challengers to nearly half of NeurIPS paper volume in five years, while OpenAI and Anthropic have nearly vanished from conference author lists. The decoupling of publication volume from research capability is the defining signal.
Stanford Marin pre-registers a paloma macro-loss of 2.04 with a 5-rung Scaling Ladder at 1% cost, then trains 535B-A23B on 11×GB200 in public with live W&B telemetry — 847 training buckets already show the most teachable frontier run.
There's no official certificate for being an 'AI top conference.' It's a community consensus built from four independent signals — CCF-A, CORE-A*, a high Google Scholar h5-index, and a low acceptance rate — and those four signals frequently disagree. ICLR being completely absent from CCF's list is a live example.
An academic-search pipeline cannot simply concatenate five APIs: use arXiv or PubMed for domain discovery, align OpenAlex and Semantic Scholar records through DOI, PMID, and arXiv IDs, then use Crossref and PubMed relationships to check the version of record, corrections, and retractions.
AG2 continues AutoGen's ConversableAgent model: agents collaborate through messages, while GroupChatManager selects the next speaker by round robin, manual choice, randomness, or an LLM.
These seven tools are not one product category: LangGraph, MAF, and Mastra emphasize durable workflows; CrewAI and AG2 emphasize multi-agent collaboration; Pydantic AI emphasizes typed Python agents; DSPy optimizes AI programs against data and metrics. Choose the control model first.
An agent should not send the user's sentence unchanged to every search service. Classify the need as exact lookup, keyword, semantic, or fielded search; move source, date, language, and field constraints into native provider parameters; then rewrite according to zero-result, overbroad, stale, or source-mismatch symptoms.
Amazon Bedrock is more than a reseller for multiple model APIs. It brings model invocation, IAM, Regions, Knowledge Bases, Guardrails, and CloudWatch into one AWS control plane. It fits teams already on AWS that value governance overhead more than the lowest token price.
Phoenix is an MIT-licensed open-source LLM observability and evaluation platform. It collects traces with OpenTelemetry and OpenInference, turns production failures into versioned datasets, compares prompt, model, or RAG changes in experiments, then writes code, human, and LLM evaluator scores back as annotations. It is not Arize AX, and self-hosting defaults require security work.
Authenticated browser state is not a convenience setting; it is a credential that can impersonate its owner. Use a dedicated low-privilege account and isolated profile, separate reading from reversible writes and high-risk transactions, and leave MFA plus final submission to a human.
Baseten puts custom-model packaging, GPU deployment, inference engines, autoscaling, and release workflows on one platform. Its value is not another OpenAI API, but retaining runtime control while operating less GPU orchestration.
Braintrust connects versioned datasets, immutable experiments, scorers, and production traces into one evaluation loop. Its value is not another score but the ability to turn production failures into offline tests. The company announced an $80 million Series B in February 2026; its customer list is company-reported.
Brave Search API exposes five endpoint families—Web, News, Images, Videos, and LLM Context—backed by Brave's own Web index and ranking models. Its core search is not merely a Google SERP wrapper.
Browserbase combines remote Chromium, persistent Contexts, proxies, and a Session Inspector in one control plane. It operates browser fleets; it does not decide an agent's next action. As of August 2026, the company reports more than 35 million monthly browser sessions and over 10,000 customers.
Cartesia's core is Sonic real-time TTS, Ink STT, and streaming inference. Although it offers the Line voice-agent platform in 2026, buyers must still separate the model layer from telephony orchestration and design consent, retention, and fallback for cloned voices.
Cerebras can dramatically accelerate generation on supported models, but agent latency still depends on prefill, tool I/O, model quality, and platform compatibility.
Chroma manages embeddings, documents, and metadata through collections; it embeds into Python locally, uses HNSW on a single node, and separates compute from storage with object storage, SSD caches, and SPANN in distributed deployments.
Anthropic interviewed 15 startups and distilled five Claude Code operating principles: everyone ships, automate the tedium, trust but verify, build for rebuilding, prototype to productionize. ClickHouse shipped 30% more features, Clay automated 100% of bug triage, Artemis Security hit 6,000+ PRs per week.
Kitesurf is a non-Chromium browser backend in Browser Run that remains in beta. It trades pixel compatibility, persistent authenticated sessions, WebGL, and full anti-bot behavior for low CPU and memory through Workers isolates, Rust/Wasm, and stateless components.
Cloudflare Sandboxes uses a Worker as the entry point, a named Durable Object as the control plane, and a Container inside an isolated VM as the execution plane. It fits Cloudflare-native fleets of ephemeral Linux workspaces, but persistence, security boundaries, and three layers of billing remain your responsibility.
Finishing 07-280 means more than reading 24 guides: produce a search engine, supervised-model comparison, CNN/GPT-2 experiments, and a small RL-plus-MCTS system before choosing 07-380, 10-301, or a specialist course.
07-280 is CMU's new Spring 2026 AI+ML core: 24 lectures and 12 main assignments move from heuristic search and CSPs to AlexNet, GPT-2, and AlphaZero. Its public material supports self-study, but complete recordings, Canvas checkpoints, Gradescope, and staff feedback remain unavailable.
Lecture 1 uses an alien autoencoder, the scope of AI and ML, and AI history to establish the course's coordinate system: an intelligent system turns inputs into representations and decisions under uncertainty.
Lecture 2 decomposes search into a problem, frontier, and priority: UCS uses paid cost, Greedy uses estimated remaining cost, and A* combines them as `f=g+h`; tree and graph search require different optimality conditions.
Lecture 3 turns a single path into a contingent plan: minimax faces an optimal opponent, alpha-beta skips branches without changing the root value, and expectimax replaces worst-case choice with probability.
Lecture 4 exposes structure through variables, domains, and constraints, then upgrades DFS with backtracking, forward checking, AC-3, MRV, and LCV; the goal is to prove failure earlier.
Lecture 5 formulates machine learning through `X → Y`, loss, risk, and empirical risk minimization: a training set only gives average observed loss, while the real objective remains generalization over an unknown distribution.
Lecture 6 recursively grows a tree from decision stumps, measures label uncertainty with entropy, and selects splits by `I(Y;W)=H(Y)-H(Y|W)`; this is computationally practical greedy ERM, not a global optimal-tree guarantee.
Lecture 7 applies ERM to linear functions and squared loss, moves from a one-dimensional slope to `argmin ||y-Xθ||²`, and derives the normal equation when `XᵀX` is invertible.
Lecture 8 moves from a one-dimensional parabola to vector gradients and compares batch GD, SGD, and mini-batches; the learning rate determines whether updates converge, oscillate, or diverge.
Lecture 9 models P(y=1|x) with a sigmoid instead of directly predicting 0 or 1, learns parameters with cross-entropy and convex optimization, and extends naturally to softmax regression.
Lecture 10 uses φ(x) to let linear models express nonlinear functions, then controls the resulting overfitting with train/validation/test separation, L1/L2 regularization, and model selection.
Lecture 11 expands a logistic unit into a multilayer network: linear layers produce z, activations produce a, and multiple neurons jointly learn a feature transform trained through a final loss.
Lecture 12 treats a network as a computation graph: the forward pass stores intermediates, the backward pass propagates upstream gradients, and local linear, activation, and softmax rules compute every parameter gradient efficiently.
Lecture 13 separates alignment into specification, distribution shift, oversight, and corrigibility, then uses benchmark selection, leakage, and post-hoc selection experiments to show why a final paper cannot audit an autonomous research workflow.
Lecture 14 replaces dense image models with local connectivity and parameter sharing, moving from convolution, stride, padding, and pooling to AlexNet, GPU data parallelism, ResNet skip connections, and BatchNorm.
Lecture 15 splits a pretrained model into representation g and task head h: freeze g and train only the head, or fine-tune some or all parameters at a smaller learning rate depending on data volume and source-target distance.
Lecture 16 starts from likelihood p(D|θ), uses i.i.d. to factor the joint probability and logs to turn products into sums; Bernoulli MLE yields sample proportions, conditional Bernoulli yields logistic cross-entropy, and Gaussian noise yields squared error.
Lecture 17 first decides how text becomes tokens, then uses N-grams to turn sequence probability into conditional probabilities estimated from corpus counts. Tokenization is the first design decision about what a model can see.
Lecture 18 truncates the chain rule with an N-gram Markov assumption, estimates probabilities from corpus counts, and contrasts greedy, categorical, and temperature sampling. The real bottlenecks are zero probability for unseen contexts and a fixed window.
Lecture 19 builds a minimal next-token model from two embedding matrices, dot-product similarity, softmax, and cross-entropy. Shared vector parameters replace the isolated count cells of an N-gram table.
Lecture 20 expands one-token embeddings into sequences, adds positional information, derives Q/K/V scaled dot-product attention and causal masking, and assembles multi-head blocks into a GPT-2 skeleton.
Lecture 21 formulates stochastic sequential decisions as an MDP with known dynamics, defines value and Q-values through Bellman backups, and solves for an optimal policy with value or policy iteration.
Lecture 22 keeps the MDP structure but removes known transitions and rewards. TD learning updates value from one sample, and Q-learning uses an off-policy target to learn optimal action values directly.
Lecture 23 replaces a huge Q-table with Qθ(s,a): first derive a gradient update for linear features from squared TD error, then add replay data and a fixed target network to form DQN.
Spring 2026 Lecture 24 is MCTS, not Fall 2026 LLM post-training. It allocates simulations through selection, expansion, rollout, backup, and UCB, then connects policy/value heads and self-play to AlphaZero.
Lectures 1–12 form one decision pipeline: define states, moves, and objectives, then use heuristics, losses, regularization, and backpropagation to control an otherwise intractable search space.
Stage II uses HW8 and HW11 to test whether representation, computation graphs, training, transfer, and generation actually connect, rather than treating CNNs and Transformers as diagrams to memorize.
Stage III connects value, policy, bootstrapping, function approximation, and MCTS into AlphaZero: a network supplies priors and estimates, search improves decisions, and self-play creates the next training set.
Spring 2026 Lecture 1 focuses on neurons, perceptrons, connectionism, and the problem framing of deep learning. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 22 focuses on latent variables, the ELBO, the KL term, and the reparameterization trick. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 3 focuses on data distributions, hypotheses, losses, empirical risk, and their roles in generalization. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 4 focuses on gradients, learning rates, parameter updates, and the training of a linear neuron. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 5 focuses on computational graphs, the chain rule, local derivatives, and gradient reuse. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 6 focuses on non-convex loss surfaces, curvature, saddle points, and momentum's accumulated direction. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 7 focuses on the tradeoffs among full-batch, mini-batch, stochastic gradients, and second-order information. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 8 focuses on AdaGrad, Adam, regularization, BatchNorm, Dropout, and loss selection. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 9 focuses on local connectivity, weight sharing, convolution kernels, and feature maps. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 10 focuses on stride, padding, receptive fields, and multi-channel convolution. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 11 focuses on stacked convolutional architectures, feature hierarchies, and design tradeoffs. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 12 focuses on CNN training, architecture selection, and the end-to-end assembly of a vision model. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 13 focuses on sequence state, temporal unrolling, parameter sharing, and recurrent computation. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 14 focuses on backpropagation through time, gradient stability, and LSTM-style gated memory. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 15 focuses on variable-length input/output, unknown alignment, and the CTC objective. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 16 focuses on blanks, collapse rules, prefix probabilities, and approximate decoding. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 17 focuses on autoregressive factorization, conditional language models, and translation decoding. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 18 focuses on queries, keys, values, scaled dot-product attention, and the Transformer block. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 19 focuses on encoder/decoder structures, masks, residual paths, and architecture variants. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 20 focuses on scaled autoregressive models, training stages, inference, and capability boundaries. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 21 focuses on bottleneck representations, reconstruction objectives, dimensionality reduction, and representation quality. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 22 focuses on latent variables, the ELBO, the KL term, and the reparameterization trick. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 23 focuses on forward noising, reverse denoising, score or noise prediction, and sampling. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 24 focuses on the generator, discriminator, minimax objective, and training instability. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 25 focuses on message passing, aggregation, node representations, and permutation symmetry. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 26 focuses on states, actions, rewards, returns, values, and policy learning. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 27 focuses on associative memory, energy functions, fixed points, and pattern retrieval. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
Spring 2026 Lecture 28 focuses on energy-based probability models, stochastic units, the partition function, and learning difficulty. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.
CMU 11-785 Spring 2026 publishes official slides and YouTube recordings for all 28 content lectures, plus extensive bootcamps and recitations. Its HW1–HW4 specifications, starters, and evaluation still depend on Autolab, Piazza, and Kaggle.
A Claude Code- or Codex-style TUI depends on long-lived sessions, typed transcripts, and approval at tool boundaries—not a screen full of color.
Cognee is a data-to-memory pipeline: a relational store preserves sources and provenance, a vector store finds semantically similar content, and a graph store represents entity relationships, exposed through remember, recall, improve, and forget.
CS124 Winter 2026 opens by mapping a ten-week path from tokenization and classification to retrieval, speech, networks, and LLMs, while PA0 establishes the Jupyter environment used throughout the quarter.
Week 10 models the Web with anchor text, PageRank, and centrality; post-training, multilinguality, and speech belong only to a public final-deck outline labeled 2025, not the 2026 live narration.
Week 2 builds three layers: a token vocabulary with BPE, sequence comparison with dynamic-programming edit distance, and probability approximation with n-grams; PA1 turns regex and BPE into executable work.
Week 3 connects text features, sigmoid probabilities, cross-entropy loss, and gradient descent, producing a classifier whose feature contributions remain inspectable.
Week 4 builds candidates with an inverted index, ranks them with tf-idf and cosine similarity, and then connects retrieved evidence to generation; PA3 exposes RAG's inspectable retrieval half.
Week 5's public materials support the distributional hypothesis, word embeddings, and cosine similarity; the paired Social NLP lecture is unrecorded and restricted, so concrete audit methods are labeled as author extensions.
Week 6 uses public neural-network slides for weighted sums, nonlinearities, loss, and backpropagation, then a public LLM/Transformer deck labeled 2025 for decoder-only architecture without treating it as the 2026 live transcript.
Week 7's public path is PA6a: implement causal self-attention, train a small Shakespeare Transformer, sample text, and compute perplexity; the live speech lecture remains an explicit source gap.
Week 8 sends text through TTS and back through STT, requiring error classification, formatting-loss analysis, and accent stress tests, while Lab 4 prepares Git collaboration for the team agent project.
Week 9 builds movie recommendations with item-item collaborative filtering, then packages recommendation, web search, databases, and memory as agent tools under API-budget and team constraints.
Lecture 3 decomposes neural-network training into computation graphs, local derivatives, and the chain rule: the forward pass computes a result; backprop accumulates gradients from the output so every parameter knows how to move.
Lecture 11 divides evaluation into what to test, how to measure it, and when the result stops being trustworthy. Benchmarks saturate or leak, prompts change scores, and an LLM judge remains a biased model.
Lecture 9 compares prompting, pruning, LoRA, prompt tuning, and adapters. Each asks the same question: how many parameters must change, and how much task-specific state must be stored, to adapt a large pretrained model?
Lecture 6 completes the Transformer picture with encoders, decoders, and cross-attention, then breaks the final project into formats, assessment, research topics, and data. A viable topic needs one explicit baseline and metric.
Winter 2026 Lecture 1 divides NLP into four eras: early exploration, symbolic systems, statistical machine learning, and deep/self-supervised learning. The point is not the dates but how each era redefined the language problem.
Lecture 15 is Been Kim's interpretability guest session, but the Winter 2026 site publishes no slides or agenda. This article does not invent lecture content; it maps the five official readings across concept discovery, agentic investigation, and new vocabulary.
Lecture 17 is Luke Zettlemoyer's multimodality guest session, but the site publishes no slides or agenda. Its official readings establish three routes: visual reasoning workspaces, early-fusion token models, and text autoregression with image diffusion.
The final lecture frames Open Questions in NLP 2026 as smart scaling: prolonged RL, Prismatic synthetic data, RL as pretraining, and open collaboration seek reasoning gains beyond adding parameters.
Lecture 8 explains how instruction tuning, preference data, and RLHF turn a pretrained model into an assistant, then derives DPO from winner–loser pairs. Every step converts human judgment into signal—and imports its biases.
Lecture 7 decomposes pretraining into scalable data, subword tokenization, three model objectives, and in-context learning. A general self-supervised objective yields reusable representations; downstream signals specify their use.
Lecture 10 moves from question answering and RAG into language agents, then decomposes them into reasoning and planning, memory, tools, data, and evaluation. An agent is an inspectable loop between a model and external state.
Lecture 12 shows that output policy is not a detail: greedy, beam, and sampling produce different text. It then moves from R1-Zero/R1 into PPO, GRPO, and DAPO, asking when longer reasoning actually helps.
Lecture 13 moves from inference efficiency to inference capability: speculative decoding drafts with a small model and verifies with a large one; on-policy distillation addresses drift; long context and test-time scaling spend inference resources.
Lecture 4 defines a language model as a next-word probability distribution, then uses an RNN to compress an arbitrarily long prefix. It also exposes recurrence's central cost: information and gradients travel one time step at a time.
Lecture 16 divides NLP's social impact into four questions: why models hallucinate, why AI-assisted creativity may homogenize output, how work is reorganized, and why value alignment cannot be reduced to one reward.
Lecture 18 is a John Schulman guest session. The official page gives only the title Tinker and LoRA Without Regret, date, and speaker—no slides, agenda, or readings—so this article records confirmed facts and unknowns only.
Lecture 14 moves from word, character/byte, and subword segmentation to BPE failures and cross-lingual fairness. A tokenizer determines sequence length, compute cost, and the units a model sees; it is not neutral preprocessing.
Lecture 5 moves from the long-range and sequential bottlenecks of RNNs to self-attention and the Transformer. It shortens information paths and enables parallel computation, at the price of quadratic attention and separately encoded position.
Lecture 2 moves from word2vec's prediction task, objective, and gradients to count-based vectors and evaluation. Meaning becomes a high-dimensional position learned from context, not a label retrieved from a dictionary.
SPINACH does not guess complete SPARQL in one shot. It searches entities and properties, inspects Wikidata entries and examples, executes small queries, and composes a final query under explicit action and stopping rules.
CHURRO represents full-page text, layout, and metadata in HDML, unifies multilingual historical data for a page-level VLM, and connects extraction to HistoryGenie for searchable, conversational archives.
The final lecture is not a complete LLM-training tutorial. It studies data efficiency under fixed data and abundant compute, revisiting epochs, batches, ensembles, self-training, and conditions for synthetic continued pretraining.
The [WikiChat paper](https://aclanthology.org/2023.findings-emnlp.157/) expands RAG into query formulation, retrieval, filtering, generation, claim extraction, renewed retrieval and verification, and removal of unsupported content—and evaluates retrieval separately from factuality.
Fall 2025 opens with computational thinking: reliability comes from decomposing retrieval, formal representation, verification, and generation into testable algorithms, not from one heroic prompt.
STORM uses perspective-guided questions, simulated interviews, and outlines to broaden research; Co-STORM keeps a person in the loop so discovering unknown questions and co-editing become part of the system.
SLIDERS induces a question-specific schema, applies semantic chunking and contextualized extraction, reconciles duplicate rows, and answers with SUQL instead of feeding every long document directly to one model.
ReactGenie annotates React components to expose data, actions, and views, parses composite voice commands into a DSL, and renders native graphical output against shared UI context.
The lecture parses patient records and trial criteria into SMT, retrieves candidates through a weaker propositional projection, and runs a solver on the reduced set. Reasoning is inspectable, but NL-to-SMT remains the main error boundary.
Automated qualitative coding defines event types and arguments in a codebook, then separates document classification, structured extraction, and entity linking. Constrained JSON fixes form, not expert judgment.
Reliable database agents map language to executable queries, resolve schemas and enumerated values, and evaluate execution separately from answer generation; hybrid questions additionally require explicit source routing.
SUQL adds answer and summary functions over text to SQL. A semantic parser emits one hybrid query, while an optimizing compiler applies predicate pushdown, top-k pruning, and lazy evaluation.
CS224V splits task-agent evaluation into state updates and complete interaction: isolate the semantic parser, then test task completion, grounded queries, and valid actions with real users.
Genie Worksheets declare task capability as a form-like specification. A contextual semantic parser updates formal dialogue state while the runtime controls queries, actions, and responses.
Lecture 3 does not turn its survey of modern LLMs into a single best recipe. It finds a conservative consensus—pre-norm, RMSNorm, no biases, SwiGLU, and RoPE—plus a small set of deviations justified by inference cost or stability.
Lecture 4 studies two kinds of sparsity: linear/recurrent attention reduces sequence-length cost, while MoE activates only part of a model for each token. Both turn saved FLOPs into routing, balancing, communication, and kernel problems.
Lecture 14 moves raw documents through language, quality, and safety filtering; exact and near deduplication; and source mixing. Each stage reshapes model behavior, while synthetic instruction and agent trajectories extend the pipeline into executable environments.
Lecture 13 traces training sources through Common Crawl, Wikipedia, GitHub, arXiv, books, and open datasets. Technically accessible is not the same as licensed, and raw data is not training data; provenance must precede cleaning and mixing.
Lecture 12 moves from perplexity to exams, chat preferences, agents, reasoning, and safety. Every benchmark changes the capability definition, scaffold, judge, and contamination risk, so evaluation must first say whether it compares a method, model, or complete system.
Lecture 5 explains GPUs through SMs, warps, and the memory hierarchy, then unifies common optimization under low precision, fusion, recomputation, coalescing, and tiling. FlashAttention combines those principles for attention.
Lecture 10 separates prefill from decode: prefill parallelizes and is often compute-bound, while decode is sequential and commonly bandwidth-bound. GQA/MLA, quantization, speculative decoding, continuous batching, and PagedAttention reshape that cost.
Lecture 6 turns GPU principles into kernels: benchmark scaling across shapes, profile actual calls and time, then implement GeLU, softmax, reductions, and tiled matrix multiplication in Triton. Speed begins with measuring correctly.
Lecture 17 organizes CLIP/SigLIP, LLaVA, Qwen-VL, and Chameleon into three paths: contrastive encoders learn semantics, vision-encoder/projector/LM stacks provide understanding, and discrete image tokens enable generation. Resolution, token budgets, and modality balance constrain them all.
CS336's first lecture does not treat building a language model from scratch as reenacting every old technique. It separates mechanics, mindset, and intuitions, then uses BPE to show how raw bytes become trainable tokens.
Lecture 7 starts below FSDP APIs, building a communication language from broadcast, all-reduce, all-gather, reduce-scatter, and all-to-all before assembling data, tensor, and pipeline parallelism.
Lecture 8 moves from parallel primitives to system design: ZeRO progressively shards optimizer state, gradients, and parameters; TP, PP, SP, and EP split width, depth, sequence, and experts. Their composition must follow topology and dynamic activation memory.
Lecture 2 reduces model training to tensors, FLOPs, bytes, and time: use einops to track dimensions, arithmetic intensity and roofline analysis to identify bottlenecks, then trade compute for memory with gradient accumulation and activation checkpointing.
Lecture 16 moves from PPO to GRPO and RLVR. Math, code, and environment outcomes provide scalable rewards and avoid some preference-model overoptimization, but group-normalized advantages introduce difficulty and length bias while rollout infrastructure becomes the dominant cost.
Lecture 9 begins with log-log linear relationships between data and error, then uses scaling laws to compare architectures, optimizers, batches, and model-data allocations. The Chinchilla dispute shows how fitting methods, observed ranges, and deployment objectives change the answer.
Lecture 11 reads public recipes from MiniCPM, DeepSeek, Qwen, and Llama 3: hold most architectural ratios fixed, sweep learning rate and batch at small scale, then choose model/data allocation with IsoFLOPs. μP helps, but normalization, optimizers, and weight decay can break transfer.
Lecture 15 divides post-training into imitation and optimization. SFT extracts pretrained capabilities from instruction-response data; RLHF uses pairwise feedback to bridge demonstrations and preferences. PPO and DPO both inherit data bias, reward overoptimization, and mode collapse.
Daytona treats a sandbox as a long-lived computer that can start, pause, snapshot, and fork. It raised a $24 million Series A in 2026, while a Laude Institute case study reports 37,000 sandboxes in one week. It fits parallel evaluations and coding agents, but its core open-source repository is no longer maintained.
Deepgram combines streaming STT, LLM orchestration, turn detection, barge-in, and streaming TTS over one WebSocket while preserving paths for standalone speech models and bring-your-own LLM or TTS.
Dify puts models, Knowledge, visual Workflows, Agents, Plugins, and application APIs in one workspace; this guide builds a minimal Workflow that can be tested, published, and called through the API, then explains when an Agent is actually warranted.
DSPy replaces handwritten prompt strings with task Signatures, execution Modules, and Optimizers that compile better instructions and examples against a dataset and metric.
E2B combines Templates, Firecracker microVMs, and process, file, and network APIs into an agent execution layer. Its real selection advantage is preserving memory and processes across pause and resume, not merely providing another code interpreter.
ElevenLabs has expanded from a TTS vendor into the ElevenAgents platform: Scribe Realtime listens, Flash speaks, and the platform connects the LLM, turn-taking, tools, and telephony. The key choice is whether you need a voice model or the whole agent control plane.
Fireworks AI puts open-weight model evaluation, dedicated GPU deployments, and LoRA customization behind one API surface. Serverless fits low-volume starts, On-demand fits sustained traffic and custom models, while reserved capacity adds enterprise capacity guarantees.
Flowise uses Assistant, Chatflow, and Agentflow to cover simple assistants, single-agent systems, and multi-agent orchestration; however, its repository was archived in August 2026 and official EOL is scheduled for August 31, so new projects should not adopt it without a maintained fork and migration plan.
Galileo connects dataset experiments, LLM/code/Luna evaluators, production traces, and runtime guardrails. It fits enterprises that need observability plus intervention, but the old Protect surface is deprecated and evaluator models do not replace human calibration or application security.
In August 2026, it's not just five frameworks moving. Beyond OMP 2, Pi v2, Opencode 2, dsh, and Claude Code, three model makers — Google (Antigravity CLI), Meta (Muse Code), and xAI (Grok Build) — are building coding agents directly. Add Amp, Cline 2.0, and the Codex CLI Rust rewrite, and eight-plus frameworks are undergoing architecture-level changes simultaneously. Factor in 110+ total CLI tools, and H2 2026 is a divergence period for harness methodology. This article analyzes four architectural approaches, one shared direction, and one emerging trust crisis.
Haystack turns indexing, retrieval, generation, and evaluation into replaceable Components connected by directed-multigraph Pipelines; it fits Python teams that want RAG flows to be tested, versioned, and deployed as code.
Helicone is an open-source LLM gateway and observability platform: requests sent through its compatible endpoint automatically capture model, latency, tokens, cost, and custom properties, while managed credits or BYOK enable routing and fallbacks.
Hugging Face Hub is a collaboration layer for versioned models, datasets, and applications. Datasets handles data, Spaces runs demos, while Inference Providers and Endpoints provide managed inference.
Hyperbrowser packages Chrome sessions, proxies, stealth, profiles, and recordings behind managed Playwright and Puppeteer APIs. It fits agents that need to scale real-browser work quickly, while profile credentials, anti-bot compliance, and proxy bandwidth costs remain application responsibilities.
Jina Reader turns a known URL into LLM-friendly Markdown; production use still requires explicit rendering, scope, token-budget, validation, and fallback decisions.
LanceDB stores vectors, metadata, and multimodal source data in the Lance columnar format. Its OSS edition embeds in Python, TypeScript, or Rust processes; distributed Enterprise becomes relevant when the data or service outgrows one machine.
LangChain v1 provides a high-level agent loop through create_agent, runs it on LangGraph, and treats tools, structured output, and middleware as its extension boundaries.
LangSmith structures LLM applications as projects, traces, runs, and threads, then uses datasets, evaluators, and experiments to turn production failures into offline regression tests. It observes any LLM application and does not require LangChain.
Letta extends MemGPT's operating-system analogy but is not a standalone memory API. The runtime persists agent state, editable in-context blocks, conversation history, and external archival memory, while the model can actively curate memory through tools.
LiteLLM is not a model provider. It is a Python SDK and self-hosted proxy that normalizes 100+ LLM APIs, then centralizes routing, fallbacks, virtual keys, budgets, and observability at the gateway layer.
LiveKit models a voice agent as a server participant in a realtime media room, with AgentSession orchestrating STT, turn detection, LLM, TTS, and interruption. It raised a $100 million Series C at a $1 billion valuation in 2026. It fits products needing WebRTC, multiple client platforms, telephony, and swappable models, but self-hosting the media server does not self-host the entire AI pipeline.
Mastra is a TypeScript agent framework that combines agents, typed workflows, memory, MCP, tracing, and scorers in one Node.js development environment.
Mem0 sits between an agent and storage: it extracts durable facts from interactions, scopes them by user, agent, or run, and searches them before a later generation. Its appeal is a small API; its risks are extraction errors, stale memories, and authorization boundaries.
Milvus separates real-time ingestion, historical queries, index building, and persistence into independently scalable components. It fits large, continuously updated retrieval services, but smaller projects often pay too much operational complexity for that architecture.
Lecture 1 of the 2026 course builds the vocabulary shared by the rest of the course: perceptrons, forward propagation, loss, and gradient descent.
Lecture 2 of the 2026 course addresses data where order changes meaning—text, audio, and time series—and connects directly to music generation in Lab 1.
Lecture 3 of the 2026 course moves from image tensors, convolution, and pooling to recognition systems, preparing for MNIST and face detection in Lab 2.
Lecture 4 of the 2026 course separates generative from discriminative tasks, organizes VAE, GAN, and diffusion objectives, and leads into Lab 2’s DB-VAE.
Lecture 5 of the 2026 course connects agent, environment, state, action, reward, and policy into an interaction loop, introducing credit assignment and exploration.
Lecture 6 of the 2026 course places deep learning in emerging applications and real constraints, emphasizing data, outputs, evaluation, and failure conditions.
Lecture 7 of the 2026 course starts from Asimov’s literary laws and examines modern safety protocols through traces, test data, and continuous evaluation.
Lecture 8 of the 2026 course uses the scientific-discovery loop to show how simulators, AI emulators, and experiments cooperate instead of reducing science to generic prediction.
Lecture 9 of the 2026 course starts with GPU memory pressure and moves through checkpointing, offloading, ZeRO, FSDP, and multiple forms of parallelism.
In the 2026 lab, students cover tensors, autograd, and modules before turning ABC notation into character sequences for LSTM music generation.
In the 2026 lab, part 1 classifies MNIST with dense and convolutional networks; Part 2 learns a facial latent distribution with a DB-VAE and changes training sampling.
In the 2026 lab, students build chat templates and generation with LFM2-1.2B, adapt style through LoRA, and combine OpenRouter with Opik for a judge workflow.
n8n is automation-first: a webhook, schedule, or application event starts a workflow, then an AI Agent may choose tools inside it; production still requires deliberate memory, approvals, credentials, execution data, and scaling architecture.
OpenRouter exposes many models and inference endpoints through an OpenAI-compatible API, with provider ordering, failover, BYOK, and zero-data-retention controls in one routing policy.
Volcano Engine's open-source OpenViking stores agent memory, knowledge, and skills as a viking:// virtual filesystem — browsable with ls, tree, and find. Three-tier loading (L0/L1/L2) averages just 550 tokens per retrieval, boosting LoCoMo memory accuracy from 24–57% to 80–83%.
Parallel Web Systems separates Search, Extract, and Task APIs into web-access layers with different latency and cost profiles, while Basis maps citations, excerpts, and confidence to output fields.
Patronus AI treats evaluators as reusable scoring units, then applies them to offline experiments and production traces. It suits teams that want managed hallucination, safety, and multimodal evaluators, but judge scores cannot replace human labels, deterministic tests, or real security validation.
pgvector is a PostgreSQL extension, not a standalone vector database. It adds exact and approximate vector search to the same data model, transactions, and operations stack, while leaving index tuning and horizontal scaling as PostgreSQL concerns.
Portkey sits between applications and model providers: one OpenAI-compatible endpoint adds routing, fallbacks, request logs, budgets, and guardrails, with an open-source gateway available for self-hosting.
Authorization must take effect before candidate generation, while ACLs, deletion events, and source versions must propagate to every derived index; freshness needs measurable event-time SLOs too.
The first private-corpus decision is not which vector database to buy. Define data classes, trust zones, policy enforcement points, and freshness SLAs so every index, model, and observability system receives only the minimum data it is allowed to process.
The repository has a 20-query Traditional Chinese/English golden dataset, but no document-level qrels, retrieval runs, raw latency data, or executable benchmark script. Reporting Recall@k, MRR, or nDCG as measured results would therefore be dishonest; this article defines the contract needed to run them reproducibly.
Private-corpus sync is not periodic refetching. It requires stable canonical IDs, source versions plus checksums for change detection, idempotent upserts, and tombstones that propagate deletion through every index.
Promptfoo combines prompts, providers, test cases, and assertions in YAML to produce repeatable local and CI evaluation matrices, with red teaming against the same targets. It lowers the testing barrier but does not remove output variance, LLM-judge bias, or hosted data-flow concerns.
Pydantic AI models an agent as Agent[Deps, Output]: dependencies, tool inputs, and final outputs are typed, and model results must pass Pydantic validation.
R2R packages document ingestion, hybrid search, knowledge graphs, RAG, Agents, and access controls behind a REST API; it fits teams that already own their product frontend and backend and need a retrieval service.
LlamaIndex and Haystack are code-first frameworks; RAGFlow and Dify are managed application platforms; R2R packages retrieval as an API service. Choose how much control your team needs over ingestion, retrieval, and operations before choosing a tool.
RAGFlow puts document parsing, human chunk review, retrieval tests, chat, and citations in one platform; it fits layout-heavy PDFs and tables, but carries more deployment weight and platform state than a Python library.
Runloop combines isolated microVMs, reproducible images, disk branching, credential proxies, and evals in one coding-agent platform; an official case study reports more than 10,000 concurrent Devboxes in one workload.
Sail Research lets each inference request declare a completion window, scheduling patient background agents on cheaper capacity, while Sailboxes provide persistent long-running execution environments.
Reliable citation is not appending URLs to an answer. Separate URLs, content copies, and source independence, then connect atomic claims to quote spans and snapshots through a rerunnable claim-source matrix.
SerpAPI primarily manages search-results-page retrieval and parsing: select an engine, receive a structured SERP, then handle location, pagination, asynchronous polling, and validation in your application.
Serper is a third-party Google SERP API: one POST request returns structured JSON such as organic, knowledgeGraph, and peopleAlsoAsk, but production code still needs optional-field validation, URL checks, retries, and source verification.
Slack Code moves AI coding agents from individual terminals into shared Slack channels where teams can see diffs, previews, and plans in real time. But it solves management's visibility anxiety, not engineers' productivity bottleneck — the real battle is over who becomes the agent control plane.
Lecture 1 of Stanford CS221 Autumn 2025 develops operational representations and algorithmic intuition through Overview: Defining Intelligence Under Resource Constraints.
Lecture 2 of Stanford CS221 Autumn 2025 develops operational representations and algorithmic intuition through Learning I: From Computation Graphs to Linear Regression.
Lecture 3 of Stanford CS221 Autumn 2025 develops operational representations and algorithmic intuition through Learning II: Linear Classification, Features, and Cross-Entropy.
Lecture 4 of Stanford CS221 Autumn 2025 develops operational representations and algorithmic intuition through Learning III: Deep Networks as Composable Computation Graphs.
Lecture 5 models search with states, actions, successors, and costs, then uses acyclic dynamic programming to show that an efficient algorithm still solves the wrong problem when state omits information needed by the future.
Lecture 6 of Stanford CS221 Autumn 2025 follows the official material on Search II: Priorities in UCS and A* and makes its assumptions and limits explicit.
Lecture 7 of Stanford CS221 Autumn 2025 follows the official material on MDPs I: Putting Uncertainty into State Transitions and makes its assumptions and limits explicit.
Lecture 8 of Stanford CS221 Autumn 2025 follows the official material on MDPs II: Learning Q-Values Without a Transition Model and makes its assumptions and limits explicit.
Lecture 9 moves from tabular RL to function approximation, derives REINFORCE with the log-derivative identity, and connects the derivation to the executable PyTorch implementation.
Lecture 10 extends single-agent search into adversarial game trees: expectimax averages chance outcomes, minimax takes the opponent's worst case, and alpha-beta removes irrelevant branches without changing the answer.
Lecture 11 first learns game values from experience with temporal-difference updates, then moves from sequential play to simultaneous games described by mixed strategies, minimax guarantees, and Nash equilibria.
Lecture 12 builds a joint distribution from random variables and factors, then uses Bayesian-network factorization to express conditional independence and make conditioning and marginalization executable.
Lecture 13 replaces costly exact inference with Gibbs sampling: resample one variable at a time from a conditional determined by its Markov blanket, then approximate query probabilities with sample frequencies.
Lecture 14 moves from maximum-likelihood counts and Laplace smoothing with complete data to EM, which alternates posterior responsibilities for latent variables with parameter updates.
Lecture 15 separates propositional syntax from semantics: model checking defines entailment through satisfying assignments, SAT finds witnesses, and inference rules must be judged for both soundness and completeness.
Lecture 16 compresses knowledge across objects with predicates, quantifiers, and functions, then derives conclusions through substitution, unification, and definite-clause forward inference while exposing termination and completeness limits.
Lecture 17 defines a language model as a chain-rule factorization of sequence probability, compares n-gram and neural conditional models, and shows how sampling, temperature, and evaluation shape generation.
Lecture 18 classifies AI's social effects as benefits, misuse, accidents, and structural harms, then connects fairness audits, research ethics, copyright, and platform terms to accountable institutional choices.
Lecture 19 uses the Economics of AI deck to connect compute, data, distribution, and organizational complements to GDP, labor, and ideas-driven growth.
Lecture 20 is Percy Liang's fireside chat on career and research, CS221 and Stanford, and AI's future, with every attribution tied to the official video and editorial synthesis kept separate from auto-caption uncertainty.
A slide-grounded reconstruction of Fall 2025 Lecture 1, covering Course map and tools, A common language for graph data, Hand-designed features and representation learning while documenting the classroom material unavailable to self-learners.
A slide-grounded reconstruction of Fall 2025 Lecture 2, covering Encoder-decoder view, Similarity and the objective, Random walks while documenting the classroom material unavailable to self-learners.
A slide-grounded reconstruction of Fall 2025 Lecture 3, covering From fixed embeddings to deep encoders, The message-passing framework, Aggregation and update while documenting the classroom material unavailable to self-learners.
A slide-grounded reconstruction of Fall 2025 Lecture 4, covering The GNN design space, Message, aggregation, and update, GraphSAGE while documenting the classroom material unavailable to self-learners.
A slide-grounded reconstruction of Fall 2025 Lecture 5, covering Graph-data augmentation, Feature and structural augmentation, Supervision and loss while documenting the classroom material unavailable to self-learners.
A Fall 2025 slide-grounded reconstruction of Lecture 6, covering What distinguishability means, The Weisfeiler–Lehman test, An upper bound for message passing while documenting unavailable classroom material.
A Fall 2025 slide-grounded reconstruction of Lecture 7, covering The perfect-GNN thought experiment, Three levels of standard-GNN failure, Identity-aware encoding while documenting unavailable classroom material.
A Fall 2025 slide-grounded reconstruction of Lecture 8, covering Self-attention and message passing, The scope of graph attention, Positional and structural encodings while documenting unavailable classroom material.
A Fall 2025 slide-grounded reconstruction of Lecture 9, covering Heterogeneous graph schemas, Relation-specific messages, R-GCN while documenting unavailable classroom material.
A Fall 2025 slide-grounded reconstruction of Lecture 10, covering Knowledge graphs and completion, Triple scoring, TransE and relation patterns while documenting unavailable classroom material.
A Fall 2025 slide-grounded reconstruction of Lecture 11, covering Graph formulation of recommendation, The matrix-factorization baseline, Message passing in NGCF while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 12, covering Limits of the tabular pipeline, Mapping relational databases to graphs, Temporal entity graphs while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 13, covering The multi-relational bottleneck, RelGNN composite message passing, Relation-specific aggregation while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 14, covering The goal of relational foundation models, Zero-shot relational transfer, PRODIGY's prompt graph while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 15, covering Limits of transductive KG embeddings, Entity-inductive link prediction, The relation graph while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 16, covering Complementary gaps in LLMs and GNNs, Text-attributed graphs, The LLM as predictor or encoder while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 17, covering From graph QA to agents, Multimodal retrieval in STaRK, Tool use and traversal while documenting the public-material boundary.
A Fall 2025 slide-grounded reconstruction of Lecture 18, covering The graph-generation problem and representation, Evaluating generation quality, GraphRNN's autoregressive factorization while documenting the public-material boundary.
The Fall 2025 conclusion studies roughly 315K GNN designs across 32 tasks: run a small set of anchor models, derive task similarity from rankings, and transfer the best designs from similar tasks.
Linear regression is more than a best-fit line: Chapter 1 connects squared loss to gradient descent, normal equations, maximum likelihood, and locally weighted regression.
Chapter 2 derives logistic loss from a sigmoid probability model, then contrasts it with the perceptron and extends it through softmax and Newton's method.
Chapter 3 uses exponential families, natural parameters, and link functions to place least squares and logistic regression inside one modeling template.
Chapter 4 models p(x|y) and p(y), using GDA, Naive Bayes, and Laplace smoothing to expose both the power and price of generative classification.
Chapter 5 replaces high-dimensional feature inner products with kernels, letting inner-product-based linear algorithms learn nonlinear functions without constructing the features.
Chapter 6 formalizes classification confidence as geometric margin, then builds an implementable SVM through Lagrange duality, kernels, and SMO.
Chapter 7 decomposes neural networks into composable modules and uses backpropagation and vectorization to explain how deep models can be trained efficiently.
Chapter 8 decomposes test MSE into irreducible noise, squared bias, and variance, then uses uniform convergence and VC dimension to explain when training performance transfers to new data. Double descent shows why parameter count is not a universal measure of complexity.
Chapter 9 presents three controls on generalization: explicit complexity penalties, optimizer-induced implicit regularization, and model selection on data excluded from training. MAP estimation then connects a Gaussian prior to an L2 penalty.
Chapter 10 introduces unsupervised learning through k-means: alternating updates make distortion non-increasing and numerically convergent, but do not guarantee a global optimum.
Chapter 11 starts from soft assignments in Gaussian mixtures, uses Jensen's inequality to construct the ELBO, interprets EM as alternating maximization over a variational distribution and model parameters, and extends the idea to VAEs through approximate posteriors and reparameterization.
Chapter 12 formulates PCA as geometric optimization: maximize projected variance along a unit direction to obtain the leading eigenvector of the covariance matrix. The top k eigenvectors give both maximum retained variance and minimum linear reconstruction error.
Chapter 13 models ICA as x=As: observations are unknown linear mixtures, and the goal is to estimate W=A^{-1} to recover independent, non-Gaussian sources. A Jacobian determinant enters the transformed density and leads to the Bell–Sejnowski likelihood update.
Chapter 14 starts with a fixed Gaussian noising Markov chain and learns to reverse each transition. The ELBO turns reverse-kernel matching into weighted noise prediction, while the continuous-time view explains reverse drift through the score ∇log p_t.
Chapter 15 compares linear probing, full fine-tuning, and LoRA—not only by trainable parameter count, but by representation movement, data needs, and memory cost.
Chapter 16 connects representation learning to systems: contrastive objectives shape an embedding space, semantic retrieval finds neighbors in it, and RAG passes retrieved context to a generator.
Chapter 17 runs from next-token loss through Transformers, KV caches, MoE, and SFT, connecting an LLM's objective and architecture to its inference costs.
Chapter 18 separates two levers for LLM reasoning: chain of thought adds test-time computation, while verifiable rewards and policy gradients train long-reasoning behavior.
Chapter 19 uses Bellman equations to turn long-horizon decisions into one-step updates, moving from value iteration in known MDPs to model learning and continuous-state approximation.
Chapter 20 exploits linear dynamics and quadratic objectives to solve LQR, then uses DDP for local nonlinearity and Kalman filtering with LQG for partially observed state.
Chapter 21 derives REINFORCE with the log-derivative trick, then uses reward-to-go, baselines, and PPO clipping to control policy-gradient variance and update size.
Steel packages Chromium sessions, CDP, proxies, stealth, and debugging behind an Apache-2.0 browser API. Its public repository has about 7,400 stars and it entered the Stripe Projects developer preview in 2026. Self-hosting fits development and data-control needs; Cloud addresses concurrency, managed proxies, CAPTCHA, recordings, and SLAs.
Together AI puts serverless APIs for open-weight models, dedicated GPU endpoints, batch inference, and fine-tuning on one platform, letting teams validate per token before moving to reserved deployment when traffic or customization justifies it.
I turned Anthropic's session-cost advice into a global skill, and the first version made the very mistakes it was meant to prevent: a description stuffed with trigger keywords, hard thresholds based on file counts and minutes, and 'protect the main context' conflated with 'spend fewer tokens overall'. Three rounds later the entry point is one page, details live in references, numeric thresholds became four judgment dimensions, and every claim from a draft post was checked against official docs.
Vapi connects phone and web audio, STT, LLMs, TTS, tool calls, and call observability in a managed voice runtime. Providers are swappable, but Vapi's realtime orchestration is not portable. In May 2026, the company reported one million developers and announced a $50 million Series B.
Vercel Sandbox isolates untrusted code in Firecracker microVMs and integrates with Fluid compute, Active CPU pricing, and Vercel OIDC. It fits agents already running on Vercel, but network defaults, memory billing, and persistence still require deliberate design.
Vertex AI is more than the Gemini API: it puts access to 200+ models, training, evaluation, deployment, and governance under one Google Cloud control plane. Since April 2026, its products and roadmap have moved into Gemini Enterprise Agent Platform, while the Vertex AI API, documentation paths, and many resource names remain in active use.
Extraction tools cannot be compared by HTTP 200s. The same 20 URLs must be scored for body text, headings, tables, code, links, metadata, noise, latency, and cost. This article publishes the corpus, adapter contract, and gates, but no winner without a same-version raw run across all four paths.
Zep does not merely vectorize chat history. It turns episodes into entities and facts with validity time, allowing new information to invalidate an old relationship without erasing history. Graphiti is the open-source framework; Zep adds managed scale and governance.
Agent Plugins 1.0 is a packaging format that bundles Agent Skills (markdown instructions) and MCP server configs into a single directory, loadable by ChatGPT, Cursor, GitHub Copilot, Kiro, and VS Code. It's not a new protocol — it's the wrapper above protocols. Vercel initiated it, OpenAI/AWS/Microsoft/Cursor co-authored it, and Google joined on launch day. Anthropic isn't on the governance board, but MCP is a core primitive of the spec.
AgentQL replaces brittle CSS and XPath selectors with queries shaped like the data you want: `query_data` returns structured values, while `query_elements` returns interactive Playwright locators. The public Starter plan lists 50 free API calls per month, but its payment, hard-stop, and remote-browser reset rules still need to be verified in Billing.
Apify is not a single crawler. It packages scraping programs as Actors, saves reusable configurations as Tasks, triggers them with Schedules, and delivers results through Datasets. It fits teams that do not want to operate queues, schedulers, and workers, but Actor fees, compute, proxies, storage, and transfer all draw from the same platform budget.
Browser Use combines browser state, model decisions, and actions such as click, type, and extract into a repeatable loop. The open-source package favors custom tools and execution control; Cloud manages browsers, profiles, proxies, and concurrent work.
changedetection.io is a web-change signal layer: narrow the monitored content, suppress noise, and notify downstream systems only when a meaningful change occurs. It is neither a search API nor a crawler replacement.
This site covers MCP thoroughly but has never written about the layer underneath it: when your agent acts for ten thousand end users reading their own Gmail, whose database holds those refresh tokens, who rotates them, who revokes them. Composio is currently the most complete answer — MIT-licensed SDKs, a commercial hosted execution and OAuth layer. It claims 1,000+ toolkits; the managed-auth page actually lists 121 with a Composio OAuth app and 96 that require your own credentials. New pricing effective 2026-08-15: 100K free tool calls, $29/mo Pro. This post takes the authorization model down to an operational level and draws the line between wiring up MCP servers yourself and buying an integration platform.
Chroma's controlled study shows that even when it fits, a full context degrades performance. Coding agent vendors have landed on seven different responses: compact, hand off, prune, defer loading, isolate, train it into the model, or change the unit of work. Amp removed /compact outright, Atlassian argues summarization should be a last resort, and Cursor's A/B test measured a 46.9% token reduction. The three real disagreements come down to what each team is measuring.
Crawl4AI handles retrieval after a URL is known: use JsonCssExtractionStrategy for stable DOMs, and switch to LLMExtractionStrategy only when extraction needs semantic judgment or must tolerate irregular layouts.
CrewAI (GitHub 57.4k stars, MIT, PyPI 11.6M weekly downloads) defines agents by role, goal, and backstory, then groups them into crews for collaboration. Unlike LangGraph's graph-first and MAF's workflow-first approach, CrewAI is team-first — you don't draw nodes and edges, you describe who's on the team and what each person does. It fully removed its LangChain dependency in late 2024 and is now a standalone framework. The commercial side splits into the open-source package and AMP, a managed platform adding visual building, deployment, tracing, and compliance.
Exa turns every indexed web page into an embedding and retrieves by vector similarity instead of keyword matching. Official pricing as checked on 2026-08-21: $7 / 1k requests for /search (first 10 results included), $1 / 1k pages for /contents, $12–15 / 1k for the deep tiers, with $20 in free credits for new accounts. This blog's CLAUDE.md puts Exa first among cloud fetch tools, 16 of its 38 skills reference it directly, and only four existing posts mention it in passing — with zero dedicated posts. This is that post.
Firecrawl puts single-page scraping, site discovery, whole-site crawling, and JSON extraction behind one API. Cloud removes browser, proxy, and worker operations; self-hosting gives infrastructure control, but not the complete Cloud feature set.
Free access is not one model: recurring allowances, balance top-ups, rate-limited access, one-time credits, and self-hosting have different steady-state costs.
Linkup separates search depth from response shape: start most agent queries with standard + searchResults, move to deep only for multi-step browsing, and treat the monthly $20 as a balance refill rather than a new $20 grant.
LlamaIndex (51,775 GitHub stars, MIT, verified 2026-08-21) has moved its center of gravity from indexing to Workflows: the standalone llama-index-workflows package pulls 2.81M weekly PyPI downloads, more than the 1.97M of the llama-index umbrella package itself. This post covers the core abstractions, the trade-off against hand-rolling a pipeline, and a hands-on test of its defaults on Traditional Chinese text — at the same chunk_size=1024, English fits 4,645 characters and Traditional Chinese only 1,332. Plus one fact you need before choosing: the TypeScript port is archived and unmaintained.
Meilisearch turns application data into fast, typo-tolerant full-text search; the hard parts are index settings, asynchronous tasks, Chinese tokenization, access filters, and tested recovery.
Microsoft merged Semantic Kernel and its own AutoGen into Microsoft Agent Framework, which hit 1.0 GA on 2026-04-02 for .NET and Python (Go is still public preview). The absorbed autogen-agentchat has not shipped since 2025-09-30. But AG2, the fork on the original authors' side, never merged — it shipped 1.0.2 six days ago, and `pip install autogen` gets you AG2, not Microsoft. This post covers MAF's abstractions, the migration clock, and how to read the tangle of names.
MIT 6.S191's 2026 edition publishes nine lecture videos, slides, three software labs, and solutions, making it an A3 self-study course. The supplied path still depends on Google/Colab, Comet, and OpenRouter for Lab 3, while unaffiliated learners do not receive MIT credit, project feedback, or API credits.
Modal is a per-second-billed serverless GPU platform that also treats agent sandboxes as a first-class primitive (company-reported: over 1 billion sandboxes launched, more than a third of revenue). The selection question isn't how convenient it is — it's your GPU utilization. Verified 2026-08-21: Modal's A100 80GB works out to $2.50/hr against RunPod's $1.59/hr for the same card, so above 64% utilization renting your own is cheaper. But on the same day, H100 SXM is $3.95/hr on Modal against $3.99 on Lambda — on that card the premium is gone.
Qdrant is not just a place to store embeddings: define the vector schema, index frequently filtered payload fields, then add dense+sparse queries, tenant boundaries, snapshots, and monitoring to build an operable retrieval service.
Scrapling puts HTTP, Playwright browsers, CSS/XPath extraction, and a Spider API behind one Python interface. Adaptive selectors save element properties and relocate a target by similarity after a layout change, but the output still needs validation.
SearXNG is a metasearch engine, not a crawler, and it does not own a web-wide index. Based on the official 2026.8.20 documentation, this guide covers Compose installation, settings.yml, engine selection, the JSON API, and empty-result diagnosis.
Three components, one job each: SearXNG finds, Crawl4AI reads, and a thin layer of your own code glues them together. Three defaults will stop you cold — `formats` only emits HTML, `secret_key` ships as the literal string `ultrasecretkey`, and turning on the limiter blocks your own code. Also, the official install path changed in March 2026, so most tutorials online point at a repository that is now archived.
Tavily and Exa are cloud-only APIs and can't be self-hosted. What you can assemble instead is SearXNG (269 upstream engines, 82 on by default) plus Crawl4AI (78.8k stars, Apache-2.0), and the ready-made Tavily-compatible wrappers are all still double-digit-star solo projects you should not depend on. But SearXNG has no index of its own, and running it from a datacenter IP gets you empty results — those two facts decide whether self-hosting is worth it.
CS124 is the first course in Stanford's NLP branch. Its textbook is Jurafsky's own Speech and Language Processing, free online, and all nine assignment repos are public. But a banner sits on the course homepage: it will not be taught at all in AY 2026–27. And the chapter numbers the syllabus points at no longer match the August 2026 textbook.
CS221 lays AI out along one axis, and reflex models — deep learning — sit in the lowest slot, with states, variables and logic above them. When Percy Liang took over in Autumn 2025 he replaced the slides with runnable Python and wrote 'Cut constraint satisfaction problems :(' into the source of the first lecture — yet ExploreCourses and Stanford Online both still advertise constraint satisfaction as a course topic. The project has gone from 20% of the grade in 2019 to extra credit only.
CS224N has kept every course website since 2000 online. In Winter 2019, Transformers were lecture 14, taught by a guest. In Winter 2026 they are lecture 5, and every lecture after that assumes you already know them. The machine translation assignment is gone; assignment 3 now has you code a decoder-only Transformer from scratch, with pytest suites that run on your laptop.
CS224U's teaching material isn't a slide deck — it's an Apache-2.0 GitHub repo holding the lecture notebooks, all three assignments, and the grading document for the final project. But the on-campus course has skipped three straight academic years since Spring 2023, and ExploreCourses has it back on the books for Spring 2026-27. The official description still lists relation extraction and semantic parsing; the 2023 syllabus covers neither. And the data-loading cell in the first assignment breaks in a fresh environment today, on a Hugging Face compatibility change.
CS224V only became Agentic AI in the 2026–2027 catalog, and the rename changed nothing underneath: the course still translates natural language into formal semantics and constrains agents with SMT solvers and knowledge graphs instead of wiring frameworks together. Seven of the eleven mandatory readings come out of the instructor's own lab. Every slide deck is public, and the course site says outright that they are deliberately incomplete.
All six CS224W Colabs download and run today, and the first one needs only NetworkX — no PyG install at all. But the exam is 35% of the grade, the largest single piece, and it's an in-person closed-book sitting. The public recordings stop at 2021 and cover none of the current syllabus's second half: graph transformers, relational deep learning, LLM+GNN.
CS228's official prerequisite is a single line — 'basic probability theory and algorithm design and analysis' — with no named course. But ExploreCourses shows it was last offered in Winter 2024, and the next slot, Winter 2027, still has a blank instructor field. What a self-learner can actually get is cs228-notes: 16 chapters, complete, last touched in June 2025.
The three things you need to self-study CS229 run on three different clocks. The lecture notes are 278 pages and were recompiled in August 2026. The newest problem sets you can download are from summer 2020. The self-assessment Stanford Online tells you to attempt before enrolling is a PDF created in 2008. Seventeen lectures from spring 2026 are public, and the last three are mislabeled.
CS25 is Stanford's 1-unit seminar where attendance is the only homework and anyone can audit. Of the nine talks in the Spring 2026 season, the three worth your time are Albert Gu on the inductive biases of SSMs vs Transformers, Charles Frye on serving inference across thousands of GPUs, and Victoria Lin on what native multimodality still hasn't solved.
CS329Z is a new three-unit agent engineering course debuting at Stanford in Autumn 2026. Its first homework asks you to build RAG, tool calling and a ReAct loop from scratch with litellm, then rewrite the same components in DSPy and hand in the comparison. The course site lives in a public GitHub repo, and the commit log shows the assignment count dropping from three to two in mid-August — the one that got cut was 'Data for Agents'.
Of the seventeen regular CS336 lectures, only nine are executable Python programs; the other eight are PDF slide decks — and the split falls exactly along the two instructors. Assignment 1's handout carries eight 'Low-Resource Tips' for finishing it on a laptop. Assignments 2 through 5 carry none. The course page lists the hourly price of a B200; the handouts list how many B200 hours each problem needs.
Tavily exposes Search, Extract, Map, and Crawl through one web API for agents. The free plan includes 1,000 credits per month; basic, fast, and ultra-fast Search cost 1 credit each, while advanced costs 2.
vLLM is the de facto standard for self-hosted LLM inference (89,470 GitHub stars, verified 2026-08-21), built on managing the KV cache the way an OS manages paged memory. But the selection question isn't how fast it is — it's your GPU utilization. Using Red Hat's measured 793 output tokens/second, a fully saturated A100 costs roughly $0.70 per million output tokens; at 10% utilization that becomes $7, more than most cloud APIs.
A web retrieval benchmark must evaluate complete tasks, not HTTP 200s: 30 fixed cases across five failure strata and three live channels, measuring answers, citations, freshness, latency, cost, and unnecessary escalation. This article delivers the harness and gates, but no fabricated ranking while the three live channels remain unconfigured.
An agent should not open a browser for every web task: route first to Search or Fetch, then escalate on explicit signals such as status codes, weak content, JavaScript shells, authentication, or challenge pages, with retry, budget, cache, deduplication, and provenance constraints at every step.
Behavioral interviews aren't about improvisation — they're about a pre-prepared story library. AI Engineer behavioral interviews have unique focus areas: AI ethics (bias, fairness, privacy), technical decision impact narratives (why you chose this model/architecture), and experience driving ML projects across teams. Strategy: build 8-10 STAR stories, practice each until you can deliver it in under 2 minutes.
AI Engineer coding interviews aren't identical to SWE — beyond LeetCode medium, you'll face ML-flavored problems (implementing a tokenizer, writing a batch inference pipeline, handling sparse matrices). Strategy: practice LeetCode medium to 70% pass rate, then spend remaining time on numpy/pandas operations, data processing pipelines, and ML-related programming problems.
Deep learning interviews don't ask you to derive backpropagation — they test whether you can explain the design intuition behind architectures. High-frequency topics: CNN's locality and translation invariance, why the evolution from RNN to Transformer was necessary, self-attention computation and complexity, BatchNorm vs LayerNorm use cases, and common training tricks (learning rate scheduling, gradient clipping, mixed precision).
LLM Application Design is the hottest new interview topic in 2025-2026. Key focus areas: RAG pipeline chunking/retrieval/reranking design, agent tool-use and planning loops, context window management strategies, guardrails and safety design, and LLM application evaluation methods. Interviewers especially value whether you've hit real-world pitfalls.
ML fundamentals interviews don't test formula memorization — they test whether you can explain concepts intuitively and hold up under follow-up questions. High-frequency topics: the practical meaning of bias-variance tradeoff, the selection logic for L1/L2 regularization, why cross-entropy beats MSE for classification, SGD vs. Adam tradeoffs, and how precision/recall priorities differ by scenario.
The core of ML System Design interviews isn't choosing the model — it's how to turn a business objective into a system that's deployable, monitorable, and iterable. Interviewers want to see if you can: translate business goals into ML objectives, design data pipelines and feature stores, choose reasonable serving strategies, and plan monitoring and A/B testing.
MLOps interviews test whether you have experience pushing models to production. Key topics: ML pipeline CI/CD (how it differs from software CI/CD), model registry and version management, A/B testing design and pitfalls, inference scaling strategies (horizontal scaling, model compression, caching), and production monitoring and alerting design.
The dividing line in LLM interviews is whether you've actually used these things. High-frequency topics: BPE tokenization logic and multilingual challenges, pretraining objectives (CLM vs MLM), three levels of fine-tuning (full/LoRA/prompt tuning), RLHF workflow and failure modes, prompting as engineering practice, and the difficulty of LLM evaluation with current methods.
AI Engineer interviews go beyond ML — big tech emphasizes system design and coding, startups look for end-to-end delivery, and AI-native companies test LLM engineering depth. Strategy: identify your target company types first, then allocate prep time across six dimensions (ML fundamentals, system design, LLM applications, coding, paper reading, and behavioral).
Paper reading interviews don't test whether you've read that specific paper — they test whether you can quickly understand a new method and identify its limitations. AI-native companies (Anthropic, OpenAI) particularly favor this format. Strategy: practice reading a paper in 30 minutes and verbally stating contribution + limitation, build your own must-read list, and practice summarizing each paper in three sentences.
CS329A is built around the generation–verification gap: models can produce the right answer but can't tell which one it is. The conclusion the course draws about itself matters more — today's methods make models more consistent, not smarter. Nine lectures are public, out of twenty.
AIF-C01, MLA-C01, and AIP-C01 are not a difficulty ladder — they are three different job surfaces: AIF tests whether you can talk about it, MLA tests putting ML into production, AIP tests integrating someone else's foundation models into a system. But in August 2026 the choice is gated by time: the official certification page announces that the last day to take MLA-C01 in English is September 28, 2026 — 40 days from today — while MLA-C02 registration does not open until September 1 and its exam guide is unpublished. Non-English candidates (Japanese, Korean, Simplified Chinese) have a materially longer window. The same page also carries two codes, MLA-C02 and ME1-C02, with no stated relationship. And the renewal graph works backwards on ordering: passing AIP-C01 renews AIF-C01, MLA-C01, and Data Engineer – Associate for three years each.
Before comparing exam objectives there is one fact that outranks all of them: registration for the Claude certifications is open only to organizations in the Claude Partner Network — individuals cannot sign up. For those who clear that gate, four things actually decide the answer. CCAO-F ($99) does not count toward partner tier eligibility while the other three do. Claude Code is 20% of CCAR-F but only 3.1% of CCDV-F — the architect exam tests the tool far more heavily than the developer exam. CCAR-P is 28% non-technical (governance 14% plus stakeholder communication 14%), which nothing else in this series is. And CCAO-F is the cheapest but only 14% Prompting; Output Evaluation at 21% is its real spine. All four are valid 12 months, with retakes at 14 / 30 / 90 days and 4 attempts per rolling 12 months.
Across Microsoft's four AI/agent certifications, only AI-103 → AI-500 is an official ladder; everything else is positioning. Three forks decide it: whether you write Python (AI-103/AI-500 vs AB-620), whether you build or judge (AB-100 vs the rest), and whether you can actually start today — AI-500's four official learning paths currently 404, AB-620 has no practice test, AB-100 has a free one. For readers who prefer Chinese there is a fourth fork: AI-103 and AB-620 offer Traditional Chinese, AI-500 and AB-100 are English only. All four cost $165, expire after one year, and renew free but only inside a six-month window.
NVIDIA's generative AI line has four exams: NCA-GENL and NCA-GENM ($125 each, associate), NCP-GENL and NCP-AAI ($200 each, professional). Three decision inputs no other vendor forces on you. One: both professional exams still show 'Coming soon' next to Register, so any near-term plan is down to the two associates. Two: NVIDIA is the only vendor in this series whose official prep courses are all paid — real cost is exam fee plus courses, and the self-paced totals are $390 (NCA-GENL), $210 for only three of five courses (NCA-GENM), and $1,620 list price across NCP-GENL's five. Three: the official documents disagree with themselves — NCP-AAI's weights total 98% on the web page and 92% in the PDF, and two cells of NCP-GENL's web table carry misplaced text, one of it about OpenUSD. Lock-in also varies sharply: NCP-AAI is 7% NVIDIA-specific, NCP-GENL is 31% GPU and model-compression work.
The 2026 consensus for agent-built slide decks: outline-first, separate content from construction, then render to images and let a fresh-eyes subagent do visual QA. Anthropic's and OpenAI's official slides skills both converged on PptxGenJS plus a visual verification loop, and the research line (PPTAgent → PreGenie → DeepPresenter) points the same way. But two later corrections matter: PresentBench shows the widely cited PPTEval scores too generously, and SeaSlides argues the model should not write free-form HTML/SVG at all.
Governance carries more weight on these exams than most engineers expect — CCAR-P is 14% governance plus 14% stakeholder work (28% non-technical), CCAO-F is 15%, AB-100's deploy-and-govern block is 40–45%, AIF-C01 is 14% responsible AI plus 14% security/compliance/governance. But across all fifteen official exam guides in this series, not one names the EU AI Act, the NIST AI RMF, or ISO/IEC 42001; the only regulations any of them names are CCAR-P's GDPR, HIPAA, and FedRAMP. So the use of this post isn't memorizing frameworks for an exam — it's using the three frameworks as a skeleton to file six certifications' scattered governance objectives. The three split cleanly: the EU AI Act is law (fully applicable 2026-08-02, high-risk duties pushed to 2027-12-02 by the AI Omnibus), the NIST AI RMF is voluntary (GOVERN/MAP/MEASURE/MANAGE, and 1.0 is currently being revised), and ISO/IEC 42001 is a certifiable management system standard whose clauses sit behind a paywall — so this post uses only what ISO's own public page states.
The AIF-C01 exam guide moved to v1.1 on April 30, 2026, adding seven objectives at once — MCP, multi-agent patterns, context engineering, token-based pricing, and hallucination detection all became testable, and Bedrock AgentCore, Kiro, and Strands Agents joined the in-scope services. Almost every summary in circulation describes the older version. This guide builds a four-week path on the official five-domain weighting (20/24/28/14/14). Official specs: $100, 90 minutes, 65 questions (50 scored), pass at 700, valid 3 years — and it is the only exam in this series offered in Traditional Chinese.
AIP-C01 is AWS's only professional-level certification dedicated to GenAI application development, and it explicitly excludes model development, advanced ML, and feature engineering — it tests integrating someone else's foundation model into a production system. Domain weights are 31/26/20/12/11, and nearly half the skills in the heaviest domain sit in vector stores and RAG. A March 2026 refresh added Bedrock AgentCore and the beta ended March 31, so anything older is stale. Official specs: $300, 180 minutes, 75 questions (65 scored), pass at 750, valid 3 years — and passing it renews AIF-C01, MLA-C01, and Data Engineer – Associate at the same time.
A complete study guide for Claude's official architect certification (CCAR-F): five domains weighted 27/18/20/20/15, four scenarios drawn from six, common anti-patterns, and hands-on preparation. Official specs are 60 items / 120 minutes / $125 / 12-month validity / 720 to pass; registration is limited to Claude Partner Network members, and on-time renewal is free and non-proctored.
CCAR-P is the most expensive and most senior of Anthropic's four exams ($175, 63 items, 120 minutes). Integration is the heaviest domain at 19%, but what really separates it from everything else in this series is the other two: Governance, Safety & Risk Management at 14% and Stakeholder Communication & Lifecycle Management at 14% — 28% combined on compliance, risk, discovery interviews, and delivery lifecycle rather than code. The guide names GDPR, HIPAA, and FedRAMP, and its Intended Audience explicitly excludes entry-level developers and anyone doing 'prompt writing without broader system design responsibility.'
CCAO-F is the cheapest of Anthropic's four exams ($99, 60 items, 120 minutes), aimed at people who work with Claude rather than build against it. The heaviest of its seven domains is Output Evaluation and Validation at 21% — spotting hallucinations, deciding when human review is required, and adapting outputs — with Governance, Risk, and Responsible Use at another 15%. Anthropic states plainly that it is not for developers building against APIs or designing agentic systems. One easily missed limitation: this credential does not count toward Claude Partner Network tier eligibility, while the other three do.
CCDV-F is the engineer's exam among Anthropic's four certifications. The official blueprint has eight domains, and the heaviest — Applications and Integration at 33.1% — is led by Claude Application Design (8.6%) and Software Engineering Foundations (7.4%), meaning a third of the exam is API mechanics and ordinary software engineering. The counterintuitive part: Claude Code is only 3.1% and Eval only 2.6%, while the sibling Architect exam gives Claude Code 20%. Official specs: $125, 53 items, 120 minutes, pass at 720, valid 12 months, registration limited to Claude Partner Network organizations.
Google PMLE, AWS AIF-C01 and AIP-C01, Microsoft AI-103 and AI-500, and NVIDIA NCP-GENL all test how to make a GenAI application fast, cheap, and reliable — and they form a three-rung ladder: AIF-C01 asks whether you know cost scales with tokens, AIP-C01 and the two Microsoft exams ask whether you can instrument and control it, NCP-GENL asks whether you can change the model and the hardware. Three different altitudes. NVIDIA works at the kernel and quantization layer (Model Optimization 17% + GPU Acceleration 14% = 31%, the heaviest single cost/latency block in the whole series), AWS and Microsoft at the application layer (three caching tiers, token caps, chargeback), and Google at the MLOps layer (CPU/GPU/TPU evaluation, data vs model parallelism, scaling serving backends by throughput). The shared core is eight levers, but each lever becomes a different question at each altitude. This post deliberately carries no prices and no hardware specs — that is the part of this topic that rots fastest.
Google's Professional ML Engineer exam guide was rewritten in 2026: Vertex AI is renamed Gemini Enterprise Agent Platform throughout, so older study material no longer matches the product names in the questions. This guide uses the official six-section weighting as its skeleton, listing what each section tests, which official materials cover it, and what to build — plus a study schedule whose reasoning is spelled out. Official specs: $200, two hours, 50–60 multiple-choice and multiple-select questions, two-year validity, 3+ years of industry experience recommended including 1+ year on Google Cloud.
`batch_runner.py` runs thousands of prompts in parallel into ShareGPT-format tool-calling trajectories, lets each prompt name its own container image, and resumes by matching prompt content rather than index. Two quality filters run before you see the data: samples with zero reasoning are discarded, and entries calling hallucinated tool names are dropped at merge time. This is why a research lab builds a personal agent — the agent is the data pipeline.
One gateway process fronts 30-plus chat platforms and denies every user not on an allowlist or paired by DM. The scheduler adds two unusual guards: pre-dispatch validation marks a misconfigured job `blocked_config` without making a single LLM call, and the model drift guard makes unpinned jobs fail closed when the global model changes — protecting you from an hourly job quietly following you onto a paid model.
Hermes install paths come in three support tiers: macOS (Apple Silicon), Windows 10/11, Linux/WSL2, and Docker are Tier 1; Termux and Nix are Tier 2; pip, brew, AUR, and Intel Macs are explicitly unsupported — fixes for those won't be merged. On the upgrade side, `hermes update` snapshots state first, then compiles nine critical files after the pull and hard-resets the checkout if any fail to parse.
Hermes Agent is Nous Research's MIT-licensed agent framework, built around a learning loop: it writes its own skills, curates its memory, and searches past sessions with FTS5. It ships `hermes claw migrate` to move you off OpenClaw — but OpenClaw was not replaced, and both projects are still moving. This is the series opener: what it is, how it differs, and when not to pick it.
Hermes memory has hard caps: 2,200 characters for MEMORY.md and 1,375 for USER.md, and an over-limit write returns an error instead of auto-compacting, forcing the agent to make room itself. Skills are maintained by a curator that runs every 7 days after 2 hours of idle, marks skills stale at 30 days and archives at 90 — but never deletes. The switches actually worth flipping are `memory.write_approval` and `skills.write_approval`, which stage the background self-improvement writes for review.
`hermes claw migrate` imports persona, memory, skills from four locations, model and provider config, platform tokens, and the approval allowlist — but secrets are never imported silently, and even `--preset full` requires an explicit `--migrate-secrets`. What can't move (cron jobs, plugins, hooks, the multi-agent list, deep channel config) isn't discarded but parked in `~/.hermes/migration/openclaw/<timestamp>/archive/` for manual work. Coming from Claude Code or Codex is a different command: `hermes import-agent`.
Hermes supports 40+ providers, and the consumer-subscription OAuth paths are where billing surprises live: Anthropic OAuth only spends Claude Max extra-usage credits, and Claude Pro can't use it at all. Auxiliary tasks default to `provider: auto`, meaning your expensive main model does compression and vision grunt work. The fallback chain is a one-shot switch per session, not continuous retry.
Approvals default to smart mode: an auxiliary model waves through low-risk commands, auto-denies genuinely dangerous ones, and escalates the uncertain cases to you. Neither `--yolo` nor `approvals.mode: off` can disable the hardline blocklist (`rm -rf /`, fork bombs, `dd` to a physical disk), and `approvals.deny` is its user-editable counterpart, evaluated before yolo. Upstream is explicit that the threat model is an honest-but-wrong agent, not an adversarial process.
Hermes can run commands on seven backends: local, ssh, docker, singularity, modal, daytona, and vercel_sandbox. The decisive trade-off isn't performance, it's approval — local and ssh run dangerous-command checks, the other five skip them entirely because the container is treated as the boundary. Also, Docker defaults to one long-lived container shared across sessions, not a fresh environment per conversation.
The Tool Gateway routes four tool categories — web search (Firecrawl), image generation (nine FAL models), TTS (OpenAI), and cloud browser (Browser Use) — through Nous infrastructure, replacing four signups with one OAuth. It's per-tool rather than all-or-nothing, and `use_gateway: true` overrides any direct key in your `.env` — the precedence rule people most often get wrong.
Attach enough MCP servers and the tool schemas alone eat your context — upstream's extreme example is Cloudflare's ~3,300 tools, whose names alone run about 32K tokens. Hermes answers with Tool Search: MCP and non-core plugin tools collapse into three bridge tools and schemas load on demand, while core tools never defer. Separately, plugins are disabled by default and only run when named in `plugins.enabled`.
AB-100 is the architect tier of Microsoft's agent line, weighted 25-30 / 25-30 / 40-45 with deployment and governance heaviest. Its outline runs on verbs like design, recommend, and propose — it tests judgment, not configuration. Three things to know first: the scope blurb on the official exam page is wrong (it is information-protection and DLP boilerplate, which I verified verbatim), so prepare from the study guide instead; it has a free practice assessment, the only one of Microsoft's three agent credentials that does; and the 15 associate certifications it lists are described as usable, not required. Official specs: $165, English only, pass at 700, one-year validity.
AB-620 is the low-code branch of Microsoft's agent certification line — it tests agent flows, adaptive cards, computer use, MCP tools, A2A, and Fabric data agents in Copilot Studio, not Python. The three skill areas weigh 30-35 / 40-45 / 20-25, with integration the heaviest. Official specs: $165, 120 minutes, pass at 700, one-year validity, and 13 languages including Traditional Chinese — the only localized exam of Microsoft's three agent credentials. It is generally available, but the practice assessment is not out yet.
AI-103 replaces AI-102, retired June 30, 2026, and the objectives were rewritten around Microsoft Foundry — prompt flow, Azure AI Studio, Azure OpenAI Service, and Azure AI Agent Service appear nowhere in them. The five skill areas weigh 25-30 / 30-35 / 10-15 / 10-15 / 10-15, with generative AI and agents the largest. Official specs: $165, 120 minutes, pass at 700, offered in Traditional Chinese, may include interactive components — and it is valid for only one year, though renewal is a free, open-book, unproctored online assessment.
AI-500 is a rare thing among the major clouds — an expert-level certification dedicated to multi-agent systems, weighted 15-20 / 30-35 / 20-25 / 20-25, naming Agent Framework, LangGraph, Hugging Face Transformers, MCP servers on Azure Functions / Logic Apps / API Management, A2A, Key Vault, and the AI Red Teaming Agent. Three constraints come first, though: it is still in beta (scores wait for rescoring), it requires AI-103 before you can take it, and the official training is not live — the four learning paths listed on the exam page all return 404 today, and the instructor-led course opens 2026-09-30.
Microsoft AI-500, AB-620, AB-100, NVIDIA NCP-AAI, and Claude CCAR-F all test multi-agent architecture, and they overlap on seven things: orchestration topologies, A2A and MCP, per-agent identity boundaries, three-layer memory, observability and agent replay, human-in-the-loop, and guardrails at four intervention points. But four vendors use four vocabularies for the same ideas, and each exam has objectives that don't transfer — Microsoft names four context-window failure modes nobody else names, 7% of NVIDIA's is locked to NeMo and NIM, and Claude tests SDK-level details like stop_reason. One correction along the way: Google PMLE's wall-to-wall 'Agent Platform' is a Vertex AI rename, not a multi-agent domain.
Pure image understanding has flattened out — four frontier models all clear 80% on MMMU-Pro within 3 points of each other. The real differentiation is video, long-document OCR, and realtime speech, each with a different leader. But the most useful lesson from assembling these rankings is that two credible sources named different Video-MME leaders more than 10 points apart — and that July and August each turned the field over again.
NCA-GENL is usually what a job posting means by 'NVIDIA Generative AI / LLM certification.' But the official blueprint diverges sharply from the name — Core Machine Learning and AI Knowledge 30%, Software Development 24%, Experimentation 22%, Data Analysis 14%, Trustworthy AI 10% — with LLM and RAG content scattered at bullet level rather than forming a domain, alongside spaCy, NumPy, Keras, and cross validation. The other thing to know first: NVIDIA's official preparation courses all cost money ($30–$500), making it the only vendor in this series without a free official learning path. Official specs: $125, 1 hour, 50–60 items, two-year validity, English only, pass/fail with no score reported.
NCA-GENM matches NCA-GENL on price, length, and level but not on emphasis: Experimentation rises to 25% (the heaviest), Core ML drops from 30% to 20%, and two new areas appear — Multimodal Data 15% and Performance Optimization 10%. The content covers U-Net, CLIP, diffusion models, multimodal loss functions, attention maps, and NVIDIA's Riva / NeMo / Triton / ACE SDKs. Watch the cost structure: two of the five recommended courses exist only as $500 workshops with no self-paced option, so a self-study path cannot cover the official set. Official specs: $125, 1 hour, 50–60 items, two-year validity, English only.
NCP-AAI is NVIDIA's professional-level agentic AI credential — $200, 120 minutes, 60–70 items, two-year validity. Two things come first: registration is not open (the Register button carries a 'Coming soon' label), and NVIDIA's own web page and PDF study guide disagree on the weights — Deployment and Scaling is 13% on the page and 5% in the PDF, Run/Monitor/Maintain is 5% on the page and 7% in the PDF, and the two versions total 98% and 92% respectively. Both are nvidia.com. This guide treats that as a range and an uncertainty rather than picking one.
NCP-GENL is NVIDIA's professional-level LLM credential — $200, 120 minutes, 60–70 items. What separates it from every other GenAI exam is where the weight sits: Model Optimization 17% plus GPU Acceleration 14% is 31% on quantization, distillation, pruning, distributed parallelism, and CUDA profiling — not on calling APIs. Two things first: the Register button says Coming soon, so you cannot sit it yet; and two description cells in the official weight table are corrupted — Fine-Tuning is described with OpenUSD data-interchange text and Model Optimization with deployment text. I verified both verbatim; the correct descriptions are in the official PDF.
Most people assume GenAI certifications are built around prompt writing. CCAO-F gives Prompting 14% while Output Evaluation gets 21%; CCDV-F gives Prompt and Context Engineering 11.0%. What actually gets tested is structured output, injection-resistant prompting, dynamic context injection, context compression and caching, prompt lifecycle governance, and proving a prompt change helped — closer to context engineering and software engineering than to writing craft. None of the ten asks you to write a prompt on the spot; they are all multiple choice, so explaining why beats having a feel for it. The single most useful line comes from CCAR-F: when business logic must be guaranteed, 'change the prompt first' is usually the wrong answer.
Four certifications genuinely test RAG and retrieval evaluation: AWS AIF-C01 (chapters 2 and 3 total 52%, covering RAG, vector stores, and FM evaluation metrics), AWS AIP-C01 (11 of the 27 skill points in its 31% Domain 1 sit in vector storage and RAG), NVIDIA NCP-AAI (Knowledge Integration 10% plus Evaluation and Tuning 13%), and Microsoft AI-500 ('multi-agent RAG architecture' inside its 30–35% Develop area). Google PMLE contributes exactly one LLM-as-a-judge objective, and Claude CCDV-F — the developer certification people most readily assume covers RAG — has no retrieval objective across its eight domains, with Eval at just 2.6%. Includes a same-vendor foundational-vs-professional comparison, a four-vendor terminology map, non-transferable objectives, and a practice project.
OpenClaw has 386k stars to Hermes Agent's 232k, yet Hermes passed it on OpenRouter daily tokens back on 2026-05-10 (224B vs 186B). The nine self-hosted agents that appeared this year aren't nine competitors — they're nine incompatible answers to one question. CVE-2026-44112 broke OpenClaw's own sandbox, and in the Meta alignment director's inbox incident there was no attacker at all: context compaction ate the safety instruction.
The Workers AI catalog currently holds 84 models. For general chat pick glm-4.7-flash ($0.06 / $0.40 per M, 131K context), for vision pick gemma-4-26b-a4b-it ($0.10 / $0.30, 256K), for cheap high-volume steps pick granite-4.0-h-micro ($0.017 / $0.112), and for embeddings pick qwen3-embedding-0.6b or bge-m3 (both $0.012 per M). This post is updated on a schedule.
A VS Code extension open-sourced by Microsoft employees that reads your local Claude Code / Codex / OpenCode session logs. The real payload is 45 Markdown rules: prompts under 30 characters, sending the next message within 15 seconds of receiving 20 lines of AI code, instruction files over 4,000 bytes — turning 'context engineering' into numbers you can argue with.
The course lists four techniques for directing agents: instruction files, hooks, commands, subagents. The instruction file is the only one loaded in full every startup, making it config rather than memory; hooks cover what instructions can't, because a rule can be ignored and a hook cannot; commands are the only one a human triggers. The course also marks just one and a half of seven task steps as human work.
Week 1 of CS146S is 'build Claude Code in 200 lines' plus a dissection of production system prompts. The agent loop really is that small. The course slides close with four things Claude does underneath, one of them being `<system-reminder>` tags scattered everywhere to stop the model drifting — which appears in no official documentation.
Factory breaks 'can an agent work in this repo' into eight pillars and five levels, and published real scores: CockroachDB L4 (74%), FastAPI L3 (53%), Express L2 (28%). The thesis is that agent readiness approximates the density of deterministic validation loops — linters, type checkers, tests are reward signals for agents.
The course measured AI SAST false positive rates at 50–100%, against 50%+ for traditional SAST — the genuinely new problem is nondeterminism: run the same prompt twice, get different results, and you can never answer "am I done scanning?" The course lists five agent attack vectors, one of which, intent breaking, attacks the agent's plan itself.
The Agent Skills spec fits in a sentence: a directory containing a SKILL.md. The real design is three levels of progressive disclosure — only name and description load at startup, the body loads on a match, bundled files load on demand. This site's own repo carries 35 skills and 7,893 lines of SKILL.md, and startup still costs only those 35 metadata pairs.
Google deployed AutoCommenter to tens of thousands of engineers and published the whole tuning process: suppressing 17 'technically correct but low-value' rules raised the useful ratio from 54% to 66%, with 80% set as the bar for the next rollout stage. Final comment-resolution rate landed around 40%. The bottleneck in AI code review was never detection — it's volume.
How an individual connects tools is a preference; how an organization does it is governance — who can touch what data, where keys live, whose budget it lands on. Anthropic's published record of ten internal teams contains a good indicator: security engineering accounts for 50% of all custom slash commands in the entire monorepo. Adoption doesn't spread evenly; it takes off first in teams that already build their own tools.
Background agents replace 'you watch it run' with 'it finishes and opens a PR.' Every vendor's design converges on the same parts: an isolated environment, external triggers (issues, Slack, Linear), and a PR as the output. The genuinely new problem is that you become the bottleneck — five agents finish at once, five diffs queue for you, and none of them know the others exist.
Fall 2026 compresses a full week of prompting into one bullet here and adds RePPIT (Research, Propose, Plan, Implement, Test) and MCP. Two RePPIT rules are worth stealing outright: always ask for exactly two proposals, and never let the instance that wrote the code review it. On the MCP side, Anthropic measured turning tools into code calls dropping 150,000 tokens to 2,000.
Stanford CS146S's Fall 2026 syllabus compresses prompting from a full week into a single bullet, drops the terminal and UI-generation weeks, and adds Agent Skills, Agent-Ready Codebases, Background Agents, and AI-Native Team. Grading moved too: the final project fell from 80% to 50%, with 30% now on open source contributions. This series reads all ten weeks.
The final session is 'self-running, self-improving software systems.' The parts all appeared in the previous nine weeks: deterministic validation loops, skills that can be written back, background agents, centralized governance. One easily missed proportion from the slides — coding is 30% of engineering time, and running it in production is the other 70%.
Researchers initially assumed neural networks are easy to fool because they're nonlinear. That was wrong — Goodfellow's 2014 paper argues the primary cause is their linear nature, and high dimensionality lets every tiny perturbation compound. The second half covers generative models: GANs' three pathologies, and why diffusion sidesteps two of them by adding noise and learning to remove it.
A BCG experiment found a jagged frontier: inside it, AI substantially improved consultants' work; outside it, AI made results worse — and people fell asleep at the wheel. The lecture also takes a strong position: avoid fine-tuning wherever possible, because by the time you're done tuning, the next model already beats your fine-tuned version.
Andrew Ng demonstrates error analysis on a deep researcher: columns are the pipeline stages, rows are 10 to 100 queries, you only look at the ones that went badly, and you mark each cell where something broke. The percentages don't have to sum to 100%. He says it takes three or four hours and saves weeks of going the wrong direction — and the fraction of people who actually do it is far below 100%.
The third reason Go can't be learned with supervision is the interesting one: the ground truth itself is ill-defined — the strongest human doesn't play their best moves every day, and even their best move isn't optimal. The last 20 minutes map RLHF fully back onto RL: the agent is the model being fine-tuned, the action is the next token, an episode is one full generation, and the reward is extremely sparse.
Andrew Ng walks a face-recognition door system through the entire project lifecycle, and the whole lecture has one thesis: speed. He gives teams a two-day deadline, on the reasoning that 'time spent preparing data should be commensurate with the time it takes to train the model once.' It closes on a line: my job is to build something that actually works, and that is not the same as building something that works on the test set.
CS230's second lecture derives embeddings through three case studies: day/night classification teaches you to use humans as a proxy for choosing resolution, trigger-word detection teaches you to manufacture a million training examples in three hours, and face verification walks you through designing your first loss function. The final step — from supervised triplets to self-supervised pairs — is why modern models can consume billions of unlabeled images.
Ask a model what a goose looks like to it and it draws a whole flock — because the labeled data tagged a flock as 'goose,' so it thinks the flock is the label. This lecture gives seven ways to open a CNN up, then says honestly: applied to transformers, even the frontier of this research only explains two layers.
CS230's first lecture is a course overview, but Andrew Ng spends most of it on three things: why scaling works, when prompting stops being enough, and why he thinks 'don't learn to code' is one of the worst pieces of career advice ever given.
I tested 10 open-source PDF parsing tools on four scanned NTU graduate entrance exams. VLM-based tools—Firecrawl, MinerU 3.4, and Marker v2—overwhelmingly beat conventional OCR on formulas and code, but installation was the real barrier: MinerU's old package name creates dependency hell, Marker's first model download takes 10 minutes, and PaddleOCR needs a separate engine. In practice, use RapidOCR for screening and MinerU or Firecrawl for close inspection.
The 2026-08-13 release added 363,246 lines and published the For You ranking weights for the first time: favorite 0.5, reply 5.0, report −234.0. But the weights are constants — the P(action) that actually decides order comes from a 2560-dim, 8-layer transformer.
Chroma tested 18 frontier models and all of them degrade as input grows — as a cliff, not a slope. Memory failures are usually retrieval failures in disguise. And the real cost of KV cache is bandwidth, not storage: every generated token reads the whole cache.
In November 2025 three frontier labs jointly broke all 12 previously proposed prompt-injection defenses. EchoLeak's payload passed Microsoft's own dedicated classifier. So the goal is not blocking every attack — it is surviving the ones that land, and that is harness work.
The line between workflow and agent is who decides the steps — the developer at design time, or the model at run time. By that definition most LLM systems in production today are workflows. Plus a usable test for choosing between RAG and an agent.
Salesforce's number from 20,000 deployments: 90% of the work on an agent happens after launch, the reverse of traditional software. Stripe merges 1,300 PRs a week with no human-written code, and credits the environment rather than the model.
MCP governs agent-to-tool, A2A governs agent-to-agent, Skills govern reusable knowledge. The test is whether the data changes: if it changes between calls you need MCP; if it's stable enough to write down, a skill file is simpler and has no runtime that can fail on its own.
Microsoft, OpenAI, Salesforce, Stripe and three others independently say the same thing: reliability comes from the engineering around the model. And 'give the deterministic parts back to code' has been shipped as a product four separate times — Agent Script, Procedures, runtime, blueprints.
Standard RAG gives a wrong answer when it retrieves the wrong chunk, and nothing in the system will notice. Agentic RAG adds a self-check, at the cost of the evaluator paradox: the ceiling on self-correction is whatever the evaluating LLM can judge about relevance.
Every AI certification an engineer can register for in 2026, listed one by one: AWS AIP-C01 / MLA-C01 / AIF-C01, Google PMLE, Microsoft AI-103 and AI-500, all twelve NVIDIA exams, Databricks, Snowflake, Oracle's Agentic AI track, IBM watsonx, Salesforce Agentforce, GitHub GH-300, Anthropic's four Claude exams, Taiwan's iPAS AI Application Planner, plus the governance and audit line (IAPP AIGP, ISACA AAISM / AAIA, CertNexus CAIP) — prices, validity, and registration gates all checked against vendor pages. Two things that hit your wallet: Google's PMLE exam guide has renamed Vertex AI to Gemini Enterprise Agent Platform throughout, making pre-mid-2026 study material worthless, and the iPAS intermediate certificate is valid 5 years, not permanently.
Firecrawl's open-source Rust conversion library turns 14 office formats (including legacy .doc / .ppt / .xls) into GFM at a 4.7ms median — 109× faster than Docling under the same timing basis. The trade-off: it does no OCR at all.
Scans and complex layouts leave you no choice but to infer structure with a model. But the technical gap between MinerU, Marker, and Docling is far smaller than the licensing gap — MinerU needs a separate license past $20M monthly revenue, Marker's model weights need payment past a funding threshold, and only Docling is cleanly MIT. Read the LICENSE before the benchmark.
The most common mistake in feeding documents to an LLM isn't picking the wrong tool — it's picking the wrong layer. Structure already in the file goes to the conversion layer (milliseconds); text without structure goes to extraction; only inferred structure needs parsing. anydoc's 4.7ms against Docling's 513.6ms is a 109× gap, and most people jump straight to the most expensive layer.
Digital-native PDFs already contain readable text — what's missing is structure, and heuristics can recover it. PyMuPDF, pdfplumber, pypdf, and Tika do this with zero GPU and zero inference cost. The biggest selection trap isn't accuracy; it's PyMuPDF's AGPL-3.0 license.
"Digital employee" isn't a technology — it's a pricing and accountability unit. Anthropic's Project Vend had Claude actually run three shops, and found the most effective intervention wasn't a smarter model but forcing it to follow procedures. Their words: "we rediscovered that bureaucracy matters." Gartner estimates only ~130 of the thousands of vendors claiming to be agentic actually are.
Every serious image-to-video model in 2026 runs latent diffusion on a DiT backbone, so visual quality is no longer a useful axis for choosing one. The real axes are native audio, self-hostability, and dollars per second. Three widely-repeated errors worth correcting: Sora's app shut down on April 26 and its API goes on September 24; Wan 2.7 is described everywhere as Apache 2.0 open weights but no first-party source has them; Veo 3.1 officially costs $0.40/s, not the $0.75/s that circulates on review sites.
There are four paths to a 3D model in 2026: AI generation (Meshy-6 / Tripo / Rodin Gen-2.5 / Hunyuan 3D), phone scanning, Text-to-CAD, and manual modeling. Picking wrong has concrete costs — AI-generated meshes can't be dimensionally edited, Rodin's STL exports usually need repair, and Meshy's free-tier assets are public. This guide selects by what the model is actually for, with current pricing from each vendor's own page.
From MarkItDown (175k stars, MIT) to curl_cffi (6k stars), a survey of 34 open-source tools for feeding data to AI. Categorized along five axes: whole-site crawling, AI browser agents, document conversion, smart extraction, and anti-detection infrastructure. The key to selection isn't which tool is best — it's scenario matching.
Uncle Bob's 4.18M-view post of 2026/7/23 isn't a manifesto — it's a reply to an engineer who started in 1983 asking whether needing to understand code psychologically makes him old-fashioned. And he doesn't skip the code entirely: his 6/1 four-stage pipeline post says 'I spot check the code,' with thresholds of crap ≤ 6 (convention is 30) and mutation runs that kill all survivors. Plus a breakdown of his open-sourced Acceptance-Pipeline-Specification and the three metric blind spots Grady Booch names.
The dominant paradigm in 3D generation in 2026 is video diffusion feeding feed-forward 3D reconstruction, and Lyra 2.0 is the flagship of that line. But three Best Papers at CVPR 2026 point at what comes next: SAM 3D brings foundation-model-scale object reconstruction, D4RT rebuilds dynamic 4D scenes in seconds from a unified transformer, and O-Voxel replaces Gaussians with structured latents. 3DGS still rules, but surface primitives are challenging it, and pixel-space diffusion is pushing back against latent space.
Every official course platform from OpenAI, Anthropic, and Google, plus Stanford CS146S/CS336, Elements of AI, Hugging Face, MIT 6.S191 and more — scraped page by page, then re-sorted into four tiers: AI-curious, vibe coding, shipping to production, and how models actually work. Also covers self-study repos still being updated in 2026 and browser-based platforms that need no local setup, filtered by last-commit date rather than star count. The conclusion: nearly all of it is free. What is scarce is not courses, it is the judgment to pick one. And tier four will not fix your tier three problem.
HeyGen's open-source HyperFrames defines video timelines with HTML data attributes, uses headless Chrome for frame-accurate seek-and-capture, then encodes via FFmpeg to MP4. 33k stars in 3 months, Apache 2.0, 21 agent skills — AI agents write HTML to produce video, no React needed.
Loop Engineering is the practice of designing systems that automatically prompt AI agents, rather than prompting them manually. Boris Cherny runs hundreds of agents, Addy Osmani coined the term, and Blake Crosley identified verification cost as the real bottleneck — this article covers primary sources, the five building blocks, applicability boundaries, and criticisms.
From the CLI tool kin3o to the CVPR 2026 paper OmniLottie — a survey of open-source approaches for converting text and images into Lottie animations, with performance benchmarks and selection guidance.
MUSE-Autoskill (2026) introduces a five-stage skill lifecycle framework. Self-created skills achieve 60.35% (+7.16%) on SkillsBench overall, and an impressive 87.94% on tasks where skill generation succeeds — surpassing the human-authored skill ceiling. This post synthesizes six arXiv papers to map the full landscape of skill evolution research.
Even with temperature=0, LLM outputs can still fluctuate by up to 15% in practice. To rigorously compare agent changes, you need a frozen golden set, at least 3 runs per query averaged out, LLM-as-judge blind evaluation (pairwise preference flip rate reaches 35%), and paired statistical tests -- not just running each version once and going by feel.
The industry has converged on using OpenTelemetry GenAI semantic conventions to turn every LLM call and tool call into a span. Detecting the three major failure modes then splits into three tracks: faithfulness + semantic entropy for hallucinations, framework-level symbolic guardrails for tool misuse, and max steps + action hash deduplication for infinite loops — all wired into a Final / Trajectory / Single-step three-layer evaluation framework.
Agent decision-making under resource constraints is bounded rationality reborn: Rational Metareasoning uses VOC rewards to save 20-37% of tokens, BATS proves that adding budget without budget awareness is futile, FrugalGPT cascades cut costs by up to 98%, and Speculative Actions reduce latency by 20%. The three constraints ultimately converge into a single Pareto curve, and the overarching trend is moving from humans tuning knobs to models making resource-rational decisions on their own.
Three seemingly distinct agent security problems — tool output injection, trust boundaries, malicious agents — share the same root cause: LLMs flatten instructions and data into a single token stream, making them architecturally unable to distinguish between the two. Understand this through-line and you can trace every attack from EchoLeak (CVE-2025-32711, zero-click) to the Morris II AI worm, and see why 'making the model behave' doesn't work — only architectural constraints (six design patterns, CaMeL) do.
Traditional RAG is a fixed pipeline of 'retrieve then answer.' Agentic RAG splits retrieval into three decision layers: when to retrieve (FLARE uses token probabilities; Adaptive-RAG uses a complexity classifier), what to retrieve (HyDE / RAG-Fusion / decomposition / Step-back), and how to fuse (RRF k=60 then cross-encoder rerank then compression -- Anthropic measured a -67% failure rate reduction). Key counter-intuitive insight: unnecessary retrieval hurts quality -- 'deciding not to retrieve' is a first-class capability.
Automatic prompt optimization (APO) has evolved from APE/OPRO to GEPA: replacing sparse rewards with linguistic reflection, winning over GRPO by ~6pp with 4-35x fewer rollouts. Meanwhile, tool descriptions are the overlooked prompt -- small wording changes can shift tool selection rates by 10x, and Anthropic's experiments show Claude self-rewriting tool descriptions outperforms human experts. These two lines are converging: eval-driven automatic optimization is eating hand-tuned prompts.
An autonomous research agent = four controllable stages: planning (decompose into sub-questions), retrieval loop (search -> read -> reflect on gaps -> search again), evidence arbitration (>=2 independent sources, typed conflict handling), and verifiable output (sentence-level citations + independent verification pass). Two approaches: training-based uses RL to learn end-to-end when to search (Search-R1 +41%); orchestration-based uses orchestrator-worker division of labor (Anthropic internal eval +90.2%, at ~15x token cost).
Inferring another's beliefs/goals/intentions from observed behavior is called Machine Theory of Mind. Three lineages: symbolic BDI, Bayesian inverse planning, and deep learning ToMnet. The biggest controversy in the LLM era is that GPT-4 still trails humans by >10 points on ToMBench — are high scores genuine reasoning or statistical shortcuts?
At 99% accuracy per step over 100 steps, the error-free completion rate drops to just 36% -- error compounding is a structural problem, not something prompt tuning can fix. Distributed systems' supervisor trees, bulkheads, circuit breakers, sagas, and durable execution can be mapped almost one-to-one into agent orchestration. But LLMs introduce a failure class that traditional systems never had -- semantic errors that don't crash -- which require Inspector agents (recovering 96.4%) and redundancy voting (MAKER: one million steps with zero errors) to address.
Cosine similarity and relevance systematically diverge across an entire class of scenarios: negation (most IR models score at or below random on NevIR), exact identifiers, numeric thresholds, and logical combinations (SoTA models achieve recall@100 < 20 on LIMIT) -- some of these hit the theoretical ceiling of the single-vector paradigm, and switching to a larger model will not help. Recommended remedy order: hybrid BM25 -> reranker (Anthropic measured -67%) -> upstream metadata routing -> domain fine-tuning / multi-vector.
As tools scale up, selection accuracy doesn't degrade gracefully — it collapses: 4 to 51 tools drops from 43% to 2%, 10 to 100+ drops from 78% to 13.62%. The root fix is to stop stuffing everything in at once — Anthropic's Tool Search Tool uses defer loading plus retrieval to cut 85% of tokens, pushing Opus 4.5 accuracy from 79.5% to 88.1%. Description quality has conditional payoff: negligible in simple scenarios, but correctness jumps from 44% to 50% in multi-tool chaining.
Traditional Chinese RAG retrieval failures are a three-layer stack: embedding granularity defects (BGE/GTE from 0.1B to 7B all mis-rank on simple queries like 'fried chicken'), Simplified Chinese / English corpus dominance causing local vocabulary drift ('premium', 'exclusion clause' alignment is unreliable), and MTEB Chinese benchmarks being Simplified Chinese making model selection signals misleading. The fix is architectural: OpenCC normalization -> hybrid + jieba segmentation -> reranker -> local fine-tuning last -- and the prerequisite for all of it is building a Traditional Chinese eval set first.
arXiv does not perform peer review, and roughly 2% of submissions are rejected. Quality judgment relies on external signals: top venue acceptance > institution + open-source reproduction > citation quality. Includes a 20-item practical checklist and a 2026 toolbox (PWC has shut down).
Making 'chunk and embed every uploaded file automatically' the default behavior means making a decision for the LLM that it could have made itself. From Self-RAG (2310.11511) and Adaptive-RAG (2403.14403) to AgenticOCR (2602.24134), the academic trajectory is pushing three layers of decision-making -- whether to retrieve, whether to parse, and how to chunk -- from the ingestion pipeline back to the agent at conversation time.
The hard part of LLM agents is not building function calling, skills, code interpreter, and document tools individually -- it is assembling them into a system that selects the right tool, writes code when needed, decomposes tasks, verifies results, and resists prompt injection. This post organizes the key papers into six engineering decisions: function calling reliability, tool/skill selection, code-as-action, multi-step planning, skill systems, and safety plus document generation.
A2UI is an agent generative UI protocol open-sourced by Google on 2025-12-15: agents send declarative JSON describing UI intent, and clients render it natively using their own component catalog whitelist, layered on top of A2A. It launched at format v0.8 and iterated to v0.9 within three months.
browse.sh, launched by Browserbase in May 2026, is two things: a browser skill catalog and the Browse CLI. The core thesis: the bottleneck for browser agents isn't reasoning — it's amnesia. By storing learned site-specific workflows as plain-text SKILL.md files, Autobrowse cut Craigslist task costs from ~$0.22 to ~$0.12 by their own metrics. Note: this has nothing to do with the 2018 Browsh text-mode browser.
CodeGraph uses tree-sitter to extract a codebase into a local SQLite/FTS5 knowledge graph, letting AI coding agents query the graph instead of scanning files. The official end-to-end benchmark (7 repos, median of 4 runs) averages 35% cost savings and 70% fewer tool calls -- but only if the agent actually walks the graph. Delegating exploration to a file-reading subagent that ignores CodeGraph turns it into pure overhead.
Reading papers is two problems stacked together: methodology (Keshav's three-pass method, 5-10 min / 1 hour / 4-5 hours) determines how to read, and tools (arXiv HTML, alphaXiv, NotebookLM, Connected Papers, Zotero) shorten the time for each pass. AI lowers the barrier to understanding; judging correctness always stays with the human.
An MIT-licensed open-source UI automation framework from ByteDance. UI actions rely solely on feeding screenshots to a vision-language model, with no DOM parsing. A single JS API works across Web / Android / iOS / desktop. The trade-offs: each step is slower and more token-expensive, and everything hinges on the model's grounding ability. Note that Midscene retired MCP after 1.9.8 in favour of Skills + CLI.
Claude has no docx_tool or pdf_tool -- it relies on bash + file tools, plus SKILL.md instructions and pre-installed libraries like pdfplumber / python-pptx inside the container, assembling file handling capabilities from three layers.
Anthropic shipped Claude Design on 2026-04-17. On 4-28, nexu-io/open-design went public -- same artifact-first loop, Apache-2.0, runs on the 16 coding-agent CLIs you already have. Two weeks from 0.1 to 0.7, 40k+ stars. A paradigm shift that flattens AI design tools from vertical SaaS into a skill bundle.
asgeirtj/system_prompts_leaks collects the raw system prompts of 40+ AI assistants, from GPT-5.5 and Claude Opus 4.7 to Gemini 3.1 Pro, with 40.3k stars, 461 commits, and an MIT license. The value isn't in obtaining secrets -- it's in turning vendors' implicit policies into comparable engineering material. What you should study is the design decisions, not the text itself.
Anthropic's 35-page startup handbook released 2026-05-14 reorganizes Idea/MVP/Launch/Scale around agentic AI. The most valuable takeaways are 'the easier it is to build, the more important validation becomes' and treating CLAUDE.md as the first MVP artifact. The part to discount: the Launch chapter puts compliance workstreams on Cowork -- but Anthropic's own docs say Cowork doesn't write audit logs.
AI agents can operate video generation tools through three approaches — Skills, MCP Connectors, and direct APIs. Choosing the right integration method matters more than choosing the right tool.
Stop stuffing all your tool descriptions into context at session start. Let the model write code, have the runtime execute it, and let tool definitions enter context only at the import line — Anthropic's GDrive→Salesforce example dropped from ~150K tokens to 2K, and Cloudflare's 2,500-endpoint schema shrank from 1.17M to 1K.
MIT research says 95% of enterprise AI pilots yield zero return. OpenAI and Anthropic announced multi-billion-dollar joint ventures in the same week, wholesale adopting the Forward Deployed Engineer model that Palantir has used for over a decade to bring AI into the enterprise battlefield.
A survey of 11 public LLM writing pipelines, distilled into three dominant patterns: multi-agent (researcher -> writer -> critic), Karpathy LLM-wiki (raw + wiki + LLM writes, humans don't), and quality guardrails (technical verifier + never fabricate + brief gate). The Princeton GEO paper (KDD 2024) quantifies the impact: inline citations +28%, adding statistics +33%, quoting source text +41%, keyword stuffing -9%.
In May 2026, OpenAI published its internal Codex deployment practices: sandboxes define technical boundaries, approval policies determine when to pause, Auto-review delegates approval decisions to a sub-agent instead of a human, and Managed configuration lets enterprise admins enforce policies top-down. The core philosophy: zero friction for low-risk actions, mandatory review for high-risk ones.
Spin up a local OpenAI-compatible endpoint at localhost:20128 that automatically routes requests from Claude Code / Cursor / Cline / Codex / Copilot through a Subscription → Cheap → Free 3-tier fallback to 40+ providers. Built-in RTK compresses tool_result (saving 20–40% input tokens), Caveman mode compresses output, OAuth auto-refresh, multi-account round-robin — install with npm install -g 9router and two commands.
Three vendors originally took three routes: Anthropic built an extension, OpenAI built its own browser, Google welded AI into Chrome. By August 2026 there are only two — OpenAI's Atlas stopped working on 9 August, with its capabilities folded back into the ChatGPT desktop app and Codex. The remaining split is 'live alongside Chrome' versus 'be Chrome'.
Stripe Minions says 'The walls matter more than the model,' but the case studies from four Silicon Valley companies never explained how to actually build those walls. This post breaks down the 15 walls we implemented in the daodao auto-dev agent: what each wall prevents, where the files live, and what the tradeoffs are. Tier 1 is mandatory, Tier 2 strengthens governance, Tier 3 is serious governance.
A PM checks a task card in Notion → the system syncs it to a GitHub issue → writes a plan → writes code → opens a PR for human review. This post explains what the system does, what it doesn't do, and why it's feasible now — written for people who don't write code.
Build a Notion task → GitHub issue → spec PR → code PR auto-dev agent from scratch. Using the daodao case as a template, this guide walks through every step — what to do, what to verify, and how to handle problems. Notion DB schema → bin/ scaffold → two Claude Code routines → cloud env vars → staging tests.
Anthropic open-sourced 12 financial-industry Agents and 11 MCP connectors. The real takeaway isn't the Agents themselves but the layered design of 'one prompt, two runtimes' and 'pure-file extensibility.'
5 rounds of consensus to write the plan, then team mode with 5 workers running 12 tasks in parallel — with plenty of pitfalls along the way. Writing it down for my future self and anyone else trying the same thing.
DeepSeek-OCR's paper is titled Contexts Optical Compression -- OCR is just the means; what it actually validates is that 'rendering text as images and feeding them to a VLM' achieves 10x compression at 97% accuracy. This is a qualitative shift for long-context LLM and RAG token costs.
For side projects, toy demos, and RAG prototypes, nobody wants to swipe a credit card on day one. This is a verified roundup of 40+ LLM inference providers still operating as of 2026/05, tiered by whether free resources auto-replenish or are one-time grants. Each entry notes credit-card requirements, supported models, paid starting prices, and catches. Chinese-origin providers including Zhipu GLM (permanently free), Doubao (2M tokens/day), Kimi, DashScope, and the Ollama local option are all included.
A Skill is a folder with a SKILL.md. Three-layer progressive disclosure lets Claude load details only when needed, eliminating the need to re-explain preferences every conversation.
Local Deep Research is a privacy-first deep research agent built on LangChain + LangGraph, integrating 20+ search engines and 30+ research strategies. Its flagship langgraph_agent_strategy takes the LLM-autonomous tool-calling approach, offering a fundamentally different paradigm from fixed-pipeline RAG graphs.
PageIndex skips chunking, embedding, and vector storage entirely. Instead it relies on LLM reasoning over a tree-structured table of contents the LLM itself wrote, reporting 98.7% on FinanceBench in its own vendor-run evaluation. It solves a different problem than vector RAG — finding the right section in a well-structured long document.
When using AI agents like Claude Code or Cursor, built-in WebFetch / WebSearch often gets blocked by Cloudflare, geo-restrictions, or rate limits. Connecting a search MCP server is the most direct fix. This post compares the options actually available in 2026.
Groq Console is the developer portal for Groq's in-house LPU chip, offering an OpenAI-compatible API, Playground, and free tier credits. Its selling point is running open-source models like Llama, Qwen, and DeepSeek at the fastest tokens/second on the market.
goose is an open-source AI Agent maintained by the Linux Foundation's AAIF, supporting 15+ LLM providers and 70+ MCP extensions, built with Rust as a Desktop App + CLI + API. It positions itself as a vendor-neutral, self-hostable alternative to Claude Code.
For running Traditional Chinese LLM workloads on Cloudflare Workers AI, the Gemma family follows instructions more reliably than same-tier Llama models. gemma-3-12b-it was marked deprecated on 2026-05-30; the current equivalent is gemma-4-26b-a4b-it: 256K context, Vision, Function calling, at $0.10 / $0.30 per M tokens.
Karpathy proposed the llm-wiki pattern in 2026, having LLMs proactively maintain a markdown wiki instead of running RAG from scratch every time. Over 100 open-source implementations now exist, ranging from local CLI tools to serverless Telegram bots.
On 2026/4/22 OpenAI launched Workspace Agents — powered by Codex, capable of long-running cloud execution, and integrating with Slack/Salesforce/Google Drive. They are the enterprise successor to Custom GPTs.
Using Weaviate Query Agent + ColQwen multi-vector model, a single prompt built a production-grade legal contract search system in 36 hours -- this post breaks down its architecture logic, technology choices, and what you actually need to watch out for.
Cloudflare ran a Multi-Agent Code Review system internally for 30 days — 131K reviews, median 3 minutes. This post breaks down their architecture and compares it with solutions from Anthropic, GitHub, CodeRabbit, Greptile, and others.
A detailed look at OpenAI's Codex agent loop design: how prompts are constructed, how multi-turn conversations are managed, how prompt caching prevents cost explosions, and how context window auto-compaction works.
OpenAI wrapped the Codex harness as a JSON-RPC over stdio App Server, enabling VS Code, JetBrains, Web, and desktop apps to share a single agent loop. Three core primitives: Item, Turn, and Thread.
An OpenAI internal team spent 5 months with 3 people and 0 lines of hand-written code, delivering a complete product using Codex. This article distills their core lessons on AGENTS.md design, repo-local knowledge bases, architecture enforcement, and entropy management.
Agentic Engineering isn't about making AI write code faster — it's about making software move through the entire delivery pipeline faster, by using multi-agent collaboration to compress cross-team coordination friction.
Agent memory isn't a plugin — it's part of the harness itself. Pick the right memory type, estimate data volume, then decide on the technology. And finally, figure out whether you actually own that memory.
AI models rationalize their own code when reviewing it. Using three different CLIs for independent review effectively catches blind spots -- this post covers the design philosophy and practical workflow patterns behind the approach.
Agentic AI is not just autocomplete — it is an AI system capable of autonomously executing multi-step tasks. This article breaks down the five phases of the SDLC, explaining where to plug in agents at each phase, how to progress from CLI tools to full-pipeline automation, and the most valuable external resources to track right now.
Encyclopedia of Agentic Coding Patterns catalogues 190 patterns to help you make the right software decisions in the age of AI-written code — and the book itself is autonomously written and maintained by an AI agent.
GitHub Copilot Coding Agent lets you assign an Issue to Copilot, which then automatically creates a branch, writes code, runs CI, and opens a PR — all inside a cloud sandbox. The key to success is setting up AGENTS.md; without it, the agent tends to go off track. Best suited for well-defined medium-sized tasks; requires Pro+ (1,500 premium requests/month) or Enterprise plan.
A six-layer deterministic pipeline that handles everything from URL ingestion to vector embedding automatically, filtering out garbage before it enters your RAG system through an eight-dimension scoring system.
A lightweight open-source tool from Microsoft that converts PDF, Office, images, audio, and more into Markdown — purpose-built for LLM pipelines.
MCP is not going away, but its effective scope is narrower than most people think. For local development, CLI and raw API almost always beat MCP. MCP's truly irreplaceable niche is the narrow gap of 'cross-agent shared local tool layer.'
Not everyone should use a coding agent to modify code directly. AI Native teams need interface specs, test-first development, monorepo, security guardrails, human-in-the-loop, and token budget controls. Building an agent platform layer on top of coding agents and clearly redefining developer roles is the right path forward.
Autoreason replaces the traditional critique-and-revise loop with a competitive multi-version evaluation mechanism (A/B/AB + blind Borda count), solving three structural problems in LLM self-refinement: prompt bias, scope creep, and lack of restraint.
An open-source coding agent reference implementation from Vercel Labs. A three-layer architecture separates the web UI, agent workflow, and sandbox VM — designed as a starting point for teams that want to self-host their own Claude Code or Cursor Background Agent.
Claude Octopus is a Claude Code plugin that simultaneously calls Codex, Gemini, Copilot, Qwen, Ollama, Perplexity, OpenRouter, and Claude to review the same code, using a 75% consensus threshold to catch single-model blind spots. It ships with 32 personas, 48 /octo:* slash commands, 51 skills, and a Dark Factory fully autonomous spec-to-code pipeline.
LLM Council is a local Web App Andrej Karpathy built over a weekend. It sends one question to multiple LLMs simultaneously, has them anonymously peer-review each other, and then a Chairman model synthesizes a final answer. Positioned as a small tool for comparing models while studying — 99% vibe coded with no plans for long-term maintenance — but the architecture itself is a minimal ensemble LLM implementation worth studying.
Claude Managed Agents is a beta service launched by Anthropic on 2026/04/08 that provides an agent harness plus cloud container sandbox, billed per token plus $0.08/session-hour. It suits long-running async tasks and is worth exploring if you don't want to build your own agent loop and sandbox.
Agent Skills is Addy Osmani's open-source collection of 19 production-grade engineering skills that drive AI agents to follow senior engineering discipline through /spec → /plan → /build → /test → /review → /ship commands, instead of cutting corners.
Graphify uses tree-sitter AST to extract code structure, then applies LLM semantic analysis to documents and images, compressing an entire project into a queryable knowledge graph. It claims to save 71.5x tokens per query compared to reading raw files.
Claw Code is a from-scratch Rust rewrite of the Claude Code CLI, featuring 48K lines of code, 40 tools, and MIT licensing. Most remarkably, the entire project was built by multiple AI agents collaborating over just 5 days, surpassing 170K GitHub stars within a week of launch.
clawhip is a Rust daemon that routes AI coding agent events (commits, PRs, session status) to Discord / Slack, solving the observability problem of not knowing who is doing what when multiple agents run in parallel.
notebooklm-py reverse-engineers Google's batchexecute RPC protocol, letting you programmatically control NotebookLM via Python / CLI / AI Agent — including audio, video, slides, quiz generation and more.
oh-my-claudecode (OMC) adds 8 collaboration modes, 19 specialized agents, and cross-model orchestration (Claude + Codex + Gemini) on top of Claude Code, transforming a single-user CLI tool into a multi-agent development platform. Features include Deep Interview for requirement clarification, Smart Model Routing that saves 30-50% on tokens, and automatic rate limit recovery.
oh-my-codex (OMX) doesn't replace Codex CLI — it adds a structured workflow layer on top of it. From requirements clarification and plan generation to multi-agent parallel execution, four core Skills transform scattered prompt conversations into a trackable development process.
oh-my-openagent (OmO) transforms OpenCode from a single-LLM tool into a multi-model agent team — Opus as the workhorse, GPT-5.2 as the architect, Gemini for frontend, Sonnet for documentation lookup — all triggered to run in parallel with a single ultrawork keyword. With 48K stars, it is the earliest project in the UltraWorkers ecosystem to establish the multi-agent coding pattern.
An open-source Agent Harness framework from HKUDS (HKU Data Science Lab) that implements tool calling, skill loading, memory, permissions, and multi-agent collaboration as complete infrastructure, supporting Anthropic / OpenAI / GitHub Copilot API formats.
There are already 6,400+ .claude/agents/*.md files on GitHub. We dissected 4 representative projects — ChemistryTimes (content production pipeline), claude-sub-agent (document-driven development pipeline), agentic (Temporal.io DAG parallel execution), and vs-copilot-multi-agent (hook-enforced memory persistence) — plus ruflo's enterprise-grade swarm architecture, distilling 6 design patterns and 5 practical trends.
Top Silicon Valley companies are independently building internal AI coding agents that automate everything from a Slack message to a merged PR. This article deep-dives into architectures from Stripe, Ramp, Coinbase, and Spotify, then expands to cover Google, Meta, Amazon, Uber, Goldman Sachs, Walmart, and more.
Andrej Karpathy proposed a framework for compiling personal knowledge wikis with LLMs — collect raw data, have the LLM compile it into .md wiki pages, run Q&A against the wiki, and file outputs back. This post compares three practical approaches: Karpathy's knowledge vault model, the community's experience vault model, and quidproquo's blog model.
After dissecting Claude Code's 18+ caching mechanisms, I found that you can't touch provider-level prompt cache, but embedding cache, tool result cache, and entity cache are not only within your reach — they deliver even better results. Includes a complete AgentCache interface design and per-tool TTL strategy.
Every one of Claude Code's 45 tools uses a prompt() method that dynamically adjusts based on user type, feature flags, and system capabilities. Applying this pattern to a ReAct Agent, tool descriptions are dynamically generated along three dimensions: orchestrator model capability, locale, and available tools. Small models automatically get few-shot examples; large models save tokens.
Claude Code runs from $20/mo Pro to $200/mo Max 20x. Quota is a rolling five-hour window with weekly limits on top, shared across Claude on web, desktop, mobile, and the terminal. When you run out you can switch to usage credits at standard API rates rather than stopping.
Cursor CLI brings the IDE agent to the terminal with an interactive TUI and headless mode, Plan/Ask/Agent modes, Cloud Handoff, and CI/CD integration. Billing now runs on two separate usage pools: Cursor's own models (Grok 4.6/4.5, Composer 2.5) and third-party models (Pro includes $20, Pro+ $70, Ultra $400).
The paying paths on Google's side: the individual free tier and Gemini CLI access on Google AI Pro / Ultra ended 2026/6/18, leaving individuals with Antigravity CLI or their own paid API key; enterprise licenses and Google Cloud are unaffected. The zero-cost starting option now belongs to someone else.
Kiro has five tiers: Free 50 credits, Pro $20/1,000, Pro+ $40/2,000, Pro Max $100/5,000, and Power $200/10,000, with add-on credits at $0.04. Auto mode mixes models to cut cost (the same task costs 1.3x credits via Sonnet), and the spec-driven flow turns vibe coding into traceable, structured development.
Codex rides your ChatGPT subscription (Free / Go $8 / Plus $20 / Pro 5x $100 / Pro 20x $200), and since 2026/4/2 billing is token-based credits. The model line is GPT-5.6 Sol / Terra / Luna; GPT-5.4 and 5.4 mini retire from ChatGPT-signed-in Codex on 2026/8/31.
OpenCode is a free, open-source TypeScript CLI agent (MIT, ~198K GitHub stars). It supports 75+ model providers including local Ollama, allows authentication via Copilot/ChatGPT accounts, and lets you switch models mid-session without losing context. There is also a desktop app and an official Zen gateway.
A comparison of six agent CLI subscriptions (Claude Code, Cursor CLI, Codex, Kiro, Antigravity/Gemini CLI, OpenCode) plus the multi-model routing pattern — cheap models for simple work, strong models for hard work. Nearly every one of these changed its billing in the first half of 2026; this version was re-verified on 8/18.
Comparing the NVIDIA DGX Spark, Apple Mac Studio M4 Ultra, ASUS Ascent GX10, MSI AI Edge, and more — helping you find the right local inference hardware.
With multi-model routing, 70% of simple tasks are directed to cheap models, and only 10-15% of complex tasks use flagship models — saving 40-85% on inference costs in practice. This article covers the architecture and implementation of five major open-source tools.
Agent CLIs are not smarter autocomplete tools -- they are AI agents that can read your codebase, execute multi-step tasks, and operate in real environments. Claude Code, Codex CLI, Gemini CLI, OpenCode, Aider, Pi, Kiro, Amp, Cursor CLI... the tools keep multiplying, but they all share a common set of design principles -- understanding these principles is how you actually get good at using them.
Sorted by GitHub Stars, a survey of 15 mainstream AI Agent frameworks in 2026 — their positioning, key features, and ideal use cases. Not a ranking — it's a map.
Use Claude Code as an orchestrator to chain Playwright screenshots, catbox.moe image hosting, Meta Graph API publishing, and Telegram notifications — generate and publish an IG carousel from a single sentence.
llama.cpp is the most widely used local LLM inference engine, implemented in pure C/C++. It supports CPU, Metal, CUDA, Vulkan, and other backends, and uses the GGUF quantization format to run multi-billion-parameter models on consumer hardware.
TurboQuant+ is an open-source implementation of a Google Research ICLR 2026 paper that uses PolarQuant + QJL two-stage quantization to compress the KV cache by 3.8-6.4x, enabling consumer hardware to run larger models with longer contexts.
The main on-device LLMs in 2026 are Gemma 3n, Qwen 3.5 Small, Llama 3.2, Phi-4-mini, Ministral 3, and SmolLM3. Sub-3B quantized models can hit 30-50 tokens/sec on phones with 8GB RAM, but RAM, thermal throttling, and context window remain hard constraints.
2026 Q1 saw a full-blown open-source model explosion: on the LLM front, GLM-5, Kimi K2.5, and Qwen3.5 caught up with closed-source models; Embedding and Reranker are dominated by Qwen3 and BGE; speech has Voxtral TTS and Whisper V3; image has FLUX.2; and video has Wan 2.2 rivaling Sora. This is the complete navigation map.
In 2025-2026, websites need to be readable not just by humans but by AI. From llms.txt and Schema Markup to GEO and RAG ingestion pipelines, this post maps out the complete technical landscape for turning your website into an AI-consumable data source.
A Harness is more than just an LLM wrapper. Tool Registry manages dynamic tool loading and selection, Guard System establishes a four-layer defense network, and Checkpoint-Resume enables long-running tasks to survive interruptions. These three patterns form the critical infrastructure of production-grade Agent systems.
A Skill is a prompt template you invoke manually. A Subagent is an independent agent that Claude routes to automatically. They look similar, but differ completely in trigger mechanism, tool isolation, and context management.
When AI agents can turn intent into a PR in minutes, the bottleneck in software engineering flips from 'planning what to do' to 'evaluating whether the output is correct.' Artifacts of the ticketing era — sprints, story points, backlog grooming — are collapsing to zero, replaced by review as the core practice.
The same model produces dramatically different results under different harness designs. Anthropic uses a dual-agent architecture, cross-session state files, and a GAN-inspired generator-evaluator loop to let Claude autonomously complete hours-long software development tasks.
Google outlined eight multi-agent design patterns: from the simplest Sequential Pipeline to the composable Composite Pattern. More complexity isn't always better — picking the right pattern matters more than stacking agents.
AI engineering has gone through three phases: Prompt Engineering (write better instructions) → Context Engineering (feed the right information) → Harness Engineering (design the entire working environment). Each evolution doesn't replace the previous one — it operates at a higher level of abstraction.
The agent loop is a serialized per-session run. The part worth studying is how it handles concurrency: an admitted run records an activeWriterRunId claim, every transcript write supplies expectedWriterRunId, and the commit transaction verifies the match — so a superseded run cannot commit stale data.
OpenClaw builds its own system prompt for every run; there is no runtime default prompt. What it builds is split by an internal cache boundary — the stable workspace prefix above, the per-turn channel context below — so backends with prefix caches can reuse the same prefix across channels.
SecretRefs keep credentials out of plaintext config, and the model-call chain sees process-local sentinels instead of the real value. But the docs say it plainly: this is not process isolation — the real value still exists in the same process's memory, and any plaintext file the agent can read bypasses the whole mechanism.
Cron is now called Automations (openclaw cron remains an alias), and automation spans six mechanisms. The core trade-off is one line: Automations give you exact timing and isolated execution, Heartbeat gives you full main-session context on a roughly-every-30-minutes cadence.
Standing orders grant an agent permanent operating authority for a defined program, written into AGENTS.md and injected into every session. They define what it may do; automations define when — and the automation prompt should reference the standing order rather than duplicate it.
Every enterprise channel is a plugin now, including Slack and Google Chat, which used to be built in. Slack has three transports — Socket Mode, HTTP Request URLs, and relay — and the docs say plainly that the first two have reached feature parity, so you pick by deployment shape, not by features.
Each channel has one gotcha that stops you cold: WhatsApp's login is QR-only and hard to do remotely, Telegram bots ship with Privacy Mode on so they never see group messages (and you must remove and re-add the bot after changing it), and Discord needs Message Content Intent or it receives nothing from servers.
The most interesting entry here is Reef — an end-to-end-encrypted side channel between OpenClaw agents owned by different people. Messages are sealed on your machine, screened in both directions by a pinned-model guard, and the relay operator can never read the content. It ships bundled.
OpenClaw supports 31 chat channels, but only WebChat lives in core — even Slack and WhatsApp are plugins you install. And group safety has two independent axes: allowlists govern who can trigger the agent, not which quotes and history the model sees. That second one is contextVisibility, and it defaults to wide open.
OpenClaw validates config strictly — one unknown key, a wrong type, or an invalid value and the Gateway refuses to start. It keeps a last-known-good copy, but neither startup nor hot reload restores it automatically; only doctor --fix does.
The Gateway binds to loopback by default, and binding anywhere else requires auth — that is enforced, not advised. Inside a detected container the effective default is auto, unless Tailscale serve/funnel is active, which always forces loopback.
Deploying OpenClaw to the cloud comes down to four decisions: where the Gateway binds, where state lives, who can reach it, and how you recover. Which platform you pick is the least important of them.
OpenClaw has six local install methods, and what separates them is not the command but whether you want reproducibility, isolation, or self-updating. The real blocker is package-manager lifecycle-script policy: both npm 12 and global pnpm installs block OpenClaw's build scripts by default.
OpenClaw's failover runs in two stages: rotate auth profiles within the provider, then fall back to another model. But what really governs behavior is who chose the model — a model you picked yourself with /model is strict, and its failure is reported rather than answered by some other model.
OpenClaw's hard requirement for a model is tool use plus a large enough context — onboarding only auto-suggests a local model when it confirms tool support and at least a 16K context window. The easier thing to get wrong is that provider, model, and agent runtime are three separate layers: an `openai/*` ref does not mean Codex.
The official provider directory now lists 60 entries. The most common failure when attaching a local model is writing Ollama's base URL with /v1 — that breaks tool calling, and the model starts emitting raw tool-call JSON as plain text.
An agent is a complete persona scope — its own workspace, auth profiles, model registry, and session store. But the isolation is not absolute: when a secondary agent's OAuth credential expires, OpenClaw reads through to the main agent's profile of the same id, and a workspace is only a default working directory, not a hard sandbox.
The best part of remote node execution is how approval binds: exec prepares a canonical systemRunPlan before approval, and once granted the gateway forwards that stored plan — not any later caller-edited command, cwd, or session fields — and re-validates the working directory before running.
OpenClaw has 200+ docs. This article helps you see the big picture, understand what each section covers, and decide where to start based on your role.
"OpenClaw is a Gateway shell around Pi" is obsolete. The docs now say the built-in runtime id is openclaw, that pi is a legacy alias which normalizes to it, and that no external agent framework packages remain. The only Pi-related third-party dependency left is a terminal component toolkit.
Node is the required runtime because the canonical state store uses node:sqlite — Bun is only for installing dependencies. Windows changed the most: there is now a native Windows Hub companion app that installs without administrator privileges and can provision its own app-owned WSL distro for the Gateway.
The iOS and Android apps are nodes, not Gateways: they do not run the Gateway service, and Telegram or WhatsApp messages land on the Gateway rather than on the phone. The Apple Watch is the exception — because watchOS blocks generic low-level networking for ordinary apps, it uses signed HTTPS polling instead.
The official framing is to treat plugin installs like running code — ClawHub and the bundled catalog are trusted sources, while arbitrary npm, git, and local paths require --force in noninteractive installs. And verification means inspect --runtime, because a bare inspect is only a cold manifest check.
Sandboxing is governed by three independent settings: mode (when it applies), scope (how many containers), and backend (where it runs). The most common failure is an expectation gap — `tools.exec.host` now defaults to auto, so 'unset means sandboxed' is no longer true, and the security audit has a check specifically for it.
By default every DM lands in one main session, and group activity and background work report back into it. Memory is entirely Markdown on disk — the model only remembers what gets saved, with no hidden state. But if more than one person can DM your agent, DM isolation is something you have to turn on.
OpenClaw's security docs open by stating the scope: this is a personal-assistant trust model, one gateway per trusted operator. It explicitly is not a security boundary for mutually adversarial users sharing one agent — and a 'not vulnerabilities by design' list pins that down.
OpenClaw's browser is a separate agent-only profile, fully isolated from your personal browser. And web_search's return shape carries an externalContent.untrusted marker — search results are typed as untrusted external content at the type level.
exec is a mutating shell surface: disabling write, edit, and apply_patch does nothing to make it read-only. And since sandboxing is off by default, host=auto actually resolves to the gateway — if you really want the sandbox, say so explicitly and it will at least fail closed.
Skills load from six sources with the highest precedence winning on name collisions, and a per-agent list replaces rather than merges. Sub-agents get no session or message tools by default — they return plain text to the parent, and the right to speak to a human stays with the parent agent.
When the tool catalog no longer fits in the prompt, OpenClaw offers two answers: Code Mode shows the model only exec and wait and has it write small programs against a hidden catalog, while Tool Search keeps structured search/describe/call controls. Neither bypasses tool policy.
The official triage flow is seven commands and two minutes to a diagnosis. And the most common symptom — the assistant feeling limited or missing tools — is usually the tool profile: minimal allows only session_status, while coding is the default for new local configs.
The Control UI gained a session rail: it uses a utility model to produce a run digest and attaches a read-only companion thread, so you can ask what a session is doing without entering or interrupting the main agent run. Its contents never enter chat.history.
The model is the CPU, the harness is the operating system, and the agent is the application. No matter how powerful a model is, without a good harness it's just a demo. Phil Schmid argues that harness is the most critical infrastructure in AI engineering for 2026.
LangGraph models LLM workflows as directed graphs, solving the pain points of multi-turn iteration, conditional branching, and parallel execution that are difficult to handle with linear pipelines.
Langfuse is currently the most mature open-source LLM Observability platform. This post covers four core capabilities — Tracing, Prompt Management, Evaluation, and Datasets — showing you how to use them in real projects.
Context Engineering is the core concept that replaced Prompt Engineering in 2025: the focus shifted from 'how to ask' to 'what information to provide.' Delivering the right information at the right time into the context window is more effective than upgrading to a stronger model. This post covers the definition, four key strategies, practical techniques, and common failure modes.
Every AI tool has its own calling format, making integration costly. MCP (Model Context Protocol) is an open standard proposed by Anthropic that unifies the communication protocol between AI Agents and external tools/data sources, enabling tools to be reused across Agents.
RAG is read-only. Agent Memory lets AI not only read but also write and persist information. Three memory types: Procedural (behavior patterns), Episodic (temporal events), and Semantic (factual knowledge) form a complete cognitive memory system.
AI Agent is not a single technology -- it is an entire architecture system. This article is a systematic navigation: starting from the Agent Three Pillars (Context/Cognition/Action), through the three-stage evolution of AI engineering (Prompt -> Context -> Harness), to eight Multi-Agent design patterns and production-grade Harness infrastructure. Each topic links to a dedicated deep-dive article.
An AI agent is not a black box — it is built from three layers: what it knows (Context), how it thinks (Cognition), and what it can do (Action). Understanding these three layers is the key to grasping why agents are sometimes brilliant and sometimes go off the rails, and how to design a truly effective agent system.
A single RAG Agent handling all queries hits knowledge boundaries and performance bottlenecks. Multi-Agent RAG dispatches retrieval tasks to multiple specialized Agents, each with its own knowledge base and retrieval strategy, coordinated by a central Orchestrator that merges results.
Traditional RAG splits documents into small chunks for retrieval, but this causes information fragmentation. LongRAG leverages 100K+ token long-context models to retrieve larger document segments (entire sections or even whole documents), reducing fragmentation while maintaining retrieval efficiency.
Speculative RAG uses small specialist models to generate multiple answer drafts from different document subsets in parallel, then a large model verifies and selects the best answer in one pass. The paper reports +12.97 points accuracy and -50.83% latency on PubHealth — but that is the best cell in the table; other benchmarks gain far less.
Ollama wraps llama.cpp in a Docker-style CLI + REST API, letting you run LLMs locally with a single command. This post covers core concepts, installation, API, hardware requirements, Modelfile customization, and what this tool is — and isn't — good for.
RAG has evolved far beyond simple 'search + generate' into a technology ecosystem spanning ten generations — and since 2025 into an Agentic/Reasoning era. This article is a systematic navigation guide: from Naive RAG to Multi-Agent/LongRAG across ten generations, the post-ten Agentic Era (Search-R1/RL search, MCP, GraphRAG 3.x, vision-native retrieval), retrieval strategies, chunking, embedding, reranking, evaluation frameworks, observability, and cost optimization. Each topic has a dedicated deep-dive article.
vLLM uses PagedAttention to eliminate KV cache memory waste, combining continuous batching and prefix caching to become the most widely adopted open-source LLM inference engine today.
Building a chatbot is more than just calling an API. Conversation state management, memory mechanisms, streaming, guardrails, observability, and tech stack selection — every layer affects the user experience.
Good prompts aren't written in one go — they're iterated into existence. Start with the simplest prompt, test with real cases, classify error types, and make targeted fixes. This article covers the three-part System Prompt structure, reasoning framework selection, few-shot optimization, token budget management, and six common mistakes.
For complex multi-hop questions, a single RAG search isn't enough. Agentic RAG lets the LLM evaluate whether retrieved results are sufficient — if not, it rewrites the query and searches again, forming a ReAct loop.
Your choice of embedding model directly determines RAG search quality. BGE-M3's multilingual training, 1024-dimensional vectors, and matching Reranker make it a practical pick for Traditional Chinese RAG.
Chunks too large and retrieval loses precision; too small and you lose context; hit a table and retrieval falls apart entirely. Chunking is the most underrated part of RAG — pick the wrong strategy and no amount of downstream optimization will save you.
Bi-Encoders are too coarse, Cross-Encoders are too slow — ColBERT's Late Interaction finds the sweet spot: token-level comparison between query and document, but with document vectors that can be precomputed.
When you split a document into chunks, each chunk loses its place in the original document. Contextual Retrieval solves the isolated-chunk problem by generating a per-chunk context from the whole document and prepending it at index time.
Filters too strict and getting zero results? CRAG automatically relaxes them and retries — far better than letting the LLM hallucinate an answer from general knowledge.
Vector search similarity scores don't equal relevance. Cross-Encoders use pairwise comparison to reorder results and push the truly relevant documents to the top.
Vector search finds similarity; graph search traverses relationships. When a question requires reasoning across multiple entities — crag → route → sender → grade distribution — GraphRAG outperforms standard RAG.
Vector search handles semantics; BM25 handles keywords. Combining them with RRF is what lets you handle both fuzzy queries and exact terms at the same time.
Have an LLM generate an 'ideal answer' first, then embed that hypothetical document for search — it outperforms searching with the raw query.
After each conversation, asynchronously extract likely user preferences and skill level, then automatically personalize search parameters on the next query — no manual setup required.
Ranking purely by relevance leaves you with five documents all describing the same route. MMR strikes a balance between relevance and diversity, and layering in popularity weighting makes results even more useful.
RAG doesn't have to be a rigid three-step process. It's a set of steps that can be dynamically enabled, skipped, or reordered. Pipeline as Code lets the system adapt its behavior without redeployment.
A single vector search on a complex query often misses relevant documents. Let the LLM rewrite the query into 3-5 sub-queries, run them in parallel, and recall improves significantly.
Climbing routes carry a ton of visual information (topos, wall photos) that text-only RAG misses entirely. Multimodal RAG makes images searchable and understandable.
Naive RAG works but has real problems. Advanced RAG patches those problems. Modular RAG rearchitects the whole system to be composable and configurable. Understanding all three generations is the key to understanding why modern RAG systems look the way they do.
For complex queries, have the LLM map out what information is needed and in how many steps — then execute that plan. More systematic than thinking on the fly.
Not every question needs full RAG. Classify queries with an LLM first, then route to the right execution path — saving cost and improving accuracy.
"Adding a Cross-Encoder feels better" is not a scientific evaluation. A/B testing tells you whether a change actually works, how much it helps, and which query types benefit.
A RAG system needs data to answer questions, but data only accumulates as the system gets used. Cold-start strategy is what bridges the gap from empty to useful.
RAG system costs come from LLM tokens, Embedding APIs, and vector search. Every stage has room for cost reduction, but you need to verify that optimizations don't sacrifice too much quality.
No industry standard mandates one RAG evaluation tool. Measure retrieval, generation, and operations separately, then choose Promptfoo, RAGAS, DeepEval, or TruLens for the actual stack.
When a RAG system breaks, 90% of the time it's one of these 10 failure modes. Identify which one first, then apply the matching fix — far more effective than optimizing blindly.
The attacks RAG systems face go beyond the technical level — Prompt Injection and Jailbreak are real threats. Both inputs and outputs need independent protection layers.
Rolling your own traces is good enough, but open-source tools save you a lot of work. Langfuse, Phoenix, and LangSmith each have their niche — the right choice depends on your trade-offs around self-hosting, open source, and integration complexity.
The hardest part of a RAG system isn't building it — it's figuring out why a particular answer went wrong. Pipeline Tracing records every step's decisions and data so debugging has a clear trail to follow.
Search found the right documents, but the LLM's answers are still poor — often the problem lies in prompt design. System prompt structure, context formatting, and instruction placement all affect output quality.
LLM generation takes 3-5 seconds, and waiting for the full response before displaying it makes for a terrible experience. SSE pushes tokens as they're generated, reducing time-to-first-character from 5 seconds to under 1 second.
Limiting request count alone is not enough — a single long query can consume ten times the tokens of a normal one. Dual quotas (request count + token count) are what truly control costs.
RAG and Fine-tuning solve different problems. RAG gives the model new knowledge; Fine-tuning changes the model's behavior and style. In most cases you use both, not pick one.
BM25, vector search, HyDE, and Multi-Query each produce separate result sets -- how do you merge them sensibly? RRF uses ranks instead of scores, sidestepping the fundamental problem that scores from different systems are incomparable.
Use another LLM to evaluate answer accuracy and quality — if the score is too low, regenerate, and automatically add appropriate disclaimers.
Caching doesn't have to match exact query strings -- semantically similar questions can hit the cache too, skipping the entire RAG pipeline execution.
BM25 only recognizes words that appear in the query. SPLADE infers related terms and adds them to the search, gaining partial semantic capability while preserving the precision of keyword search.
Questions like 'how many routes did I complete this year' will never be answered well by RAG semantic search — querying the database directly is far more accurate. Let the LLM identify intent, extract parameters, and execute predefined SQL templates.
Vector database selection is more constrained by deployment platform than LLM selection. Determine your platform and scale requirements first, then evaluate features — don't just look at benchmarks.