How to Use LLMs for Free: OpenCode 2 Built-in Models and OpenRouter
Install OpenCode 2, select a built-in free model, and complete your first everyday task. Connect OpenRouter later if you want more models or a free API for your own application.
#llm
188 postsAI Common Failures: Contrast Framing, Significance Inflation, Indicative Summaries, Overgeneralization
LLM prose has four recurring failures: contrast framing (refuting an X nobody said), significance inflation (dressing a plain fact up as a turning point), indicative summaries (announcing the topic, withholding the conclusion) and overgeneralization (summaries that drop conditions and scope). Fix the first three by cutting the filler and writing the answer; fix the fourth by checking each sentence against the source. Contrast framing has been traced to preference data, and models revising a draft move it into another contrast form, so banning a phrase in the prompt may not stop it.
Fixing "It's Not X, It's Y": Three Forms of Contrast Framing, How Common It Is, and When to Keep It
Ask two questions of every contrast: do readers actually hold X, and does rejecting X tell them anything new? If either answer is no, drop X and state Y. 'Rather than' became about seven times as frequent in NLP papers from 2019 to 2026, and five commercial models writing Chinese use the equivalent construction 2.5–9.2 times as densely as a human reference corpus. Models almost never misuse it, but more than half the uses are unnecessary, and with eight in one paper, at least one tends to annoy the reader.
Gemini, Claude, ChatGPT, Grok: Which to Pick for Long Video, Code, Live X Search, and App Platforms
A two-hour lecture is about 720,000 tokens, which Gemini's API accepts and Claude's and ChatGPT's API docs don't list video input for. On code, Claude and GPT are 0.8 points apart on the Vals board, so compare time and cost instead. Grok for X sentiment and auto-fill checkout, ChatGPT for ready-made apps and voice.
Learning AI Security: Practical Paths Through Attacks, Defense, and Investigation
Follow one support assistant through AI-assisted vulnerability discovery, protection of AI data and tools, and AI-assisted investigation and remediation. Each path produces reproducible evidence.
Maintenance: Keeping Notes Alive Past Three Months, and What to Hand an LLM
Schedule a 30-minute weekly review: empty the inbox, deal with finished projects, and pick one question from an MOC to think further about. Summarizing needs no schedule; add a layer only when you touch a note anyway. An LLM can find duplicates, suggest possibly related notes and flesh out key points you've written first; the sentence in your own words, deciding what matters and whether to delete are yours. In one 30-person study comparing three kinds of AI help, the group given complete AI notes learned the least and edited the least.
Post-Training from Chatbots to Agents: A Public-Sources Guide | Stanford CS329A Lecture 9
A chat model is rewarded for a single response people prefer; an agent has to be rewarded for whether a multi-step task actually finished, which needs different data and environments.
Autonomous Agentic Systems: Rewards and Understanding Are the Bottleneck | Stanford CS329A Lecture 18
Laskin's view is that engineers spend 80% of their time understanding systems rather than writing code, and that how far RL goes depends on clean rewards, a problem he considers as hard as superintelligence itself.
Multimodal Agents in Robotics: How a VLA Connects to Actions | Stanford CS329A Lecture 19
A VLA adds an action module to a vision-language model; letting that module's gradients flow back harms language understanding, so knowledge insulation blocks the gradient, while π0.5 co-trains on web data and data from many robots to gain generalization.
Language Models: Why a Model That Only Guesses the Next Word Can Work Without Fine-Tuning | MIT 6.7960 Lecture 21
A language model does one thing: given the preceding text, output a distribution over the next word. At large enough scale, the way to control it becomes changing the input: a few examples (in-context learning), or asking it to write its reasoning first (chain-of-thought; the example cited in class is one sentence, Let's think step by step, taking an arithmetic task from 18% to 80%). But following instructions and saying 'I don't know' do not come from next-word prediction alone; instruction tuning and RLHF add them.
Large Language Models: Next-Token Loss and How Attention Is Computed | Stanford CS229 Lecture 14
This lecture takes a large language model apart in three layers: tokenization turns text into token IDs, the chain rule splits the probability of a whole text into a sequence of next-token conditionals, and training sums the cross-entropy at every position. The second half opens up the Transformer: inner products of queries and keys go through a softmax to give weights, the weights average the values, and a causal mask blocks future positions. The cost of a single attention layer grows with the square of the sequence length T.
Why Generating an Answer Still Requires Verification | Stanford CS329A Lecture 1
An agent needs trustworthy feedback to correct its actions; improving one task’s result does not mean its model weights have learned.
Open-Ended Evolution: A Reading Guide | Stanford CS329A Lecture 7
Open-ended evolution changes agent code and workflows; archives preserve branches, while held-out tests assess generalization.
Software Engineering Agents: A Reading Guide | Stanford CS329A Lecture 13
Program candidates need requirement-based counterexamples; speed matters only after correctness checks pass.
Agent Memory: A Reading Guide | Stanford CS329A Lecture 14
Retrieval, trained KV representations, and cache reuse address different bottlenecks; speed does not guarantee freshness.
Test-Time Compute: Finding the Right Answer Among More Attempts | Stanford CS329A Lecture 2
More candidates increase the chance of a correct solution; credible selection turns that chance into delivered success.
Robust Verification: Checking Steps Before Trusting an Answer | Stanford CS329A Lecture 3
A correct final answer does not validate its derivation; step feedback locates errors, but verifiers still need checking.
Tool and Code Feedback: Changing the Next Action | Stanford CS329A Lecture 4
Observations, tests, and principle-based critique supply different signals; identify the signal before choosing revision or training.
Multi-Step Planning: Decompose the Step That Fails | Stanford CS329A Lecture 5
Distinguish search, parallel execution, and training; subtask artifacts must fulfill the original task criterion.
pplx-embed: Perplexity's Embedding Model Family and How to Choose Between Its Three Lines
pplx-embed is the embedding family Perplexity has been releasing since February 2026, in three lines: v1 single-vector (0.6B/4B, API at $0.004/$0.03 per 1M tokens), context embeddings (v1 plus a 9B v2 preview), and v2-late multi-vector (0.6B/9B, searches PDF page images directly). They solve different problems; v2 does not replace v1.
AI Agent Interview Prep: From Tool Calling and Memory to MCP and Prompt Caching
Agent interviews keep returning to eight topics: the agent loop, structured output, function calling, large tool sets, memory, design trade-offs, MCP versus A2A, and prompt caching. This post strings them into one line (concept, mechanism, how to answer) and corrects when the 2026-07 MCP spec changes actually landed.
ML/LLM System Design and Coding Interviews: Four Architectures, Three Python Problems, and a STAR Skeleton
Four system design questions that come up often (enterprise RAG, an LLM serving stack, a customer-support agent, online A/B testing with a data flywheel), three Python problems (dynamic batching, sampling, backoff retries plus a token bucket), and a STAR skeleton for behavioral rounds. Each design question has a speakable answer and a self-check list, and all code was actually run.
LLM Engineering Interview Prep: Fine-Tuning, Alignment, Inference Optimization, Evaluation, and Safety
Seven LLM engineering topics that come up often in interviews: LoRA/QLoRA and fine-tuning vs RAG, RLHF/DPO/GRPO alignment, inference optimization (PagedAttention gives vLLM 2-4x throughput), sampling and decoding, LLM evaluation, the OWASP LLM Top 10 (2026 edition), and hallucination. Each section covers the concept, the mechanism, and how to answer.
Prompt, Context, Harness: The Three Layers, Their Boundaries, and Evaluation Gates
Prompt engineering shapes a single instruction, context engineering decides what the model sees right now, and harness engineering wraps the whole non-deterministic system. Bsharat's 26 principles, Breunig's four context failures, and OpenAI's roughly one-million-line, zero-handwritten-code harness experiment each map to one layer. Evaluation is the piece teams skip most, and it belongs in a CI gate that blocks silent regressions.
RAG Interview Prep: From the Three-Stage Pipeline to Agentic RAG, and How to Tell Nine Variants Apart
Answer RAG interview questions in four steps: the three-stage pipeline, where each stage fails, which of nine variants patches which hole, and the 6 kinds of problems RAG cannot solve. Self-RAG, CRAG and Adaptive-RAG add judgment; Graph RAG and KG-RAG change the index; CAG skips retrieval; Agentic RAG and Reasoning RAG hand decisions to the model.
AI Engineering from Scratch: A Guide to the Free Course That Builds Everything by Hand, from Linear Algebra to Multi-Agent Systems
AI Engineering from Scratch is a free, MIT-licensed course on GitHub with about 63k stars: 20 phases and 523 lessons, running from linear algebra, classical ML, and deep learning to LLMs, agents, and AI safety. Each lesson builds the algorithm by hand, then repeats it with a framework. The early math and deep-learning-core phases are worth following step by step; skip the Phase 10 SFT/RLHF/DPO lessons, and treat the rest as a map of topics.
Scaling and MoE: Compute Budgets and Conditional Computation | CMU 10-423 Topic Guide
Scaling laws study model and data allocation under a budget. MoE increases expert capacity while activating selected experts. The former concerns allocation and empirical models; the latter concerns architecture and routing. Both deserve separate checks.
Accelerating Transformers on GPUs, and Where LightSeq Finds the Time | CMU 11-868 Lecture 10
Lecture 10 of 11-868 uses Lei Li's own LightSeq and LightSeq2 as the case study and breaks them into four techniques: fuse every small operation outside matrix multiplication into one kernel, rewrite the LayerNorm and Softmax formulas to cut thread synchronizations, store parameters and gradients in FP16 but compute updates in FP32, and reuse memory based on backward-pass dependencies. The slides report 1.4-3.5x training speedups on WMT14 English-German. The official syllabus lists no public recording link; this guide works from slide page numbers and the two papers.
Why LLMs Need Systems — the Scale Curve, Low-Level Operators, and Three Layers of Abstraction | CMU 11-868 Lecture 1
CMU 11-868's first lecture spends 51 slides on one argument: the LLM bottleneck isn't only the model, it's computing larger LLMs on bigger datasets with fewer GPUs, less memory, and less power, faster. It breaks a Transformer into four low-level operators (matrix multiply, reduction, map, memory movement), sorts the hard problems into kernel, framework, and distributed-system layers, and warns that fast computation isn't enough because moving data takes time too.
LLM Systems: A Learning Path through Training, Inference, and Deployment | CMU 11-868 Course Overview
CMU 11-868 is Lei Li's graduate course on LLM systems: it goes from CUDA kernels and your own MiniTorch framework to distributed training, SGLang serving, and RLHF. All 28 Spring 2026 slide decks, 7 assignment pages, and 7 starter-code repos are public, which rates it A3. What's missing: videos, GPUs and a PSC account, the quizzes, and any official statement of which two assignments are optional.
Transformer and LLM: The Architecture Seen Through an Efficiency Lens | MIT 6.5940 Lecture 12
In Lecture 12, 6.5940 switches from CNNs to Transformers. The lecture doesn't dwell on theory. It points to where memory and compute go. Attention is O(N²). If Llama-2-70B used MHA, its KV cache at batch 16 and length 4096 would take 160GB. GQA shrinks that 8x and MQA shrinks it 64x. MoE adds total parameters while keeping per-token compute flat. This post bridges into Lecture 13 on LLM deployment.
LLMs Are Simpler Than You Think — Next-Word Prediction, Temperature, and Your Own Benchmark | NCCU Yen-Lung Tsai Generative AI Lecture 4
L04 reduces a large language model to one sentence: look at the preceding words, score every word in the vocabulary, turn the scores into probabilities with softmax, and sample the next word. To give the model a memory of what came before, the lecture covers RNNs and then gives a first look at Transformer Q/K/V. GPT-2's 1.5 billion and GPT-3's 175 billion parameters illustrate scale; temperature and top-p explain why every answer comes out different. The second half covers running open models locally and estimating VRAM. Week 4 homework: write test prompts on a topic you know well and compare at least two LLMs.
LLM Applications and Ethical Challenges — Hallucination, Privacy, DeepSeek, and a One-Paragraph System Prompt Called the Lucky Vicky Generator | NCCU Yen-Lung Tsai Generative AI Lecture 6
The first half of L06 is about ethics. Yen-Lung Tsai quotes Karpathy's line that hallucination is a feature of LLMs, then works through plagiarism, whether your data gets used for training, and DeepSeek's censorship and corpus skew, and closes with seven principles of responsible use. The second half is about applications: give the model the right information and clear instructions, and one system prompt becomes a Lucky Vicky positivity generator, a social-media copywriter, or a biased college-major counselor. The week-6 assignment moves that prompt into an OpenAI-compatible API with a Gradio front end: a chatbot with a persona.
Dialogue Systems and Tool Use: From Four Modules to LLMs That Look Things Up and Ask Back | NTU Yun-Nung Chen Applied Deep Learning Lecture 13
A dialogue system that helps a user get something done is traditionally split into four modules: understand the utterance, track the state so far, decide the next action, and write the action as a sentence. A language model can act out all four in one go, and without external tools its "let me check" checks nothing. The first half of this lecture therefore covers representative systems that let a model use tools: LaMDA drafts first and then verifies and revises, BlenderBot adds search and long-term memory, WebGPT learns to search from records of human actions, and Toolformer has the model generate and filter its own training data. The second half moves from a single reply to multiple turns: the model can ask the user, learn an expert's strategy, and guess what the other party is thinking. It ends with how conversations are evaluated.
Post-Training and Efficient Adaptation: How a Model Learns to Follow Instructions and How to Tune It Cheaply | NTU Yun-Nung Chen Applied Deep Learning Lecture 7
A pre-trained model continues text well and still may not do what an instruction asks. The first half of this lecture covers the two steps of post-training: instruction tuning uses many different tasks to teach the model to read task descriptions, and RLHF then moves the model toward whole responses that people prefer, with a reward model and pairwise comparisons handling the cost of human labor and inconsistent scores, while DPO and KTO drop the reinforcement-learning part. InstructGPT and ChatGPT are these two steps chained together. The second half covers LLM adaptation: models are too large to fine-tune in full, so only a small number of parameters are trained, with Adapter, LoRA, and prompt tuning, and no single method is best on every task.
RAG: Looking Things Up Before Answering When the Model Cannot Remember or Is Out of Date | NTU Yun-Nung Chen Applied Deep Learning Lecture 8
A language model stores knowledge in its parameters: rare facts are poorly remembered, events after training are unknown, internal company documents were never seen, and answers cannot be checked. RAG lets it take the exam open-book: retrieve relevant passages for the question first, then hand them to the model with the question. This lecture covers how retrieval is done (chunking, sparse and dense retrieval, bi-encoders and cross-encoders, training a retriever without labeled pairs), what can be added before and after retrieval (query rewriting, reranking, multiple rounds of retrieval), the lines along which RAG developed (what to retrieve, how to use it, when to retrieve), and finally which situations suit prompting, RAG, or fine-tuning.
Merging a Japanese Model and a Math Model, With No Training, Into One That Solves Japanese Math Problems | NTU Hung-yi Lee Machine Learning Assignment 7
HW7 hands you two models fine-tuned from Mistral-7B-v0.1: shisa-gamma-7b-v1, strong in Japanese, and WizardMath-7B-V1.1, strong in math. You may only merge them at the parameter level (no further training, no MoE or ensembles), and the merged model has to answer 20 Japanese math questions written by a TA. Part 1 (60%) is tuning the method, weights, and density in mergekit, with simple and strong baselines at 50% and 75% accuracy. Part 2 (40%) is 8 multiple-choice paper questions. The spec, Colab, and Kaggle notebook are public, but JudgeBoi returned 502 on 2026-09-30 and the paper questions live on NTU COOL, so outside readers can only check accuracy inside the notebook.
Transformers and Large Language Models: Nine Lectures from Transformers to AI Agents | Stanford CME295 Course Overview
CME295 is a two-unit Stanford course with no homework; your grade is the midterm and the final, 50% each. The 2025 edition's nine lectures are fully public: videos, slides, and both exams with solutions. The 2026 edition rewrites the agent lecture around context compaction, harnesses, coding agents, and skills, and adds three full lectures on LLM systems, reinforcement learning, and Diffusion LLMs.
The Knobs You Turn When an LLM Generates, from Temperature and Top-p to Chain of Thought | Stanford CME295 Lecture 3
CME295 Lecture 3 defines an LLM as a decoder-only next-token predictor, uses MoE to explain why a huge model only touches part of its weights per token, and spends most of its time on the knobs you can turn at generation time: greedy, beam search, top-k, top-p, temperature, guided decoding, plus three prompting techniques (few-shot, chain of thought, self-consistency). The 2026 edition folds this lecture into Lecture 2, prompting is no longer a separate syllabus item but still gets a brief segment in the recording.
One Question Asked for Forty Years, and How a 2023 NLU Course Defines Understanding | Stanford CS224U Topic Guide
The first CS224U lecture of Spring 2023 asks "Which U.S. states border no U.S. states?" of every system from Chat-80 (1980) to text-davinci-001. The answers show that the progress is real. The lecture then questions whether that progress counts as understanding, using Levesque's "cheap tricks," models that invent links, and benchmarks that saturate within a year or two. That splits the course map in two: the first half teaches you to build systems with Transformers and retrieval-augmented in-context learning, and the second half teaches you to test them with harder benchmarks, behavioral evaluation, and causal explanation methods.
LLM Training and Applications, Self-Supervised Learning | Berkeley CS189 Lectures 23–24
Lec 23 wires a transformer into a next-token predictor: tokenize, look up embeddings, stack L layers of masked attention, multiply back by the embedding table and apply softmax, and train with cross-entropy (that is, MLE). Pretraining supplies knowledge; to chat, a model also needs SFT, LoRA, RLHF, or DPO, and at inference time it leans on in-context learning, RAG, chain-of-thought, and tool calls. Lec 24 generalizes "invent a fake supervised task" to images: autoencoders, colorization, inpainting, rotation, jigsaw puzzles, clustering, and finally contrastive learning, SimCLR, and CLIP. Discussion 11 practices positional encodings, RoPE, causal masks, and the KV cache.
Deep Research Landscape: Taxonomy of 80+ Implementations, Roadmap, and Trade-offs
The entire Deep Research field has 80+ implementations, but the core structure is just three-stage roadmap × four components × three optimization methods. This article maps the full landscape: from Agentic Search to Full-stack AI Scientist, from query planning to answer generation, from workflow prompting to end-to-end RL.
One Missing 'Write a Script': How Skill Instructions Determine LLM Success or Failure
An AI assistant platform running Opus 4.6 hit stream_stall (90s timeout) twice consecutively when generating docx. Root cause: skill instructions lacked one sentence — 'Write a script' — causing the model to output JS code inline instead of writing a file and executing with node. Claude.ai's official SKILL.md has that sentence, and the model consistently takes the safe path.
Gemma — Google's Open-Weights Flank: Gemma 4 Moves to Apache 2.0, From Mobile to Workstation Full-Size Open Weights
Gemma is Google's open-weights family paired with the closed-source Gemini flagship; Gemma 4 (E2B / E4B / 26B-MoE / 31B) released in April 2026 switched its license from Google Gemma Terms of Use to Apache 2.0 — the single most important change in this article.
Laguna: From a 33B Local Workhorse to 118B Long-Horizon Reasoning, Poolside's Three-Releases-in-Three-Months Bet
Laguna is Poolside's agentic coding model family: XS 2.1 packs 33B-A3B into a 36GB Mac, while S 2.1 brings 118B-A8B with 1M context to 70.2% on Terminal-Bench 2.1 and 40.4% on DeepSWE, both open under OpenMDW-1.1.
Ling — From Trillion-Parameter Flagships to 5.1B Execution Nodes, Ant Group's Three-Line AGI Strategy
Ling separates execution, Ring handles deep reasoning, and Ming covers multimodal tasks; the family prioritizes intelligence per unit of compute, so selection starts with workload, deployment budget, and modality.
Muse Spark: Meta's Closed-Source Agentic Model Line, from Llama to 1.3
Muse Spark is Meta's closed-source agentic model line: version 1.3 combines a 1M-token context, multimodal inputs, and long-horizon tool loops. Standard pricing is $1.25/$4.25 per 1M input/output tokens, while Contributor drops to $0.10/$0.20 in exchange for training rights. It is not the next Llama; it is a separate product line built around models, APIs, and coding agents.
Nex-N2.5: The Open Agent Family That Treats Vision as an Interface, From 35B mini to 1.6T Max
Nex-N2.5 is Nex AGI's open agentic model family: mini scores 82.9 on OSWorld-G at 35B-A3B, Pro tops Claude Opus 5 with 87.4 at 397B-A17B, and Max leads the whole official table on BrowseComp with 92.6 at 1.6T, all open under Apache-2.0.
AI Security Cert Showdown — SecAI+ vs CAISP vs GAIPS vs AAISM
Four AI security certs, four different bets: SecAI+ ($298) is CompTIA's mid-level expansion play, CAISP ($1,099 all-in) has the strongest hands-on labs and fullest OWASP LLM Top 10 coverage, GAIPS ($999/$9K) is the SANS gold-standard defender cert with CyberLive exams, and AAISM ($459+) is governance-layer but requires CISM or CISSP first. Under $400 → SecAI+. Want to actually hack and fix → CAISP. Company paying → GAIPS. For the offensive side (AI red teaming), OSAI+, HTB COAE, and COASP launched in 2026 and form their own group; see the section near the end.
Should You Rent a GPU to Learn Model Training? GPUtw.ai, LoRA, Jupyter, and the First Experiment
GPUtw.ai makes sense as a short-rental GPU learning tool: start with Jupyter, Ollama, or ComfyUI, then try LoRA/QLoRA on a small model. It is not a large foundation-model training platform, and the first run should verify deployment, billing, and data retention with a small budget.
Training Small LLMs From Scratch in the Chinese Community: Corpus, Tokenizers, and Three Open-Source Projects
A close look at three open-source projects training LLMs from scratch in the Chinese community — baby-llama2-chinese (218M, 63.4B tokens), ChatLM-mini-Chinese (0.2B T5, 10.23M dialogues), and Steel-LLM (1.12B, 1T tokens, 8 months) — comparing corpus strategy, tokenizer decisions, and community ecosystem. Honest evaluation included: baby-llama2 scored a bottom-ranking 21 in MiniMind's side-by-side test; ChatLM has the strongest knowledge (62) but weak coding.
How to Spend Every Parameter: OpenELM's Layer-wise Scaling and MiniCPM's Three-stage Unfreezing
OpenELM uses layer-wise scaling to shift parameters toward layers near the output; with 1.08B parameters and 1.5T tokens it beats OLMo 1.2B (+2.36% on the LLM360 average) despite OLMo training on 3T tokens. MiniCPM trains multimodal small models from scratch with a three-stage unfreezing recipe (Resampler first, vision encoder next, everything unfrozen last); MiniCPM-V 4.5 reaches sub-30B SOTA on VideoMME with only 8B parameters, and 4-bit quantization squeezes fp16's 16–17GB memory footprint down to about 5GB for phones.
Karpathy's nano lineage: nanoGPT, llm.c, and nanochat
A tour of Karpathy's three teaching repos: nanoGPT (2022, ~300 lines each to reproduce GPT-2 124M), llm.c (pure C/CUDA training), and nanochat (2025-10, one speedrun.sh from tokenizer to WebUI). $100 and 4 hours on 8×H100 buys a chatty model; GPT-2-grade capability is now down to about 2 hours and $48.
The framework for training from scratch: LitGPT's no-abstraction rewrites, pretrain flow, and the TinyLlama track record
LitGPT (Lightning AI, ~13,600 stars, Apache 2.0) rewrites 20+ mainstream LLMs — Llama 3, Qwen2.5, Phi 4 — from scratch as single-file, no-abstraction implementations, with a full pretrain / finetune / evaluate / serve CLI. TinyLlama (1.1B parameters, 3T tokens) was trained on this codebase. This post breaks down how it differs from MiniMind, how to actually use it, and where it stops.
MiniMind: Train an LLM From Scratch for $0.40
MiniMind is an open-source project for training LLMs from scratch: a 64M Dense model and a 198M-A64M MoE model that run the entire chain — Pretrain → SFT → LoRA → DPO → PPO/GRPO/CISPO → Agentic RL — in ~2 hours on a single RTX 3090 at roughly 3 RMB (~$0.40). Every core algorithm is implemented natively in PyTorch with no high-level wrappers.
National-Team LLM Training: Apertus' Compliance Route and LLM-jp's Japanese Ecosystem
Switzerland's Apertus (8B/70B, 15T tokens, 1,000+ languages) filters opt-outs and personal data before training to satisfy the EU AI Act; Japan's LLM-jp consortium shipped LLM-jp-4 (12T tokens) in April 2026, claiming wins over GPT-4o and Qwen3-8B on standard benchmarks. Both prove that from-scratch training outside the English sphere is a data-governance problem, not a technical one — plus a note on RWKV-7 as the non-Transformer alternative.
How fully transparent LLMs are built: OLMo 3's model flow and LLM360 K2's 360-degree openness
"Open-source LLM" is a spectrum: weights-only (Llama), weights plus data (most fully open projects), or data order, intermediate checkpoints, and training logs all released (LLM360 K2, OLMo 3's model flow). This piece unpacks the two projects that pushed transparency furthest: OLMo 3 shipped the first fully open 32B thinking model in November 2025, and K2 is the first 65B-class model whose checkpoints even include optimizer states.
Running MiniMind on RunPod: From Zero to a Chatting Model
The hands-on installment of the series: rent an RTX 3090 on RunPod (Secure Cloud $0.5/hr, Community Cloud $0.22/hr), follow the MiniMind README through pretrain (~1.21h) + SFT (~1.10h), spend roughly $0.55–1.50 USD total, and chat with your own 64M model trained from scratch in the terminal.
Training an LLM from Scratch: A Project Spectrum from $0.4 to 65B
Open-source projects have pushed the cost of training an LLM from scratch absurdly low: MiniMind runs the full PreTrain-to-RL pipeline for about $0.4 (2 hours on a single RTX 3090), while at the other end OLMo 3 and LLM360 K2 publish everything — data, code, and stage-by-stage checkpoints of 65B models. This series walks the whole project spectrum from $0.4 to 65B in 11 articles, and flags where the map is biased.
When to Train an LLM From Scratch: A Decision Tree and the Full Cost Ladder
Training from scratch only makes sense in three cases: you want to learn how training works, you have 10B+ clean tokens no open model has seen, or you need a fully transparent training process for research. Otherwise fine-tuning or RAG is almost always cheaper. This post collapses the series' main routes into one cost ladder and a decision tree.
YuLan-Mini: Squeezing a 2.4B Flagship-Scale Small Model Out of 1.08T Tokens
YuLan-Mini is a 2.4B open-source model from Renmin University's AI Box lab, trained on 48 A800 GPUs with only 1.08T tokens — scoring 37.8 on MATH-500 and 64.0 on HumanEval, beating Qwen2/Qwen2.5 peer models trained on 7T–18T tokens at math and code. What's public isn't a slogan: per-phase data mixes, pre-annealing optimizer states, and even W&B logs of the ablation studies.
Inkling: From an OpenAI Exodus Team to a 975B Open Flagship, and Tinker's Fine-Tuning Bet
Thinking Machines Lab (founded 2025 by Mira Murati, $2B seed at a $12B valuation) released Inkling in July 2026 under Apache 2.0 (975B total / 41B active params, 1M context, native multimodality, controllable thinking effort) plus a smaller Inkling-Small (276B / 12B), paired with the Tinker fine-tuning platform—turning customizability itself into the product.
How to Use Cloudflare AI Gateway: Logging, Caching, Rate Limits, and Fallbacks
AI Gateway is the control plane for AI calls: one layer for logs, analytics, cache, rate limits, retry/fallback, BYOK, and Unified Billing. In Workers, use env.AI.run(..., { gateway }); with external SDKs, change the baseURL or provider-native endpoint.
Learning Design from Mature Coding Agents (35): Model Catalogs and Per-Role Routing — looplane's Role Aliases and Reviewer Lane
looplane now has static ModelRole/ModelRoute candidates, opt-in aliases such as --model @cheap, cross-provider fallback, and a no-tool reviewer lane that runs after verification. Role inheritance/override rules and automatic summarizer, parser, or scout routing remain open.
Looplane's provider-neutral native loop: from one model turn to a verified terminal state
Looplane's native lane is controlled by AgentRunner: prepare a workspace, request a model turn, execute tool calls, append observations, and enter verification only when the model stops calling tools. Step, wall-time, repetition, token, and cancellation guards can terminate the run independently of the model. Protocol translation belongs to the next article.
LLM API Routing: Direct, Aggregator, or Cloud — A Price Comparison
The same model can cost 2-5× more depending on the channel. Direct API is simplest, aggregators (OpenRouter) are most flexible, cloud platforms (Bedrock/Vertex) suit enterprises. This post compares actual August 2026 prices across six channels with a decision tree.
OLMo: The Only Language Model Family That Open-Sources Its Training Data
Allen AI's OLMo is the only language model family that fully publishes weights, training data (Dolma, 9.3T tokens), training code, all intermediate checkpoints, and evaluation tools. OLMo 3's 32B Think model hits 96.1% on MATH — and you can use OlmoTrace to trace any output back to the exact training data that produced it.
Tokens, Context Windows, and Inference vs Training: Three Things to Know Before Using AI Models
Models don't read words — they read tokens. A Chinese character is typically 1-2 tokens; an English word is 1-3. The context window is the token limit per request. Inference is using a model; training is teaching one. What you do every day is inference.
Fine-tuning vs RAG: When to Teach the Model vs When to Look Things Up
Data changes often and you need citations → RAG. Need consistent style or want to run on a small device → fine-tuning. In practice, many production systems use both: fine-tune a small model that speaks your domain language, then use RAG to supply up-to-date facts.
Scaling Laws: How Big Should a Model Be, and Why Bigger Isn't Always Better
Scaling laws show that loss decreases predictably with more parameters, data, and compute — following power-law relationships. The Chinchilla paper's key finding: most models were too large and undertrained. Given the same compute budget, training a smaller model on more data produces better results. This reshaped the entire industry's training strategy.
Understanding AI Models: 18 Articles from Tokens to Self-Hosting
You don't need to become a researcher to understand AI models systematically. This series starts from what you can see (tokens, context windows) and works up to self-hosting open-source models — 18 articles covering everything you need to choose models, read benchmarks, and estimate costs.
Tokenization: The BPE Algorithm, and Why Chinese Costs More Than English
Models charge by tokens, not characters. The BPE algorithm starts from individual bytes and repeatedly merges the most frequent adjacent pair to build a vocabulary. English 'understanding' might be 1-2 tokens, but Chinese '理解' could take 2-3 — same meaning, higher cost.
Ahead of AI: How a Scholar Built 200K Subscribers by Publishing Monthly, Not Daily
Computational biology PhD turned UW-Madison professor Sebastian Raschka launched Ahead of AI on Substack in 2022, publishing monthly deep dives into LLM papers and architectures. Four years later: 200K+ subscribers, zero sponsorships, and a book-newsletter flywheel that proves low frequency and high depth can win in a crowded AI newsletter market.
Learning Design from Mature Coding Agents (10): Edit Tool Trade-offs — unified diff, exact edit, hashline, and whole-file
LLMs break unified diffs on bookkeeping: wrong hunk counts, hallucinated context lines. The five reference projects split into two camps — simplify the diff grammar (Codex drops line numbers), or drop diffs entirely (Claude Code/Pi/OpenCode exact replace); OMP goes further by binding read state into the format via hash anchors. looplane took the minimal-intervention path: keep the guarded apply_patch, add a zero-fuzzy replace_text, and its qwen3:4b eval went from stable failure to 5/5.
The Complete Unsloth Guide: Fine-Tune and Run LLMs Locally, Faster
Unsloth is the fastest, most VRAM-efficient local LLM fine-tuning tool — 2× training speed and 70% less VRAM. In 2026 it added a Desktop app that bundles inference, training, image/video generation, web search, and agent integration into a complete local AI workstation.
Apple Foundation Models: Privacy-first Ecosystem AI with a 20B Sparse Model on Phones
Apple Foundation Models (AFM) is Apple's closed-ecosystem AI family. It evolved from a 3B dense model with LoRA adapters in 2024 into five models in 2026. AFM 3 Core Advanced runs a 20B IFP sparse architecture on phones while activating only 1–4B parameters; Cloud Pro runs on Google Cloud NVIDIA GPUs and is refined through Gemini distillation. There is no public API price or third-party benchmark, and access is limited to Apple's Foundation Models framework.
A Guide to the Top AI Conferences of 2023: Machine Learning
In 2023, LLMs took over the machine-learning conference agenda. NeurIPS received more than 12,000 submissions; both Outstanding Papers addressed large models, while runner-up DPO became a practical alternative to RLHF within two years. DreamFusion opened the text-to-3D field, ICML spotlighted LLM watermarking and learning-rate adaptation, and the Mamba preprint emerged as the first serious architectural challenger to the Transformer.
What Topics Dominated the Top AI Conferences of 2023? The Year LLMs Rewrote the Research Agenda
2023 was the first year in which LLMs comprehensively rewrote the AI research agenda. DPO received a NeurIPS Outstanding Paper Runner-Up award, ReAct became an ICLR Oral, and hallucination grew from a marginal term into a major track at every conference. Meanwhile, 3D Gaussian Splatting swept through computer vision after its SIGGRAPH debut, Mamba emerged at the end of the year to challenge the Transformer attention monopoly, and publication volume for traditional NLP pipelines began a clear decline.
How Marin Trains 535B: Scaling Ladder, MoE Expert Parallel, Harrier Data and Live W&B
Stanford Marin pre-registers a paloma macro-loss of 2.04 with a 5-rung Scaling Ladder at 1% cost, then trains 535B-A23B on 11×GB200 in public with live W&B telemetry — 847 training buckets already show the most teachable frontier run.
AI Model Evaluation Sources: How to Judge Whether a Model Is Actually Good
You cannot take model vendors' self-reported scores at face value. This guide covers the most important independent evaluation platforms, domain benchmarks, adoption indicators, and official sources in 2026: what each measures, how to read it, where it is biased, and which figures matter for different use cases.
Claude——From AI Safety Lab to SWE-bench Champion, the Strongest Closed-Source Agent Choice
Claude is Anthropic's closed-source LLM family, known for Constitutional AI training, agent capabilities, and coding performance. In July 2026, Opus 5 scored 96% on SWE-bench Verified to claim the coding crown, while Fable 5 led general capability at 83% on LiveBench. Four tiers (Fable / Opus / Sonnet / Haiku) span $1–$10, making this the only family in the series with zero open weights.
Cohere — The RAG-Native Outlier: How Command, Embed, Rerank, and Aya Fit Together
Cohere is the only family that ships generation, retrieval, reranking, and multilingual as distinct products. Command A runs 256K context on two GPUs at 111B, Embed v4 does mixed image-text retrieval, Rerank v4 handles 32K semi-structured data, and Aya covers 101 languages — a four-piece stack built for RAG. This post breaks down each pillar's positioning, licensing, and selection guide.
DeepSeek: From an MoE Lab to OpenRouter's Most-used Open Model
DeepSeek used MLA and MoE innovations to drive inference costs to an industry low. V4 Flash activates only 13B parameters while approaching frontier-model quality and ranks first by OpenRouter usage. This guide traces V1 through V4, the R1 reasoning branch, and how to choose each version.
Gemini——Google's Native Multimodal Flagship: 1M Context and Scientific Reasoning Champion
Gemini is Google DeepMind's native multimodal LLM family, famed for a 1M-token context window and native video/speech input plus scientific reasoning. 3.1 Pro tops GPQA Diamond 94.1% and ARC-AGI-2 77.1% to claim science-reasoning dual crowns, at $2/$12—1/6 of Claude. 3.8 Flash delivers agentic coding results close to frontier models at $0.75/$3.75 (Terminal-Bench 2.1 89.4%; DeepSWE v1.1 73.7%), with the same introductory pricing across 3.6, 3.7, and 3.8.
GLM——From a Tsinghua Lab to a 744B Open-Source Flagship, and GLM-5.3's Cybersecurity Surge
GLM is Zhipu AI (Z.ai)'s open LLM family from Tsinghua's KEG Lab. GLM-5.3 (2026/08) lifts coding +50% over the previous generation, hits 84.5% on CyberGym ahead of Anthropic Mythos 5 and OpenAI GPT-5.6 Sol, and scores 60 on the Artificial Analysis Intelligence Index tied with Kimi K3 for open-source #1. The only frontier open model trained entirely on Huawei Ascend.
GPT——Closed API for Revenue, Open GPT-OSS for Ecosystem: the Unified Routing Platform Behind the World's Largest AI Service
GPT is OpenAI's LLM family, from 117M parameters in 2018 to the three-tier GPT-5.6 Sol/Terra/Luna lineup in 2026, serving 1B+ users and 2M enterprise customers. GPT-5.6 Sol leads LiveBench 81.1%, Terminal-Bench 2.1 88.8%, and Artificial Analysis Coding Agent Index 80 across multiple agentic benchmarks, while OpenAI's first open-weight model GPT-OSS ships under Apache 2.0.
Grok — From a 314B Open-Source Bet to Grok 4.6/Build/Imagine, xAI's Distribution-Driven Catch-Up
Grok is xAI's LLM family: founded July 2023, opened with a 314B MoE under Apache 2.0 in March 2024, and two and a half years later spans Grok 4.6 (500K, $2/$6, four reasoning levels), Grok 4 Fast (2M), Imagine for image/video, and Grok Build for terminal coding — its moat is distribution (X / grok.com / Tesla / Bedrock), not single-model supremacy. This post traces Grok 1→4.6, sub-line positioning, pricing, and licensing traps.
Kimi——From a 200K Long-Context Tool to a 2.8T Open-Source Frontier, and K3's Architectural Leap
Kimi is Moonshot AI's LLM family, born from ultra-long context. Kimi K3 (2026/07) is the world's first open 3T-class model—2.8T params, 104B active, 1M context, scoring 60 on the Artificial Analysis Intelligence Index tied with GLM-5.3 for open-source #1. Its Kimi Delta Attention brings a 2.5× scaling efficiency gain.
Llama——From Open-Source Experiment to the Most Deployed Open LLM, and Meta's Closed-Source Pivot
Llama is Meta's open-source LLM family, with the largest enterprise deployment footprint and the most mature ecosystem. Llama 4 Scout (10M context) and Maverick (17B active / 400B total MoE) are the current open multimodal benchmarks, but Meta pivoted to closed-source Muse Spark in April 2026—Llama 4 is likely the last major open Llama, and its license is not truly open (Llama 4 Community License, separate license required above 700M MAU).
Mistral——Europe's Open AI Challenger: Smaller Models and European Sovereignty as a Different Bet
Mistral is Europe's most successful AI startup, cutting through the market with a 'smaller, faster, cheaper' strategy and European data-sovereignty positioning. Mistral Large 3 is Europe's strongest commercial LLM, Small 4 is the 24B efficiency king, and Medium 3.5 is the open Modified-MIT model optimized for agentic coding. Its moat is not technical scale but the 'European compliance' card.
Qwen: Open Weights at Every Size from 0.8B to 2.4T — How HuggingFace's Download Champion Runs a Two-Track Play
Qwen is the most-downloaded model family on HuggingFace, spanning sizes from 0.8B to 2.4T. In August 2026, Alibaba open-sourced a Max-tier flagship for the first time (Qwen3.8-2.4T-A95B) — but swapped the customary Apache 2.0 license for custom terms. Meanwhile the other new release, Qwen3.8-27B, runs native vision on laptop-class hardware and is the only one shipping under Apache 2.0. This post traces the family from 2023 through generation 3.8, explains how the open line and the commercial line split apart, and helps you pick the right model at each tier.
AI Model Landscape: The 2026 Map You Need
In 2026, AI models span seven major categories and more than 20 subcategories. This introduction to the AI Model Families series maps use cases to models and models to families, with current rankings and selection advice for each use case.
Building a Taiwan Stock Research Agent (Part 1): Why Taiwan Needs Its Own Research Agent
US-stock LLM agents have attracted nearly 100,000 GitHub stars, yet no Taiwan-stock project has even passed 10. I consolidated three side projects into a Taiwan-stock research agent where every conclusion must first survive a backtest; this article explains why.
Building a Taiwan Stock Research Agent (Part 6): Making Every Number in an LLM Report Auditable
Numbers are the easiest part of an LLM report to hallucinate. I therefore put every trusted number into a SHA-256-addressed evidence manifest and let the LLM cite only {{fact.id}} placeholders. If it writes a bare number, the entire output is discarded and replaced with a deterministic template.
Baseten: The Model Inference Lifecycle from Truss Packaging to Autoscaling
Baseten puts custom-model packaging, GPU deployment, inference engines, autoscaling, and release workflows on one platform. Its value is not another OpenAI API, but retaining runtime control while operating less GPU orchestration.
Cerebras Inference: Know the Bottleneck Before Putting Wafer-Scale Speed in an Agent Loop
Cerebras can dramatically accelerate generation on supported models, but agent latency still depends on prefill, tool I/O, model quality, and platform compatibility.
Introduction and Setup: Turning Language Problems into Computable Components | Stanford CS124 Week 1
Week 1 has no algorithms. The live introduction explains what the course teaches: which week covers each component of an LLM, which language algorithms outside LLMs are also taught, and why language is hard. PA0 sets up the Python and Jupyter environment the later assignments use.
Neural Networks and LLMs: From Units and Backpropagation to Decoder-Only Models | Stanford CS124 Week 6
Week 6 has two halves. The pre-recorded material teaches neural networks: how one unit computes, why hidden layers are needed, the softmax, classification and language modeling with word vectors, and backpropagation. The live lecture introduces large language models: what they are, the three architectures, why so many tasks can be recast as continuing text, how to sample, how pretraining works, and how a model becomes an assistant.
Transformers and Speech Processing: How Attention Lets Each Token Use Its Context | Stanford CS124 Week 7
Week 7 opens up the Transformer: how attention lets each token draw on the tokens before it, what parts make up a block, and how the input and output ends are attached. PA6a has you write causal self-attention, train a small model and compute perplexity. The same week's live speech lecture introduces the speech recognition task and word error rate.
Collaborative Filtering and LLM Agents: From Movie Similarity to Search and Memory Tools | Stanford CS124 Week 9
Week 9 teaches recommender systems: content-based recommendation looks at the features of items, while collaborative filtering looks only at the ratings matrix and estimates unrated items from similar users or similar items. PA7 writes item-item collaborative filtering as a function and hands it to an LLM agent that makes tool calls, then adds ticket booking, web search and memory.
Turning Hallucinating LLMs into Dependable Assistants | Stanford CS224V Lecture 1
Fall 2025 opens with computational thinking: reliability comes from decomposing retrieval, formal representation, verification, and generation into testable algorithms, not from one heroic prompt.
Transformers Have Many Variants but Few Stable Defaults | Stanford CS336 Lecture 3
Lecture 3 does not turn its survey of modern LLMs into a single best recipe. It finds a conservative consensus—pre-norm, RMSNorm, no biases, SwiGLU, and RoPE—plus a small set of deviations justified by inference cost or stability.
Attention Has Alternatives, and MoE Does Not Scale for Free | Stanford CS336 Lecture 4
Lecture 4 studies two kinds of sparsity: linear/recurrent attention reduces sequence-length cost, while MoE activates only part of a model for each token. Both turn saved FLOPs into routing, balancing, communication, and kernel problems.
Filtering, Deduplication, and Mixing Turn Raw Web Data into Training Data | Stanford CS336 Lecture 14
Lecture 14 moves raw documents through language, quality, and safety filtering; exact and near deduplication; and source mixing. Each stage reshapes model behavior, while synthetic instruction and agent trajectories extend the pipeline into executable environments.
Data Does Not Fall from the Sky, and Every Source Has Access and License Costs | Stanford CS336 Lecture 13
Lecture 13 traces training sources through Common Crawl, Wikipedia, GitHub, arXiv, books, and open datasets. Technically accessible is not the same as licensed, and raw data is not training data; provenance must precede cleaning and mixing.
From Bytes to a Tokenizer—and What Deserves to Scale | Stanford CS336 Lecture 1
CS336's first lecture does not treat building a language model from scratch as reenacting every old technique. It separates mechanics, mindset, and intuitions, then uses BPE to show how raw bytes become trainable tokens.
Count FLOPs and Memory Before Asking Whether a Model Fits | Stanford CS336 Lecture 2
Lecture 2 reduces model training to tensors, FLOPs, bytes, and time: use einops to track dimensions, arithmetic intensity and roofline analysis to identify bottlenecks, then trade compute for memory with gradient accumulation and activation checkpointing.
Scaling Laws Are Extrapolation Tools, Not Crystal Balls | Stanford CS336 Lecture 9
Lecture 9 begins with log-log linear relationships between data and error, then uses scaling laws to compare architectures, optimizers, batches, and model-data allocations. The Chinchilla dispute shows how fitting methods, observed ranges, and deployment objectives change the answer.
Scaling Laws in Practice Must Scale Learning Rate and Batch Too | Stanford CS336 Lecture 11
Lecture 11 reads public recipes from MiniCPM, DeepSeek, Qwen, and Llama 3: hold most architectural ratios fixed, sweep learning rate and batch at small scale, then choose model/data allocation with IsoFLOPs. μP helps, but normalization, optimizers, and weight decay can break transfer.
Fireworks AI: From Serverless APIs to Custom Model Deployments
Fireworks AI puts open-weight model evaluation, dedicated GPU deployments, and LoRA customization behind one API surface. Serverless fits low-volume starts, On-demand fits sustained traffic and custom models, while reserved capacity adds enterprise capacity guarantees.
LiteLLM: From a Python SDK to a Self-Hosted AI Gateway
LiteLLM is not a model provider. It is a Python SDK and self-hosted proxy that normalizes 100+ LLM APIs, then centralizes routing, fallbacks, virtual keys, budgets, and observability at the gateway layer.
Promptfoo Deep Dive: Local-First LLM Evaluation and Red Teaming
Promptfoo combines prompts, providers, test cases, and assertions in YAML to produce repeatable local and CI evaluation matrices, with red teaming against the same targets. It lowers the testing barrier but does not remove output variance, LLM-judge bias, or hosted data-flow concerns.
Together AI: From Serverless Inference to Dedicated Endpoints and Fine-Tuning
Together AI puts serverless APIs for open-weight models, dedicated GPU endpoints, batch inference, and fine-tuning on one platform, letting teams validate per token before moving to reserved deployment when traffic or customization justifies it.
Vertex AI Explained: From Model APIs to Gemini Enterprise Agent Platform
Vertex AI is more than the Gemini API: it puts access to 200+ models, training, evaluation, deployment, and governance under one Google Cloud control plane. Since April 2026, its products and roadmap have moved into Gemini Enterprise Agent Platform, while the Vertex AI API, documentation paths, and many resource names remain in active use.
Inference-time Compute, Reasoning, and Embodied Agents | Berkeley CS288 Topic Guide
Units 15–18 place NLP models inside perception, reasoning, tool, and environment loops; the question shifts from next-token prediction to allocating inference compute and validating multi-step action.
Language Models and NLP: From Foundational Representations to Retrieval and Agents | Berkeley CS288 Course Overview
CS288 moves from n-grams to RAG, reasoning, and agents through 18 public slide units and three assignments; Berkeley-only recordings make this an A3 materials route, not a public video course.
Pre-training, Post-training, Generation, and Evaluation | Berkeley CS288 Topic Guide
Units 08–12 turn a base model into an interactive system: pre-training establishes capability, post-training shapes behavior, and generation plus evaluation determine how outputs are used.
Turning Retrieval, RAG, and Advanced Architectures into a System | Berkeley CS288 Topic Guide
Units 13–14 connect models to external knowledge; A3 requires data collection, QA annotation, indexing, and ablations under CPU and latency constraints.
Seven Answers to a Full Context Window, and No Consensus
Chroma's controlled study shows that even when it fits, a full context degrades performance. Coding agent vendors have landed on seven different responses: compact, hand off, prune, defer loading, isolate, train it into the model, or change the unit of work. Amp removed /compact outright, Atlassian argues summarization should be a last resort, and Cursor's A/B test measured a 46.9% token reduction. The three real disagreements come down to what each team is measuring.
Crawl4AI Complete Guide: From Markdown Crawling to Structured Extraction
Crawl4AI handles retrieval after a URL is known: use JsonCssExtractionStrategy for stable DOMs, and switch to LLMExtractionStrategy only when extraction needs semantic judgment or must tolerate irregular layouts.
Nine Lectures and Three Labs Are Public, but the Full Path Still Uses Three External Services | MIT 6.S191 Course Overview
MIT 6.S191's 2026 edition publishes nine lecture videos, slides, three software labs, and solutions, making it an A3 self-study course. The supplied path still depends on Google/Colab, Comet, and OpenRouter for Lab 3, while unaffiliated learners do not receive MIT credit, project feedback, or API credits.
From Languages to Information: An NLP Intro That Teaches Both tf-idf and LLMs | Stanford CS124 Course Overview
CS124 is the first course in Stanford's NLP track: ten weeks from tokenization, n-grams, text classification, information retrieval and embeddings to LLMs and Transformers. The textbook is the free Speech and Language Processing, and the nine assignment repos plus the lab problems and solutions are public; the pre-recorded videos, quizzes and autograding need a Stanford account.
NLP with Deep Learning: From word2vec to Building Your Own Transformer | Stanford CS224N Course Overview
CS224N is Stanford's course on natural language processing with deep learning: it goes from word vectors and backpropagation to Transformers, every later lecture builds on Transformers, and the four assignments include writing a decoder-only Transformer yourself. The current slides and assignments are public, and Assignment 3 ships unit tests that run on a laptop; the complete public recordings come from an earlier offering and no longer match the current syllabus.
Agentic AI: Formal Methods Against Hallucination | Stanford CS224V Course Overview
CS224V is Stanford's Agentic AI course: it teaches how to constrain LLM agents with formal semantics, knowledge graphs and formal methods so they fabricate less, with two assignments and then a project that carries most of the grade. The fourteen lecture decks, the reading list, the assignment handouts and the open-source systems the course uses (STORM, WikiChat, SUQL, Genie Worksheets) are public; there are no public recordings, the posted slides are deliberately incomplete, and the credentials the assignments need go only to enrolled students.
Stanford CS25 V6: A Course Called Transformers United Whose First Two Talks Weren't About Transformers
CS25 is Stanford's 1-unit seminar where attendance is the only homework and anyone can audit. Of the nine talks in the Spring 2026 season, the three worth your time are Albert Gu on the inductive biases of SSMs vs Transformers, Charles Frye on serving inference across thousands of GPUs, and Victoria Lin on what native multimodality still hasn't solved.
Language Modeling from Scratch: Five Assignments and a GPU Budget You Work Out Yourself | Stanford CS336 Course Overview
CS336 is Stanford's course on building a language model from scratch: five assignments cover the tokenizer and Transformer, training-systems optimization, scaling laws, data processing, and alignment and reasoning RL, with no code skeleton, only tests. The lectures, recordings, all five assignment repos and the leaderboards are public; from Assignment 2 on you supply your own compute, and the course-sponsored compute and grading are not available.
AI SDK Message Parts: The Data Skeleton of a Conversation UI
The AI SDK splits an AI message into a parts array — text, reasoning, source-url, tool-* — each an independent typed fragment (introduced in v5, unchanged since). That data structure dictates how modern AI conversation UIs are written: render by switching on part.type, handing each fragment to its component. This post unpacks the design logic of the parts model, useChat's streaming behavior, and how it became the foundation for component libraries like AI Elements.
LLM Application Design Interview Guide: From RAG to Agent Architecture
LLM Application Design is the hottest new interview topic in 2025-2026. Key focus areas: RAG pipeline chunking/retrieval/reranking design, agent tool-use and planning loops, context window management strategies, guardrails and safety design, and LLM application evaluation methods. Interviewers especially value whether you've hit real-world pitfalls.
NLP & LLM Interview Guide: From Tokenization to RLHF
The dividing line in LLM interviews is whether you've actually used these things. High-frequency topics: BPE tokenization logic and multilingual challenges, pretraining objectives (CLM vs MLM), three levels of fine-tuning (full/LoRA/prompt tuning), RLHF workflow and failure modes, prompting as engineering practice, and the difficulty of LLM evaluation with current methods.
Self-Improving Agents: The Course and Reading Route | Stanford CS329A Course Overview
Generating a correct candidate and recognizing it are different capabilities. This series follows the Autumn 2025 syllabus through a course overview and seventeen session guides covering all seventeen content sessions; five guest sessions have no public material and use public material from the speakers on similar topics instead; video Parts 7–9 map to sessions 8, 17 and 20.
A Reading Guide to Stanford's CS Courses: Ordered by Prerequisites, from CS106A to CS336 | Stanford's Main-Line CS Courses Course Map
Stanford CS rests on CS103, CS107, CS109, CS111, and CS161; CS221 names three of those plus CS106B as preparation. This guide combines official prerequisites with an explicitly editorial reading order and marks public-material and offering risks.
Cost, Latency, and Availability Across Six Exams: One Topic Tested From Three Altitudes
Google PMLE, AWS AIF-C01 and AIP-C01, Microsoft AI-103 and AI-500, and NVIDIA NCP-GENL all test how to make a GenAI application fast, cheap, and reliable — and they form a three-rung ladder: AIF-C01 asks whether you know cost scales with tokens, AIP-C01 and the two Microsoft exams ask whether you can instrument and control it, NCP-GENL asks whether you can change the model and the hardware. Three different altitudes. NVIDIA works at the kernel and quantization layer (Model Optimization 17% + GPU Acceleration 14% = 31%, the heaviest single cost/latency block in the whole series), AWS and Microsoft at the application layer (three caching tiers, token caps, chargeback), and Google at the MLOps layer (CPU/GPU/TPU evaluation, data vs model parallelism, scaling serving backends by throughput). The shared core is eight levers, but each lever becomes a different question at each altitude. This post deliberately carries no prices and no hardware specs — that is the part of this topic that rots fastest.
Multimodal Models, First Half of 2026: Native Fusion vs. Bolted-On Vision, and Why Leaderboards Contradict Each Other
Pure image understanding has flattened out — four frontier models all clear 80% on MMMU-Pro within 3 points of each other. The real differentiation is video, long-document OCR, and realtime speech, each with a different leader. But the most useful lesson from assembling these rankings is that two credible sources named different Video-MME leaders more than 10 points apart — and that July and August each turned the field over again.
NVIDIA NCA-GENL (Generative AI LLMs Associate): The Name Says LLM, Half the Blueprint Is Classical ML
NCA-GENL is usually what a job posting means by 'NVIDIA Generative AI / LLM certification.' But the official blueprint diverges sharply from the name — Core Machine Learning and AI Knowledge 30%, Software Development 24%, Experimentation 22%, Data Analysis 14%, Trustworthy AI 10% — with LLM and RAG content scattered at bullet level rather than forming a domain, alongside spaCy, NumPy, Keras, and cross validation. The other thing to know first: the official recommended training was replaced in autumn 2026, and each area now lists just two items, free open-source deep learning course materials on GitHub and one LLM application development course. Official specs: $125, 1 hour, 50–60 items, two-year validity, English only, pass/fail with no score reported.
NVIDIA NCP-GENL (Generative AI LLMs Professional): 31% Is GPU and Model Optimization, and Registration Isn't Open Yet
NCP-GENL is NVIDIA's professional-level LLM credential — $200, 120 minutes, 60–70 items. What separates it from every other GenAI exam is where the weight sits: Model Optimization 17% plus GPU Acceleration 14% is 31% on quantization, distillation, pruning, distributed parallelism, and CUDA profiling — not on calling APIs. Two things to know first: the Register button says Coming soon, so you cannot sit it yet; and the description text for Model Optimization and Fine-Tuning in the web page's weight table is misplaced, so prepare those two areas from the official PDF study guide.
Cloudflare Workers AI Model Picking Guide: By Use Case, Price, and Context
The Workers AI catalog currently holds 84 models. For general chat pick glm-4.7-flash ($0.06 / $0.40 per M, 131K context), for vision pick gemma-4-26b-a4b-it ($0.10 / $0.30, 256K), for cheap high-volume steps pick granite-4.0-h-micro ($0.017 / $0.112), and for embeddings pick qwen3-embedding-0.6b or bge-m3 (both $0.012 per M). This post is updated on a schedule.
A Coding Agent Is, Underneath, a While Loop | Stanford CS146S Week 1
Week 1 of CS146S is 'build Claude Code in 200 lines' plus a dissection of production system prompts. The agent loop really is that small. The course slides close with four things Claude does underneath, one of them being `<system-reminder>` tags scattered everywhere to stop the model drifting — which appears in no official documentation.
Deep Reinforcement Learning: Putting RLHF Back Inside the RL Frame | Stanford CS230 Lecture 5
The third reason Go can't be learned with supervision is the interesting one: the ground truth itself is ill-defined — the strongest human doesn't play their best moves every day, and even their best move isn't optimal. The last 15 minutes or so map RLHF fully back onto RL: the agent is the model being fine-tuned, the action is the next token, an episode is one full generation, and the reward is extremely sparse.
Introduction to Deep Learning: The Two Moments Prompting Stops Being Enough | Stanford CS230 Lecture 1
CS230's first lecture is a course overview, but Andrew Ng spends most of it on three things: why scaling works, when prompting stops being enough, and why he thinks 'don't learn to code' is one of the worst pieces of career advice ever given.
Context and Memory: Where Agents Actually Fail
Chroma tested 18 frontier models and all of them degrade as input grows — as a cliff, not a slope. Memory failures are usually retrieval failures in disguise. And the real cost of KV cache is bandwidth, not storage: every generated token reads the whole cache.
Security: Prompt Injection Can Only Be Contained in the Harness
In November 2025 three frontier labs jointly broke all 12 previously proposed prompt-injection defenses. EchoLeak's payload passed Microsoft's own dedicated classifier. So the goal is not blocking every attack — it is surviving the ones that land, and that is harness work.
Drawing the Lines: Agent, Workflow, RAG, and MCP
The line between workflow and agent is who decides the steps — the developer at design time, or the model at run time. By that definition most LLM systems in production today are workflows. Plus a usable test for choosing between RAG and an agent.
The Protocol Layer: MCP, A2A, ACP, Skills
MCP governs agent-to-tool, A2A governs agent-to-agent, Skills govern reusable knowledge. The test is whether the data changes: if it changes between calls you need MCP; if it's stable enough to write down, a skill file is simpler and has no runtime that can fail on its own.
The Model Is a Component, the Harness Is the System
Microsoft, OpenAI, Salesforce, Stripe and three others independently say the same thing: reliability comes from the engineering around the model. And 'give the deterministic parts back to code' has been shipped as a product four separate times — Agent Script, Procedures, runtime, blueprints.
anydoc: 14 Office Formats to Markdown, Firecrawl's Rust Answer
Firecrawl's open-source Rust conversion library turns 14 office formats (including legacy .doc / .ppt / .xls) into GFM at a 4.7ms median — 109× faster than Docling under the same timing basis. The trade-off: it does no OCR at all.
The Three-Layer Ladder of Document Parsing: Pick the Layer Before the Tool
The most common mistake in feeding documents to an LLM isn't picking the wrong tool — it's picking the wrong layer. Structure already in the file goes to the conversion layer (milliseconds); text without structure goes to extraction; only inferred structure needs parsing. anydoc's 4.7ms against Docling's 513.6ms is a 109× gap, and most people jump straight to the most expensive layer.
AI Makes Things Smooth Exactly Where They Should Be Hard: What Generative AI Does to Learning
The single most-cited meta-analysis on ChatGPT in education (g = 0.867, ~500k views) was retracted by Nature in April 2026. But the positive finding was not overturned — the issue is that it measures performance while the AI is available. Bastani's PNAS RCT measured something else: +48% accuracy during practice with GPT-4, then 17% below never-users once access was removed.
AI Web Scraping Tools Landscape: A Selection Guide for 34 Open-Source Projects
From MarkItDown (175k stars, MIT) to curl_cffi (6k stars), a survey of 34 open-source tools for feeding data to AI. Categorized along five axes: whole-site crawling, AI browser agents, document conversion, smart extraction, and anti-detection infrastructure. The key to selection isn't which tool is best — it's scenario matching.
Text / Image to Lottie: A Landscape Overview of AI Animation Generation Tools
From the CLI tool kin3o to the CVPR 2026 paper OmniLottie — a survey of open-source approaches for converting text and images into Lottie animations, with performance benchmarks and selection guidance.
The Skill Management Revolution for LLM Agents: A Complete Landscape of Skill Lifecycle from Voyager to MUSE-Autoskill
MUSE-Autoskill (2026) introduces a five-stage skill lifecycle framework. Self-created skills achieve 60.35% (+7.16%) on SkillsBench overall, and an impressive 87.94% on tasks where skill generation succeeds — surpassing the human-authored skill ceiling. This post synthesizes six arXiv papers to map the full landscape of skill evolution research.
How to Rigorously Compare Before and After Agent Changes: From Golden Sets to Statistical Testing
Even with temperature=0, LLM outputs can still fluctuate by up to 15% in practice. To rigorously compare agent changes, you need a frozen golden set, at least 3 runs per query averaged out, LLM-as-judge blind evaluation (pairwise preference flip rate reaches 35%), and paired statistical tests -- not just running each version once and going by feel.
Agent Observability: From OTel Traces to Catching Hallucinations, Tool Misuse, and Infinite Loops
The industry has converged on using OpenTelemetry GenAI semantic conventions to turn every LLM call and tool call into a span. Detecting the three major failure modes then splits into three tracks: faithfulness + semantic entropy for hallucinations, framework-level symbolic guardrails for tool misuse, and max steps + action hash deduplication for infinite loops — all wired into a Final / Trajectory / Single-step three-layer evaluation framework.
Resource Rationality for Agents: Optimal Decisions Across Tokens, Tool Calls, and Latency
Agent decision-making under resource constraints is bounded rationality reborn: Rational Metareasoning uses VOC rewards to save 20-37% of tokens, BATS proves that adding budget without budget awareness is futile, FrugalGPT cascades cut costs by up to 98%, and Speculative Actions reduce latency by 20%. The three constraints ultimately converge into a single Pareto curve, and the overarching trend is moving from humans tuning knobs to models making resource-rational decisions on their own.
The Single Crack in Agent Security: From Prompt Injection to Trust Boundaries to Multi-Agent Worms
Three seemingly distinct agent security problems — tool output injection, trust boundaries, malicious agents — share the same root cause: LLMs flatten instructions and data into a single token stream, making them architecturally unable to distinguish between the two. Understand this through-line and you can trace every attack from EchoLeak (CVE-2025-32711, zero-click) to the Morris II AI worm, and see why 'making the model behave' doesn't work — only architectural constraints (six design patterns, CaMeL) do.
How Agents Decide Whether to Retrieve, What to Retrieve, and How to Merge: Three Decision Layers of Agentic RAG
Traditional RAG is a fixed pipeline of 'retrieve then answer.' Agentic RAG splits retrieval into three decision layers: when to retrieve (FLARE uses token probabilities; Adaptive-RAG uses a complexity classifier), what to retrieve (HyDE / RAG-Fusion / decomposition / Step-back), and how to fuse (RRF k=60 then cross-encoder rerank then compression -- Anthropic measured a -67% failure rate reduction). Key counter-intuitive insight: unnecessary retrieval hurts quality -- 'deciding not to retrieve' is a first-class capability.
Stop Hand-Tuning Prompts: From GEPA to Tool Descriptions, Automating Agent Behavior Optimization
Automatic prompt optimization (APO) has evolved from APE/OPRO to GEPA: replacing sparse rewards with linguistic reflection, winning over GRPO by ~6pp with 4-35x fewer rollouts. Meanwhile, tool descriptions are the overlooked prompt -- small wording changes can shift tool selection rates by 10x, and Anthropic's experiments show Claude self-rewriting tool descriptions outperforms human experts. These two lines are converging: eval-driven automatic optimization is eating hand-tuned prompts.
How to Build a Deep Research Agent: Multi-Turn Search Planning, Conflict Resolution, and Verifiable Conclusions
An autonomous research agent = four controllable stages: planning (decompose into sub-questions), retrieval loop (search -> read -> reflect on gaps -> search again), evidence arbitration (>=2 independent sources, typed conflict handling), and verifiable output (sentence-level citations + independent verification pass). Two approaches: training-based uses RL to learn end-to-end when to search (Search-R1 +41%); orchestration-based uses orchestrator-worker division of labor (Anthropic internal eval +90.2%, at ~15x token cost).
Machine Theory of Mind: How Agents Infer Other Agents' Intentions, Knowledge, and Goals
Inferring another's beliefs/goals/intentions from observed behavior is called Machine Theory of Mind. Three lineages: symbolic BDI, Bayesian inverse planning, and deep learning ToMnet. The biggest controversy in the LLM era is that GPT-4 still trails humans by >10 points on ToMBench — are high scores genuine reasoning or statistical shortcuts?
Multi-Agent Error Propagation and Recovery: Borrowing Thirty Years of Weapons from Distributed Systems
At 99% accuracy per step over 100 steps, the error-free completion rate drops to just 36% -- error compounding is a structural problem, not something prompt tuning can fix. Distributed systems' supervisor trees, bulkheads, circuit breakers, sagas, and durable execution can be mapped almost one-to-one into agent orchestration. But LLMs introduce a failure class that traditional systems never had -- semantic errors that don't crash -- which require Inspector agents (recovering 96.4%) and redundancy voting (MAKER: one million steps with zero errors) to address.
Semantic Similarity ≠ Retrieval Relevance: Scenarios, Detection, and Remedies for Systematic Embedding Retrieval Failures
Cosine similarity and relevance systematically diverge across an entire class of scenarios: negation (most IR models score at or below random on NevIR), exact identifiers, numeric thresholds, and logical combinations (SoTA models achieve recall@100 < 20 on LIMIT) -- some of these hit the theoretical ceiling of the single-vector paradigm, and switching to a larger model will not help. Recommended remedy order: hybrid BM25 -> reranker (Anthropic measured -67%) -> upstream metadata routing -> domain fine-tuning / multi-vector.
How to Pick the Right Tool from Hundreds: The Collapse Curve of Tool Selection and Engineering Solutions
As tools scale up, selection accuracy doesn't degrade gracefully — it collapses: 4 to 51 tools drops from 43% to 2%, 10 to 100+ drops from 78% to 13.62%. The root fix is to stop stuffing everything in at once — Anthropic's Tool Search Tool uses defer loading plus retrieval to cut 85% of tokens, pushing Opus 4.5 accuracy from 79.5% to 88.1%. Description quality has conditional payoff: negligible in simple scenarios, but correctness jumps from 44% to 50% in multi-tool chaining.
A More Expensive Embedding Won't Save Your Traditional Chinese RAG: Three Layers of Failure and the Fix Order
Traditional Chinese RAG retrieval failures are a three-layer stack: embedding granularity defects (BGE/GTE from 0.1B to 7B all mis-rank on simple queries like 'fried chicken'), Simplified Chinese / English corpus dominance causing local vocabulary drift ('premium', 'exclusion clause' alignment is unreliable), and MTEB Chinese benchmarks being Simplified Chinese making model selection signals misleading. The fix is architectural: OpenCC normalization -> hybrid + jieba segmentation -> reranker -> local fine-tuning last -- and the prerequisite for all of it is building a Traditional Chinese eval set first.
arXiv Paper Quality Assessment Guide: From Endorsement Mechanisms to a Practical Checklist
arXiv does not perform peer review, and roughly 2% of submissions are rejected. Quality judgment relies on external signals: top venue acceptance > institution + open-source reproduction > citation quality. Includes a 20-item practical checklist and a 2026 toolbox (PWC has shut down).
Assembling LLM Agent Skills / Tools / Code Interpreter for Real: A Paper Reading Map
The hard part of LLM agents is not building function calling, skills, code interpreter, and document tools individually -- it is assembling them into a system that selects the right tool, writes code when needed, decomposes tasks, verifies results, and resists prompt injection. This post organizes the key papers into six engineering decisions: function calling reliability, tool/skill selection, code-as-action, multi-step planning, skill systems, and safety plus document generation.
How Do People Read arXiv Papers? A Complete Guide to Methods and Tools
Reading papers is two problems stacked together: methodology (Keshav's three-pass method, 5-10 min / 1 hour / 4-5 hours) determines how to read, and tools (arXiv HTML, alphaXiv, NotebookLM, Connected Papers, Zotero) shorten the time for each pass. AI lowers the barrier to understanding; judging correctness always stays with the human.
LLM Agent Tool Descriptions Determine Tool Selection: Three Bug Fixes
Rewriting tool descriptions from soft suggestions to hard rules (whitelist + consequence explanation) eliminated the LLM's incorrect tool selection; adding skip_signal=True fixed vector store double-indexing.
2026 LLM Inference Provider Free Tiers & Pricing: 40+ Services Ranked by Tier
For side projects, toy demos, and RAG prototypes, nobody wants to swipe a credit card on day one. This is a verified roundup of 40+ LLM inference providers still operating as of 2026/05, tiered by whether free resources auto-replenish or are one-time grants. Each entry notes credit-card requirements, supported models, paid starting prices, and catches. Chinese-origin providers including Zhipu GLM (permanently free), Doubao (2M tokens/day), Kimi, DashScope, and the Ollama local option are all included.
PageIndex: RAG Without Vectors — Turning Long Documents Into a Book With a Table of Contents
PageIndex skips chunking, embedding, and vector storage entirely. Instead it relies on LLM reasoning over a tree-structured table of contents the LLM itself wrote, reporting 98.7% on FinanceBench in its own vendor-run evaluation. It solves a different problem than vector RAG — finding the right section in a well-structured long document.
Groq Console: The Developer Platform for Running Open-Source Models on LPU Inference
Groq Console is the developer portal for Groq's in-house LPU chip, offering an OpenAI-compatible API, Playground, and free tier credits. Its selling point is running open-source models like Llama, Qwen, and DeepSeek at the fastest tokens/second on the market.
Gemma on Cloudflare Workers AI: A Pragmatic Choice for Traditional Chinese Applications
For running Traditional Chinese LLM workloads on Cloudflare Workers AI, the Gemma family follows instructions more reliably than same-tier Llama models. gemma-3-12b-it was marked deprecated on 2026-05-30; the current equivalent is gemma-4-26b-a4b-it: 256K context, Vision, Function calling, at $0.10 / $0.30 per M tokens.
AEO / GEO Tool Landscape: Input, Traffic, and Output Layers — From isitagentready to aeo-radar to Profound
AEO/GEO tools aren't a single category — they span three distinct layers: the input layer (is your website ready for AI to read), the traffic layer (how much are AI bots actually crawling), and the output layer (how is your brand mentioned in AI answers). This post maps out all three layers, from open-source self-hosted options to commercial SaaS.
A Book Written by AI Itself, Teaching You How to Build Software with AI
Encyclopedia of Agentic Coding Patterns catalogues 190 patterns to help you make the right software decisions in the age of AI-written code — and the book itself is autonomously written and maintained by an AI agent.
MarkItDown: Convert Any File to Markdown Before Feeding It to an LLM
A lightweight open-source tool from Microsoft that converts PDF, Office, images, audio, and more into Markdown — purpose-built for LLM pipelines.
Autoreason: Teaching LLMs When to Stop Self-Refining
Autoreason replaces the traditional critique-and-revise loop with a competitive multi-version evaluation mechanism (A/B/AB + blind Borda count), solving three structural problems in LLM self-refinement: prompt bias, scope creep, and lack of restraint.
2026 Personal AI Hardware Buying Guide: DGX Spark, Mac Studio, MSI AI Edge Compared
Comparing the NVIDIA DGX Spark, Apple Mac Studio M4 Ultra, ASUS Ascent GX10, MSI AI Edge, and more — helping you find the right local inference hardware.
NVIDIA DGX Spark: A Desktop AI Supercomputer That Fits a Petaflop on Your Desk
The NVIDIA DGX Spark is powered by the GB10 Grace Blackwell Superchip, 128 GB of unified memory, and delivers 1 petaFLOP of FP4 compute — starting at around $3,999 USD. It lets developers run 200B-parameter models locally and fine-tune 70B models, making it the most accessible NVIDIA AI development platform available today.
2026 Q1 Open-Source LLM Landscape: From Frontier Models to On-Device, a Complete Survey
2026 Q1 saw a full-blown open-source model explosion: on the LLM front, GLM-5, Kimi K2.5, and Qwen3.5 caught up with closed-source models; Embedding and Reranker are dominated by Qwen3 and BGE; speech has Voxtral TTS and Whisper V3; image has FLUX.2; and video has Wan 2.2 rivaling Sora. This is the complete navigation map.
OpenClaw Models, Advanced: Two-Stage Failover, the Real Cooldown Numbers, and Prompt Caching
OpenClaw's failover runs in two stages: rotate auth profiles within the provider, then fall back to another model. But what really governs behavior is who chose the model — a model you picked yourself with /model is strict, and its failure is reported rather than answered by some other model.
OpenClaw's Model Requirements and Provider Ecosystem: Provider, Model, and Runtime Are Three Different Things
OpenClaw's hard requirement for a model is tool use plus a large enough context — onboarding only auto-suggests a local model when it confirms tool support and at least a 16K context window. The easier thing to get wrong is that provider, model, and agent runtime are three separate layers: an `openai/*` ref does not mean Codex.
Langfuse Complete Guide: LLM Application Observability from Scratch
Langfuse is currently the most mature open-source LLM Observability platform. This post covers four core capabilities — Tracing, Prompt Management, Evaluation, and Datasets — showing you how to use them in real projects.
The Three Core Pillars of AI Agents: Context, Cognition, Action
An AI agent is not a black box — it is built from three layers: what it knows (Context), how it thinks (Cognition), and what it can do (Action). Understanding these three layers is the key to grasping why agents are sometimes brilliant and sometimes go off the rails, and how to design a truly effective agent system.
The Complete Ollama Guide: Run LLMs Locally with One Command
Ollama wraps llama.cpp in a Docker-style CLI + REST API, letting you run LLMs locally with a single command. This post covers core concepts, installation, API, hardware requirements, Modelfile customization, and what this tool is — and isn't — good for.
Prompt Engineering in Practice: Iteration Methodology, Common Mistakes, and Few-shot Optimization
Good prompts aren't written in one go — they're iterated into existence. Start with the simplest prompt, test with real cases, classify error types, and make targeted fixes. This article covers the three-part System Prompt structure, reasoning framework selection, few-shot optimization, token budget management, and six common mistakes.
Query Classification: Teaching Your RAG System How to Answer Each Question
Not every question needs full RAG. Classify queries with an LLM first, then route to the right execution path — saving cost and improving accuracy.
RAG Guardrails: Adding a Defense Layer to Inputs and Outputs
The attacks RAG systems face go beyond the technical level — Prompt Injection and Jailbreak are real threats. Both inputs and outputs need independent protection layers.
RAG Prompt Engineering: How to Design System Prompts and Context
Search found the right documents, but the LLM's answers are still poor — often the problem lies in prompt design. System prompt structure, context formatting, and instruction placement all affect output quality.
RAG vs Fine-tuning: It's Not Either/Or
RAG and Fine-tuning solve different problems. RAG gives the model new knowledge; Fine-tuning changes the model's behavior and style. In most cases you use both, not pick one.
NobodyClimb AI Architecture: Building a 20-Node RAG Pipeline on Cloudflare Workers
A dynamically composable RAG pipeline built on Cloudflare Workers AI (gemma-3-12b-it + bge-m3): 14 base steps + 6 LangGraph-specific nodes, with three strategy graphs (Baseline / Agentic / Plan-Execute) selected at runtime.