Google PMLE, AWS AIF-C01 and AIP-C01, Microsoft AI-103 and AI-500, and NVIDIA NCP-GENL all test how to make a GenAI application fast, cheap, and reliable — and they form a three-rung ladder: AIF-C01 asks whether you know cost scales with tokens, AIP-C01 and the two Microsoft exams ask whether you can instrument and control it, NCP-GENL asks whether you can change the model and the hardware. Three different altitudes. NVIDIA works at the kernel and quantization layer (Model Optimization 17% + GPU Acceleration 14% = 31%, the heaviest single cost/latency block in the whole series), AWS and Microsoft at the application layer (three caching tiers, token caps, chargeback), and Google at the MLOps layer (CPU/GPU/TPU evaluation, data vs model parallelism, scaling serving backends by throughput). The shared core is eight levers, but each lever becomes a different question at each altitude. This post deliberately carries no prices and no hardware specs — that is the part of this topic that rots fastest.
Microsoft AI-500, AB-620, AB-100, NVIDIA NCP-AAI, and Claude CCAR-F all test multi-agent architecture, and they overlap on seven things: orchestration topologies, A2A and MCP, per-agent identity boundaries, three-layer memory, observability and agent replay, human-in-the-loop, and guardrails at four intervention points. But four vendors use four vocabularies for the same ideas, and each exam has objectives that don't transfer — Microsoft names four context-window failure modes nobody else names, 7% of NVIDIA's is locked to NeMo and NIM, and Claude tests SDK-level details like stop_reason. One correction along the way: Google PMLE's wall-to-wall 'Agent Platform' is a Vertex AI rename, not a multi-agent domain.
Pure image understanding has flattened out — four frontier models all clear 80% on MMMU-Pro within 3 points of each other. The real differentiation is video, long-document OCR, and realtime speech, each with a different leader. But the most useful lesson from assembling these rankings is that two credible sources named different Video-MME leaders more than 10 points apart — and that July and August each turned the field over again.
Most people assume GenAI certifications are built around prompt writing. CCAO-F gives Prompting 14% while Output Evaluation gets 21%; CCDV-F gives Prompt and Context Engineering 11.0%. What actually gets tested is structured output, injection-resistant prompting, dynamic context injection, context compression and caching, prompt lifecycle governance, and proving a prompt change helped — closer to context engineering and software engineering than to writing craft. None of the ten asks you to write a prompt on the spot; they are all multiple choice, so explaining why beats having a feel for it. The single most useful line comes from CCAR-F: when business logic must be guaranteed, 'change the prompt first' is usually the wrong answer.
Four certifications genuinely test RAG and retrieval evaluation: AWS AIF-C01 (chapters 2 and 3 total 52%, covering RAG, vector stores, and FM evaluation metrics), AWS AIP-C01 (11 of the 27 skill points in its 31% Domain 1 sit in vector storage and RAG), NVIDIA NCP-AAI (Knowledge Integration 10% plus Evaluation and Tuning 13%), and Microsoft AI-500 ('multi-agent RAG architecture' inside its 30–35% Develop area). Google PMLE contributes exactly one LLM-as-a-judge objective, and Claude CCDV-F — the developer certification people most readily assume covers RAG — has no retrieval objective across its eight domains, with Eval at just 2.6%. Includes a same-vendor foundational-vs-professional comparison, a four-vendor terminology map, non-transferable objectives, and a practice project.
OpenClaw has 386k stars to Hermes Agent's 232k, yet Hermes passed it on OpenRouter daily tokens back on 2026-05-10 (224B vs 186B). The nine self-hosted agents that appeared this year aren't nine competitors — they're nine incompatible answers to one question. CVE-2026-44112 broke OpenClaw's own sandbox, and in the Meta alignment director's inbox incident there was no attacker at all: context compaction ate the safety instruction.
A VS Code extension open-sourced by Microsoft employees that reads your local Claude Code / Codex / OpenCode session logs. The real payload is 45 Markdown rules: prompts under 30 characters, sending the next message within 15 seconds of receiving 20 lines of AI code, instruction files over 4,000 bytes — turning 'context engineering' into numbers you can argue with.
The course lists four techniques for directing agents: instruction files, hooks, commands, subagents. The instruction file is the only one loaded in full every startup, making it config rather than memory; hooks cover what instructions can't, because a rule can be ignored and a hook cannot; commands are the only one a human triggers. The course also marks just one and a half of seven task steps as human work.
Week 1 of CS146S is 'build Claude Code in 200 lines' plus a dissection of production system prompts. The agent loop really is that small. The course slides close with four things Claude does underneath, one of them being `<system-reminder>` tags scattered everywhere to stop the model drifting — which appears in no official documentation.
Factory breaks 'can an agent work in this repo' into eight pillars and five levels, and published real scores: CockroachDB L4 (74%), FastAPI L3 (53%), Express L2 (28%). The thesis is that agent readiness approximates the density of deterministic validation loops — linters, type checkers, tests are reward signals for agents.
The course measured AI SAST false positive rates at 50–100%, against 50%+ for traditional SAST — the genuinely new problem is nondeterminism: run the same prompt twice, get different results, and you can never answer "am I done scanning?" The course lists five agent attack vectors, one of which, intent breaking, attacks the agent's plan itself.
The Agent Skills spec fits in a sentence: a directory containing a SKILL.md. The real design is three levels of progressive disclosure — only name and description load at startup, the body loads on a match, bundled files load on demand. This site's own repo carries 35 skills and 7,893 lines of SKILL.md, and startup still costs only those 35 metadata pairs.
Google deployed AutoCommenter to tens of thousands of engineers and published the whole tuning process: suppressing 17 'technically correct but low-value' rules raised the useful ratio from 54% to 66%, with 80% set as the bar for the next rollout stage. Final comment-resolution rate landed around 40%. The bottleneck in AI code review was never detection — it's volume.
How an individual connects tools is a preference; how an organization does it is governance — who can touch what data, where keys live, whose budget it lands on. Anthropic's published record of ten internal teams contains a good indicator: security engineering accounts for 50% of all custom slash commands in the entire monorepo. Adoption doesn't spread evenly; it takes off first in teams that already build their own tools.
Background agents replace 'you watch it run' with 'it finishes and opens a PR.' Every vendor's design converges on the same parts: an isolated environment, external triggers (issues, Slack, Linear), and a PR as the output. The genuinely new problem is that you become the bottleneck — five agents finish at once, five diffs queue for you, and none of them know the others exist.
Fall 2026 compresses a full week of prompting into one bullet here and adds RePPIT (Research, Propose, Plan, Implement, Test) and MCP. Two RePPIT rules are worth stealing outright: always ask for exactly two proposals, and never let the instance that wrote the code review it. On the MCP side, Anthropic measured turning tools into code calls dropping 150,000 tokens to 2,000.
Stanford CS146S's Fall 2026 syllabus compresses prompting from a full week into a single bullet, drops the terminal and UI-generation weeks, and adds Agent Skills, Agent-Ready Codebases, Background Agents, and AI-Native Team. Grading moved too: the final project fell from 80% to 50%, with 30% now on open source contributions. This series reads all ten weeks.
The final session is 'self-running, self-improving software systems.' The parts all appeared in the previous nine weeks: deterministic validation loops, skills that can be written back, background agents, centralized governance. One easily missed proportion from the slides — coding is 30% of engineering time, and running it in production is the other 70%.
Researchers initially assumed neural networks are easy to fool because they're nonlinear. That was wrong — Goodfellow's 2014 paper argues the primary cause is their linear nature, and high dimensionality lets every tiny perturbation compound. The second half covers generative models: GANs' three pathologies, and why diffusion sidesteps two of them by adding noise and learning to remove it.
A BCG experiment found a jagged frontier: inside it, AI substantially improved consultants' work; outside it, AI made results worse — and people fell asleep at the wheel. The lecture also takes a strong position: avoid fine-tuning wherever possible, because by the time you're done tuning, the next model already beats your fine-tuned version.