A Generative AI Learning Path: Reading the 35 Core Guides | CMU 10-423 Course Overview

🌏 中文版

Video status: recordings require sign-in or course authorization. This guide uses public slides and assignments; numerical examples are site-authored, not a lecture transcript.

This series uses the public Spring 2026 materials for CMU 10-423/623/723 Generative AI. Its goal is not a vocabulary of model names. You should be able to follow data into a model, trace learning signals into parameters, explain generation steps, and judge whether an output or experiment supports a claim. Each lecture has one guide; when a lecture covers several topics, the guide splits them into sections. Assignments, the practice exam, the paper report, and the final project each have their own guide.

The series contains 35 core teaching guides and 12 retained navigation guides: one overview, one guide for each of the 26 lectures, five for Assignments 0–4, and one each for the practice exam, paper report, and final project. Core guides follow prerequisite dependencies. Retained pages preserve existing URLs and compare related mechanisms while linking to complete lessons.

Diagnose your starting point

Prerequisites are Python, vectors and matrices, conditional probability, derivatives, and introductory machine learning. You need not have trained a large model, but you should distinguish data, tensor shapes, and parameters.

Original diagnostic A: three conditional token probabilities are 0.5, 0.4, and 0.25. Their prefix probability is 0.05. If you added them, begin with the RNN guide's chain rule. Diagnostic B: ŷ=wx, x=2, w=1, y=5, and L=(ŷ−y)²/2. The derivative with respect to w is −6. If you cannot trace it, use Assignment 0's scalar exercise.

Diagnostic C: a 4×4 grayscale image becomes nonoverlapping 2×2 patches. There are four tokens, each with four raw values. If you counted sixteen tokens, the CNN/ViT guide clarifies the distinction between pixels and patches. These are site-authored orientation exercises, not CMU admissions or exam requirements.

Three routes with concrete starting points

Language modeling: start with sequence likelihood, the Assignment 0 gradient exercise, causal attention, training/decoding, RoPE/GQA, and Assignment 1. Then study PEFT, ICL, SFT/RLHF, and DPO. Separate parameter updates, context changes, and output selection at every step.

Images and multimodality: establish CNN/ViT representations, then read GANs, Diffusion I/II, VAE, and Assignment 2. Continue with latent diffusion, VLMs, cross-attention/DiT, and Q-Former. The audio guide covers representation/recognition first, then generation; video introduces temporal relationships, and world models introduce actions. An attractive image or smooth video cannot validate every capability.

Systems and research: readers with modeling foundations can begin with scaling laws, MoE, distributed training, and efficient inference, followed by long context, reasoning, SSMs, risks, code agents, and alignment. Finish with the practice exam, paper report, and project. These connect a question to baselines, ablations, and evidence. Systems topics still require attention shapes and autograd rather than bypassing them.

Core reading map

The first column is this site's reading order, not an official lecture number. The second records the original scope. Lectures 11, 12, 15, 24, and 26 each cover two topics, handled as two halves of one guide. Worked examples and answers are site-authored rather than attributed to the instructor or presented as official solutions.

Reading order	Original scope	Core guide
0	Overview	A Generative AI Learning Path: Reading the 35 Core Guides
1	L1	RNN Language Models: Sequence Probabilities and Gradient Paths
2	HW0 / recitation	Autodiff in Practice: From Hand Gradients to a Training Loop
3	L2	Causal Attention: From Q, K, and V to the Next Token
4	L3	Training and Decoding: Why the Same Logits Produce Different Sequences
5	L4	Modern Transformers: What Position Rotation and KV Sharing Change
6	HW1	Testing minGPT Changes: Shapes, Masks, and Positions
7	L5	CNNs and ViT: Turning an Image into Model Inputs
8	L6	GANs: What the Generator and Discriminator Learn
9	L7	Diffusion Models: Why Adding Noise Creates a Training Task
10	L8	Diffusion Training and Sampling: Random Timesteps and Reverse Chains
11	L9	VAEs: Reconstruction, KL, and Reparameterization
12	HW2	Testing DDPM: Matching Timestep Coefficients to a Batch
13	L10	LoRA: What Changes When Most Weights Stay Frozen
14	L11 / L9 ICL section	ICL, SFT, and RLHF: From Changing Inputs to Changing Weights
15	L12	DPO and Latent Diffusion: Updating on Preferences, Conditioning the Denoiser
16	HW3	Testing LoRA Fine-Tuning: Gradients, Data, and Evaluation
17	L13	Vision-Language Models: Turning Images into Language-Model Inputs
18	L14	Cross-Attention and DiT: How Conditions Enter a Denoiser
19	L15	Q-Former and Scaling Laws: Bridging Frozen Models, Allocating a Fixed Budget
20	HW4	Testing Q-Former Text-to-Image: Gradients through Frozen Backbones
21	L16	Mixture of Experts: Separating Capacity from Active Computation
22	L17	Distributed Training: Splitting Data, Models, and Optimizer State
23	L18	Efficient Inference: FlashAttention, KV Caches, and Speculative Decoding
24	L19	Long Context: Fitting Input Is Not Reliable Retrieval
25	L20	Reasoning Models: Spending Compute on Verifiable Answers
26	L21	State Space Models: From Recurrence to Convolution
27	L22	Generative AI Risks: Turning Failures into Tests
28	L23	Code Generation and Agents: A Reliable Test–Repair Loop
29	L24	Audio Understanding and Generation: Samples to Text, Tokens Back to Sound
30	L25	Video Generation: Modeling the Time Axis
31	L26	World Models and Alignment Theory: Action Interventions and Corrigibility Assumptions
32	Official practice exam	Practice Exam: Diagnose Conceptual Gaps
33	HW623	Paper Reports: Turn a Model Summary into a Testable Argument
34	Final project	Final Projects: Build a Reproducible Generative-Model Experiment

Use assignments as learning checks

Assignment 0 matches hand derivatives to autograd. Assignment 1 checks shapes, masks, positions, and GQA. Assignment 2 aligns diffusion timesteps with image batches. Assignment 3 checks adapters and data splits. Assignment 4 follows gradients through frozen foundations and a trainable bridge. Establish a starter baseline, run its tests, add controlled variations, and then train.

Spring 2026 materials define separate Slot A/B rules, with an initial human-only submission and later AI-assisted corrections under specified conditions. Enrolled students should follow their semester's syllabus and handouts. Our independent examples neither replace those rules nor provide official submission answers. Public materials do not imply access to grading systems, manual rubrics, or every recitation supplement.

Small numerical exercises and unit checks can run on CPU; that does not make complete homework training equally cheap. Confirm hardware, environment, and data requirements for the actual scope. Record completed steps separately: passing tests, generating a sample, and evaluating quality establish different things.

What the retained guides do

These twelve pages keep their original URLs and provide comparisons and links to the full core lessons. Return to them when you need a relationship explained. They are not additional lectures after the core guides.

• Audio, video, and world models
• Cross-attention, DiT, and Q-Former
• Distributed training and efficient inference
• Exam, paper report, and project
• SFT, RLHF, and DPO
• Long context and SSMs
• PEFT and ICL
• Risks and alignment
• Scaling laws and MoE
• Text-to-image and VLMs
• Transformers and decoding
• Variational inference and VAE

What the sources establish

The primary basis is the official schedule, public slides, assignments, and assessment materials. A schedule is an index, not an exhaustive description of a deck. Lecture 8 introduces VAE; Lecture 9 closes with zero/few-shot material; Lecture 26 has separate world-model and alignment decks. Each guide's scope therefore follows the slides' actual content rather than filenames alone.

These are site-authored bilingual teaching guides, not lecture transcripts. The recording index is hosted on SCS Panopto. An anonymous folder view remains unloaded and requests sign-in; that establishes an access limitation, not an absence of recordings. Each guide retains official entry points. Unavailable speech, subtitles, or spoken whiteboard explanations are not invented.

Choose a route and complete its first numerical check before proceeding. If you can repeat model names but cannot explain the worked input and result, pause there and repair the gap rather than rush through the schedule.

Next

Course Video Sources

The official recording index is on SCS Panopto. The anonymous folder view is unloaded and requests sign-in; this is an access limitation, not evidence that recordings do not exist. Original audio and subtitles require the appropriate course access.

• Official Panopto recording index
• Official course homepage

References

• Official course homepage and syllabus
• Spring 2026 schedule
• Coursework
• Previous
• Aran Nayebi
• Matt Gormley
• a practice exam with solutions
• project handout
• HW0 recitation Colab
• Lecture 1 slides: Course Overview + RNN-LMs + Automatic Differentiation
• Lecture 2 slides: Transformer Language Models
• HW1 handout (zip)
• HW3 handout (zip)
• HW4 handout (zip)
• Practice Exam