Lecture 9 starts from the two steps of k-means, assign and average, and explains why the algorithm always stops yet only reaches a local optimum. Replacing hard assignments with probabilities gives the Gaussian mixture model and the two steps of EM: an E-step that is Bayes' rule and an M-step that is a weighted average. The lecture ends with the picture behind EM, drawn with Jensen's inequality: each round builds a lower bound that touches the log-likelihood and climbs to its top. The full derivation waits for Lecture 10.
The first half uses Jensen's inequality to build a lower bound on the log-likelihood, the ELBO, then picks Q to be the posterior so the bound is tight at the current parameters. From that it follows that no round of EM can lower the likelihood, and plugging in a Gaussian mixture recovers last lecture's weighted-average formulas. The second half writes PCA as maximizing uᵀΣu over unit vectors; the answer is the eigenvector of the covariance matrix with the largest eigenvalue.
Each forward step multiplies the data by √(1−β_t) and adds Gaussian noise with variance β_t. Unrolled, x_t = √ᾱ_t·x_0 + √(1−ᾱ_t)·ε̂_t, and nothing here is learned. The reverse direction is learned with a Gaussian p_θ(x_{t−1}|x_t): apply last lecture's ELBO to the whole trajectory, split it with the KL chain rule, and the loss at each step is ‖μ̃_t(x_t,x_0) − μ_θ(x_t,t)‖²/(2β̃_t). The lecture stops at this expression; simplifying it into noise prediction is left for Lecture 12.
The first half collapses the diffusion ELBO into one expression: every time step is a squared error for guessing the noise that was added, which gives a five-step training procedure and a one-loop sampler. The second half opens the foundation-model unit: the two-stage paradigm of pretraining plus adaptation, linear probe, finetune, LP-FT, and why LoRA trains only 2dr parameters yet saves very little compute.
This lecture unpacks the SimCLR loss into a B×B similarity matrix: each row is one softmax classification problem whose answer is the diagonal entry. The diagonal gets pulled up and everything else gets pushed down, so the model learns that similar things have nearby vectors without any labels. The second half uses the same vectors for semantic retrieval, then connects that to RAG.
This lecture takes a large language model apart in three layers: tokenization turns text into token IDs, the chain rule splits the probability of a whole text into a sequence of next-token conditionals, and training sums the cross-entropy at every position. The second half opens up the Transformer: inner products of queries and keys go through a softmax to give weights, the weights average the values, and a causal mask blocks future positions. The cost of a single attention layer grows with the square of the sequence length T.
During generation, every head has to keep the keys and values of all past positions in GPU memory, and this KV cache caps the batch size. GQA lets τ query heads share one set of keys and values, shrinking the cache to 1/τ; sliding window attention cuts the T² compute down to T·w; MoE runs only 8 of 128 experts for each token. The second half covers how to use a trained model: few-shot and zero-shot prompting leave the parameters alone, SFT changes them, and its loss is computed only on the answer tokens.
The first 55 minutes of this lecture write sequential decision making as an MDP (states, actions, transition probabilities, rewards, discount factor) and show that the Bellman equation for a fixed policy is a system of linear equations. The last 20 minutes derive policy gradient: ∇p = p∇log p turns the gradient back into an expectation, the transition probabilities drop out of the gradient because they don't contain θ, and the lecture stops at the formula "sum of log-probability gradients × sum of discounted rewards", leaving variance reduction for the next lecture.
The last lecture of the term finishes policy gradient: past rewards and any baseline that depends only on the state can be removed from the weight without changing the expectation. It then derives PPO's clipped objective from importance sampling, splits it into four cases to see which samples still carry a gradient, and compares the change CISPO makes. Finally it writes token-by-token generation in a language model as an MDP, gives a reward of 1 only when the answer is right, and trains long chains of thought using the average over 8 samples of the same question as the baseline.