Reasoning: From CoT and Test-Time Scaling to RL for Reasoning | NTU Yun-Nung Chen Applied Deep Learning Lecture 12
With pre-training data close to exhausted, models now improve by thinking more before they answer. Lecture 12 of NTU ADL covers this in two halves. The first is test-time scaling: the model stays fixed and inference spends more compute, through short CoT, parallel sampling followed by selection (majority voting, answer confidence, an outcome reward model), step-level scoring with a process reward model combined with search, and sequential revision. The second is learning to reason: the weights are updated, with reasoning data written by people, generated by the model and filtered, or produced by a stronger teacher model, or with reinforcement learning that only tells the model whether its answer was right, as in DeepSeek-R1-Zero.