Skip to content

Audio, Video, and World Models: A Reading Map | CMU 10-423 Topic Guide

Sep 30, 20261 min
Home›ai
TL;DRAudio representation, temporal relationships in video, and action-conditioned world transitions are different learning tasks. The earlier combined guide checked only audio at the end. This URL now serves as a reading map to one full guide each for Lectures 24–26; the Lecture 26 guide also covers the separate alignment deck.
#cmu-10423#ai-course#cmu#speech-to-text#video-generation#world-model#diffusion-model#multimodal
Series: Reading CMU 10-423 Generative AI (45 / 47)
Operational Risks and Alignment Theory: Tests and Assumptions | CMU 10-423 Topic GuideExams, Papers, and Projects: Three Forms of Learning Evidence | CMU 10-423 Topic Guide

Look up & save

Loading dictionary…

🌏 中文版

Video status: recordings require login or course authorization. Recording source

Audio representation, temporal relationships in video, and action-conditioned world transitions are different learning tasks. The earlier combined guide checked only audio at the end. This URL now serves as a reading map to one full guide each for Lectures 24–26; the Lecture 26 guide also covers the separate alignment deck.

Choose a lesson

The course map lists the core path. This page is a topic guide, not an additional lecture. Choose the question you need to answer and follow its lesson.

A worked comparison

Teaching world model: the state is character position x=0, with actions right and left. Define toy transitions f(x,right)=x+1 and f(x,left)=x−1. From the same initial state, next positions are 1 and −1; two right actions produce 2. This tests action intervention rather than visual continuity alone. A model that always moves to 1 fails the control test even if its frames look convincing. The audio lesson’s 98-frame spectrogram example tests representation, not video coherence or world dynamics.

Check your understanding

If a video follows a “walk right” text prompt but has no action input, does it pass an interactive-world-model check?

Answer and criterion

Not by itself. Change actions under the same state and history, inspect subsequent transitions, and test multi-step rollout. Text-conditioned video does not automatically provide that interface or capability.

How to use this map

Before reading the answer, identify the inputs, what changes, and the outputs. Then complete the linked lesson’s variation. If two mechanisms seem identical, trace where their conditions or data paths diverge. This page checks relationships and boundaries; it does not replace the derivations, implementations, or evaluations in the individual lessons.

The source edition is Spring 2026. Teaching examples are not official assessments or reconstructed classroom speech. Recordings remain subject to course authorization.

Course video sources

The official Spring 2026 course site directs recordings to SCS Panopto. On 2026-10-12, the anonymous reader returned an unloaded folder and a sign-in prompt; it did not provide playable lecture content. This establishes an access limit, not the absence of recordings. This guide uses public written materials and provides no reconstructed classroom speech or original-audio transcript.

Update Log

  • 2026-10-11: Rebuilt the teaching scope, worked example, and self-check; retained the original URL and publication date.

  • 2026-10-10: Added explicit video status and checked recording sources and access notes.

  • 2026-10-10: Rechecked video status. The login wall is confirmed (anonymous Panopto folder lists no videos and prompts sign-in); the official pages link no public YouTube version, so the status is unchanged.

References

Related

Condition Injection: The Roles of Attention, DiT, and Q-Former | CMU 10-423 Topic Guide1 minDiffusion Models: Why Adding Noise Creates a Training Task | CMU 10-423 Lecture 71 minTesting DDPM: Matching Timestep Coefficients to a Batch | CMU 10-423 Assignment 21 min