🌏 中文版
Video status: recordings require login or course authorization. Recording source
“Text plus images” does not specify the task. Text-to-image generation produces an image conditioned on text; a VLM may consume an image and produce an answer. Direction, output objective, and evaluation differ even when both use Transformers.
Choose a lesson
- Text-conditioned generation: work through its mechanism, example, and answer criteria.
- Vision-language models: work through its mechanism, example, and answer criteria.
- Condition injection: work through its mechanism, example, and answer criteria.
The course map lists the core path. This page is a topic guide, not an additional lecture. Choose the question you need to answer and follow its lesson.
A worked comparison
Teaching example: the prompt is “a red cup on a table.” A text-to-image system starts from a noisy latent, uses text conditioning, denoises, and decodes an image. A VLM starts with an existing picture, encodes visual features, bridges them into an LLM, and answers that the cup is red. The former needs image-content checks; the latter needs evidence that the answer is grounded in the supplied image. Replacing a red-cup input with a blue-cup image while holding the text fixed tests visual grounding, not image generation.
Check your understanding
Does a VLM naming the right color prove it can generate that image?
Answer and criterion
No. Reading and generation use different output spaces and objectives. A multimodal system may support both, but each capability needs its own test.
How to use this map
Before reading the answer, identify the inputs, what changes, and the outputs. Then complete the linked lesson’s variation. If two mechanisms seem identical, trace where their conditions or data paths diverge. This page checks relationships and boundaries; it does not replace the derivations, implementations, or evaluations in the individual lessons.
The source edition is Spring 2026. Teaching examples are not official assessments or reconstructed classroom speech. Recordings remain subject to course authorization.
Course video sources
The official Spring 2026 course site directs recordings to SCS Panopto. On 2026-10-12, the anonymous reader returned an unloaded folder and a sign-in prompt; it did not provide playable lecture content. This establishes an access limit, not the absence of recordings. This guide uses public written materials and provides no reconstructed classroom speech or original-audio transcript.
Update Log
-
2026-10-11: Rebuilt the teaching scope, worked example, and self-check; retained the original URL and publication date.
-
2026-10-10: Added explicit video status and checked recording sources and access notes.
-
2026-10-10: Rechecked video status. The login wall is confirmed (anonymous Panopto folder lists no videos and prompts sign-in); the official pages link no public YouTube version, so the status is unchanged.
References
- CMU 10-423/623/723 Generative AI (Spring 2026): home page
- Course schedule — L12 (Feb 23), L13 (Feb 27), Quiz 4 coverage
- Lecture 12 slides: DPO / Text-to-image / Latent diffusion
- Lecture 13 slides: Vision-language models (inked version)
- HW4 handout (hw4.zip) — written-section points
- Bie et al. 2023: RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model — source of the slides' timeline
- Yu et al. 2022: Scaling Autoregressive Models for Content-Rich Text-to-Image Generation (Parti)
- Rombach et al. 2022: High-Resolution Image Synthesis with Latent Diffusion Models
- Radford et al. 2021: CLIP
- van den Oord et al. 2017: Neural Discrete Representation Learning (VQ-VAE)
- Beyer et al. 2024: PaliGemma
- Bai et al. 2023: Qwen-VL
- Liu et al. 2024: World Model on Million-Length Video And Language With Blockwise RingAttention (LWM)