Testing LoRA Fine-Tuning: Gradients, Data, and Evaluation | CMU 10-423 Assignment 3

🌏 中文版

Video status: recordings require login or course authorization. Sources

This guide follows CMU 10-423 Spring 2026 public assignment handouts, starter code, and related slides. Worked examples are original teaching additions.

Falling loss is not the whole assignment

Official HW3 combines written ICL, PEFT, and DPO questions with GPT-2 LoRA programming. This guide uses an original miniature test, not solutions to assigned questions; original task details and references remain below. Read LoRA and DPO. LoRA chooses trainable parameters, while SFT or DPO chooses their objective.

A credible experiment checks three separate things: adapter-only updates, correct next-token data, and performance on unseen data. One training loss does not certify all three. A tiny batch is useful for wiring tests but memorizing it is not generalization.

An original forward/backward test

Use fixed W=0 with shape 4×3, A=[1,0,−1], zero B, rank one, scale one, input x=[2,1,1]ᵀ, target t=[1,0,0,0]ᵀ, and loss=0.5‖y−t‖². This is a wiring test, not a sentiment model. Initially Ax=1, y=0, and loss=0.5. B's gradient is [−1,0,0,0]ᵀ and A's is zero. An SGD step with learning rate 0.1 changes B to [0.1,0,0,0]ᵀ. The next output is [0.1,0,0,0]ᵀ and loss is 0.405.

W must remain identical; B changes; A need not change on the first step. A can receive a gradient on the next backward pass. This detects no_grad around the whole block, an optimizer missing B, or initializing both A and B to zero. The last case stalls both gradients and differs from the valid first-step zero gradient for A.

Connect the test to language-model data

A sentiment instruction supplies a prompt and a short label response. A label is not necessarily one token, so taking one final position may discard part of the answer. Inspect one record's token IDs, decoded text, attention mask, and labels. Ignore prompt and padding in the loss; include response and EOS according to the design. After shifting, the position predicting the first answer token must see only the prompt.

Try overfitting two original examples first. Success supports basic gradient and data wiring; failure calls for checking masks, shifts, trainable parameters, and learning rate. Then use fixed train/validation/test splits and avoid duplicated reviews or paraphrases crossing splits. Tune rank and learning rate on validation, not test.

Evaluation and reload tests

Report complete-answer classification accuracy and output-format errors separately from trainable parameters, peak memory, and time. Comparisons need the same backbone, data, tokenizer, and budget. Fewer trainable parameters do not alone establish better performance.

Save the adapter, start a fresh process, and load the same backbone version. With a fixed input, eval mode, and decoding policy, compare logits or greedy output. This catches missing layers, forgotten scaling, and mismatched backbone versions. With dropout enabled, bitwise equality is not a sensible requirement.

Check your understanding

With learning rate 0.2, B's first component becomes 0.2 and the next loss is 0.5·0.8²=0.32. If loss falls but W changes, the implementation fails adapter-only training. Perfect train accuracy with unchanged test performance is not evidence of successful adaptation; investigate overfitting, label mapping, and distribution differences. Run official starter tests unchanged: original tests supplement, rather than replace, assignment specifications.

Connect the tests to the official assignment

The HW3 handout and starter define the task. Spring 2026's Applying and Adapting LLMs assignment totals 66 points, with written ICL, PEFT, and DPO questions plus GPT-2-medium LoRA instruction tuning on Rotten Tomatoes reviews. The toy test checks wiring and does not replace its data, model, or experiments.

The starter includes seven tests in test_lora.py. Implement the module before import and testing. After tests pass, establish the untuned baseline and compare rank, alpha, full tuning, and prompt variations. The assignment uses the Best Val Checkpoint, not a checkpoint selected on test performance. Its default experiment has rank 128, alpha 512, LoRA dropout 0.05, learning rate 2.5e−4, 80 iterations, batch four, and accumulation 32. These are assignment settings rather than universal optima. The notebook's full-tuning dropout differs from the handout default; follow and document the individual question's setting.

The handout estimates 25–30 minutes per T4 training run and about two minutes for zero-shot evaluation. This rewrite did not rerun those timings. Requirements are not comprehensively pinned, and WandB logging is part of the experiments. Record environment versions and verify current quotas and downloads rather than treating historical retrieval as current availability. Slot A prohibits AI assistance; Slot B permits AI and collaboration, subject to current announcements. External readers can study the public starter, but this guide does not provide Gradescope feedback, private recitations, or official assignment solutions.

Course video sources

The official Spring 2026 course site directs recordings to SCS Panopto. On 2026-10-12, anonymous retrieval returned an unloaded folder and a sign-in prompt, without playable classroom content. This establishes an access limit, not the absence of recordings. This guide uses public assignment handouts, starter code, related slides, and linked primary papers; examples are original teaching additions, not a transcript. Official recording folder.

Update Log

• 2026-10-11: Rebuilt the teaching body, integrated the assignment specification, and added original numerical tests and answered variations.
• 2026-10-10: Added explicit video status and checked recording sources and access notes.
• 2026-10-10: Rechecked video status. The login wall is confirmed (anonymous Panopto folder lists no videos and prompts sign-in); the official pages link no public YouTube version, so the status is unchanged.

References

• CMU 10-423/623/723 Generative AI (Spring 2026): home page and syllabus — Slot A/B rules
• Coursework page — HW3 handout and Overleaf links
• HW3 handout (hw3.zip) — hw3.pdf, starter code, test_lora.py, run_in_colab.ipynb
• HW3 read-only Overleaf template
• Course schedule — HW3 coverage, release and due dates, recitation link, Quiz 3 and programming test dates
• HW3 recitation slides (Google Slides, 401 from outside CMU)
• Rotten Tomatoes dataset (Hugging Face) — train 8,530 / validation 1,066 / test 1,066
• Hu et al. 2021: LoRA: Low-Rank Adaptation of Large Language Models
• Rafailov et al. 2023: Direct Preference Optimization
• Weights & Biases — experiment tracking in train.py