ML fundamentals interviews test whether you can diagnose the gap between 'the metric looks great' and 'production is on fire.' Today covers four high-frequency topics: why AUC-ROC inflates under heavy class imbalance, why cross-entropy beats MSE for classification (it comes down to vanishing gradients), whether bagging or boosting fixes variance versus bias, and the common misconception that multicollinearity hurts prediction — it only hurts interpretability.
Week 4 enters ML: supervised classification (k-NN, SVM, Perceptron), model evaluation, RL basics (MDP, Q-learning, ε-greedy). Projects: Shopping (purchase prediction with k-NN) and Nim (learning to play via Q-learning).
A/B testing turns a product change into an estimate with uncertainty. A useful report covers effect size, confidence, guardrails, randomization, and launch risk.
Large-sample normal approximation describes the behavior of estimators, not raw data. It is useful, but dependence, boundaries, and distribution shift can make it unreliable.
Bayesian inference updates uncertainty about an unknown parameter by combining prior belief with the likelihood from observed data, producing a posterior distribution.
Bias checks whether an estimator is centered correctly, variance checks sampling fluctuation, MSE combines both, and consistency asks whether the estimator approaches truth as sample size grows.
Bootstrap estimates uncertainty by resampling from the observed sample with replacement, rebuilding many sample-like datasets, and watching the statistic fluctuate.
Chi-square tests compare observed counts with expected counts. First decide whether the problem is goodness-of-fit for one categorical variable or independence for two categorical variables.
Distributions are names for data-generating situations, not formula cards. Learn when Bernoulli, Binomial, Poisson, and Normal distributions fit a problem.
A confidence interval puts a point estimate back inside sampling fluctuation. Computing bounds is only the first step; you also need to explain standard error, critical values, and coverage.
Data type determines the statistical tools you can use. Start with categorical, numeric, count, and time-ordered data, then choose summaries that fit the question.
At the final review stage, train problem recognition: identify data type, unknown quantity, and decision goal before choosing a formula and writing a contextual conclusion.
Expectation describes long-run center; variance describes fluctuation. This post computes E[X], E[X^2], and Var(X), then connects them to average loss and model stability.
Experimental design decides whether a result can be interpreted. Randomization, control, blocking, replication, blinding, and pre-specified outcomes give inference a usable foundation.
Fisher information uses likelihood curvature to measure how well the data locate a parameter; larger information usually means a smaller standard error for the MLE.
A confidence interval is built by defining the target estimate, describing its sampling error, and choosing a rule that turns uncertainty into a range.
A generalized linear model starts from the response type, chooses a suitable distribution, and uses a link function to connect the mean to a linear predictor.
A hypothesis test is a decision process under uncertainty: write H0/H1, choose alpha, compute a test statistic and p-value, then decide whether the data is strong enough to challenge H0.
The inference map starts with the question type: point estimate, uncertainty interval, decision test, likelihood model comparison, Bayesian update, or resampling.
The likelihood-ratio test compares the log likelihood of a restricted model with a full model; the usual chi-square reference only makes sense under nested-model and approximation conditions.
OLS is a useful baseline, but coefficient interpretation, inference, prediction, and diagnosis depend on assumptions about linearity, errors, independence, and variance.
Logistic regression estimates probabilities first. Classification decisions come later, when thresholds turn those probabilities into actions under real error costs.
Logistic regression connects a linear score to a probability between 0 and 1. Understanding odds, log odds, and odds ratios prevents wrong coefficient interpretations.
MAP maximizes the posterior. After taking logs, the prior becomes a penalty term, which connects Bayesian estimation to L1, L2, and regularized ML objectives.
MLE fixes the observed data and compares which parameter values make that data most plausible; log likelihood turns products into sums and connects directly to negative log loss.
Method of Moments matches sample moments to theoretical population moments, then solves for parameters. It is not always the most efficient method, but it builds the first intuition for parameter estimation.
Nonparametric methods are not assumption-free. They relax fixed distributional forms, often gaining flexibility while paying in efficiency, interpretation, or overfitting risk.
Past papers train question-analysis discipline, not fortune-telling. Each problem should return to data type, unknown quantity, statistical tool, calculation path, and contextual conclusion.
Joint PMF problems require listing every cell. Marginalization, conditional probability, and variable transformations are all sums or regroupings of the original cells.
Probability problems are often hard because the viewpoint changes. Define events first, then distinguish conditioning, independence, mutual exclusivity, and Bayes' rule.
A sample is the data, a statistic is a function of the sample, and a sampling distribution is the distribution of that statistic under repeated sampling.
Random variables turn uncertain outcomes into numbers. PMF, PDF, and CDF then let you compute discrete probabilities, continuous interval probabilities, thresholds, and model-score distributions.
A regression table is not a p-value list: coef, SE, t, F, and R-squared answer effect size, uncertainty, single-coefficient tests, overall model signal, and in-sample explanation.
Regularization adds a preference against extreme parameters. Ridge, Lasso, and weight decay trade some training fit for a model that generalizes more reliably.
A reproducible workflow preserves the evidence chain from data to conclusion. Results need data versions, code, seeds, environment, metrics, and raw outputs.
Sampling makes sample statistics fluctuate, and standard error describes that fluctuation. This post separates SD, SE, sampling distributions, and CLT, then connects them to benchmark uncertainty.
A sampling distribution describes how a statistic fluctuates under repeated sampling. Means, proportions, and variances each connect to common distributions used in intervals and tests.
The series does not finish all of statistics. It gives beginners a working map for exams, ML/AI evaluation, causality, Bayesian thinking, time series, and mathematical statistics.
Simple linear regression uses one X to describe the average change in Y. Slope, intercept, residuals, and squared error form the smallest supervised learning model.
Do not start statistics exam prep by memorizing formulas. Start with the sequence of data, probability, sampling, inference, regression, then connect those ideas to model evaluation, A/B testing, and uncertainty in ML/AI.
Time-series data have order. Random splits can leak future information into training and make forecasting or monitoring results look better than they are.
Two-group comparisons start by classifying the outcome and the design: numeric or binary, independent or paired. That choice determines the standard error, test statistic, and conclusion.
GPUtw.ai is a Taiwan-based short-rental GPU cloud. Its main value is not maximum scale, but Taiwan data centers, prepaid credits, Jupyter/ComfyUI/Ollama/vLLM templates, Vault storage, and team billing. Public information is enough for a service introduction, not enough for procurement or production endorsement.
HW0 checks CS181 prerequisites in four problems — y=Xw solvability, optimizing an objective, reasoning about randomness, and OLS in Python. The problem that slows you down most is the gap to patch before HW1.
HW1 uses an 800k‑year ice‑core temperature dataset to implement three regression models (OLS, RBF kernel, MLP) and compare them on the same data, laying the groundwork for later classification and deep‑learning assignments.
CS181 2026 is A3 with hw0–6 as the weekly clock (no public recordings); 2025 adds a practical, 2024 has two midterms, 2023 was taught by Weiwei Pan. Start with HW0, then follow hw1→hw6.
Computational biology PhD turned UW-Madison professor Sebastian Raschka launched Ahead of AI on Substack in 2022, publishing monthly deep dives into LLM papers and architectures. Four years later: 200K+ subscribers, zero sponsorships, and a book-newsletter flywheel that proves low frequency and high depth can win in a crowded AI newsletter market.
2021 was the year diffusion models surpassed GANs, self-supervised learning made theoretical breakthroughs, and reinforcement learning confronted weaknesses in its evaluation methodology. NeurIPS received a then-record 9,122 submissions, ICLR’s Score-Based Generative Modeling paper became a theoretical foundation for the diffusion ecosystem, and ICML delivered substantial work on optimization theory and the dynamics of self-supervised learning.
2022 was the year diffusion models took center stage, Chinchilla scaling laws rewrote large-model training, and Chain-of-Thought turned reasoning into an ability that prompts could elicit. NeurIPS passed 10,000 submissions; three of its 13 Outstanding Papers directly concerned diffusion; and Chinchilla and data pruning both challenged the belief that bigger was always better. On the eve of ChatGPT’s release, every required piece fell into place at that year’s conferences.
In 2023, LLMs took over the machine-learning conference agenda. NeurIPS received more than 12,000 submissions; both Outstanding Papers addressed large models, while runner-up DPO became a practical alternative to RLHF within two years. DreamFusion opened the text-to-3D field, ICML spotlighted LLM watermarking and learning-rate adaptation, and the Mamba preprint emerged as the first serious architectural challenger to the Transformer.
ML conference submissions exploded in 2024: NeurIPS received a record 15,671 papers, while ICML and ICLR passed 9,000 and 7,000. Research shifted from training ever-larger models toward spending inference compute more intelligently, making test-time compute scaling the year's defining new direction. VAR beat diffusion with next-scale image prediction, Rectified Flow became the theoretical foundation for Stable Diffusion 3, and ICLR gave its inaugural Test of Time Award to the original VAE paper.
ML conferences broke every submission record in 2025 and pushed peer review to its limit. NeurIPS received 21,575 papers and used more than 20,000 reviewers; ICML passed 12,000 for the first time, and ICLR reached 11,565. Reasoning and agents were the strongest trends. One NeurIPS runner-up, the conference's only perfect-score paper, challenged whether RLVR creates new reasoning ability. Awards for Alibaba Qwen's Gated Attention and a mechanistic theory of neural scaling laws showed a community moving from scaling at all costs toward understanding why scaling works.
ML fundamentals interviews don't test whether you can recite definitions — they test whether you can walk through a structured diagnostic when handed a train/val accuracy gap. Today covers four high-frequency topics: bias-variance decomposition and learning curve interpretation, geometric intuition for L1/L2 regularization and when to pick which, aligning loss functions with business objectives instead of accepting defaults, and why AdamW decouples weight decay from L2 regularization.
Finishing 07-280 means more than reading 24 guides: produce a search engine, supervised-model comparison, CNN/GPT-2 experiments, and a small RL-plus-MCTS system before choosing 07-380, 10-301, or a specialist course.
07-280 is CMU's new Spring 2026 AI+ML core: 24 lectures and 12 main assignments move from heuristic search and CSPs to AlexNet, GPT-2, and AlphaZero. Its public material supports self-study, but complete recordings, Canvas checkpoints, Gradescope, and staff feedback remain unavailable.
Lecture 1 uses an alien autoencoder, the scope of AI and ML, and AI history to establish the course's coordinate system: an intelligent system turns inputs into representations and decisions under uncertainty.
Lecture 5 formulates machine learning through `X → Y`, loss, risk, and empirical risk minimization: a training set only gives average observed loss, while the real objective remains generalization over an unknown distribution.
Lecture 6 recursively grows a tree from decision stumps, measures label uncertainty with entropy, and selects splits by `I(Y;W)=H(Y)-H(Y|W)`; this is computationally practical greedy ERM, not a global optimal-tree guarantee.
Lecture 7 applies ERM to linear functions and squared loss, moves from a one-dimensional slope to `argmin ||y-Xθ||²`, and derives the normal equation when `XᵀX` is invertible.
Lecture 8 moves from a one-dimensional parabola to vector gradients and compares batch GD, SGD, and mini-batches; the learning rate determines whether updates converge, oscillate, or diverge.
Lecture 9 models P(y=1|x) with a sigmoid instead of directly predicting 0 or 1, learns parameters with cross-entropy and convex optimization, and extends naturally to softmax regression.
Lecture 10 uses φ(x) to let linear models express nonlinear functions, then controls the resulting overfitting with train/validation/test separation, L1/L2 regularization, and model selection.
Lecture 11 expands a logistic unit into a multilayer network: linear layers produce z, activations produce a, and multiple neurons jointly learn a feature transform trained through a final loss.
Lecture 12 treats a network as a computation graph: the forward pass stores intermediates, the backward pass propagates upstream gradients, and local linear, activation, and softmax rules compute every parameter gradient efficiently.
Lecture 16 starts from likelihood p(D|θ), uses i.i.d. to factor the joint probability and logs to turn products into sums; Bernoulli MLE yields sample proportions, conditional Bernoulli yields logistic cross-entropy, and Gaussian noise yields squared error.
Lectures 1–12 form one decision pipeline: define states, moves, and objectives, then use heuristics, losses, regularization, and backpropagation to control an otherwise intractable search space.
DSPy replaces handwritten prompt strings with task Signatures, execution Modules, and Optimizers that compile better instructions and examples against a dataset and metric.
Hugging Face Hub is a collaboration layer for versioned models, datasets, and applications. Datasets handles data, Spaces runs demos, while Inference Providers and Endpoints provide managed inference.
Linear regression is more than a best-fit line: Chapter 1 connects squared loss to gradient descent, normal equations, maximum likelihood, and locally weighted regression.
Chapter 2 derives logistic loss from a sigmoid probability model, then contrasts it with the perceptron and extends it through softmax and Newton's method.
Chapter 3 uses exponential families, natural parameters, and link functions to place least squares and logistic regression inside one modeling template.
Chapter 5 replaces high-dimensional feature inner products with kernels, letting inner-product-based linear algorithms learn nonlinear functions without constructing the features.
Chapter 7 decomposes neural networks into composable modules and uses backpropagation and vectorization to explain how deep models can be trained efficiently.
Chapter 8 decomposes test MSE into irreducible noise, squared bias, and variance, then uses uniform convergence and VC dimension to explain when training performance transfers to new data. Double descent shows why parameter count is not a universal measure of complexity.
Chapter 9 presents three controls on generalization: explicit complexity penalties, optimizer-induced implicit regularization, and model selection on data excluded from training. MAP estimation then connects a Gaussian prior to an L2 penalty.
Chapter 10 introduces unsupervised learning through k-means: alternating updates make distortion non-increasing and numerically convergent, but do not guarantee a global optimum.
Chapter 11 starts from soft assignments in Gaussian mixtures, uses Jensen's inequality to construct the ELBO, interprets EM as alternating maximization over a variational distribution and model parameters, and extends the idea to VAEs through approximate posteriors and reparameterization.
Chapter 12 formulates PCA as geometric optimization: maximize projected variance along a unit direction to obtain the leading eigenvector of the covariance matrix. The top k eigenvectors give both maximum retained variance and minimum linear reconstruction error.
Chapter 13 models ICA as x=As: observations are unknown linear mixtures, and the goal is to estimate W=A^{-1} to recover independent, non-Gaussian sources. A Jacobian determinant enters the transformed density and leads to the Bell–Sejnowski likelihood update.
Chapter 14 starts with a fixed Gaussian noising Markov chain and learns to reverse each transition. The ELBO turns reverse-kernel matching into weighted noise prediction, while the continuous-time view explains reverse drift through the score ∇log p_t.
Lectures 19–25 connect rational decisions and VPI to machine learning, while Project 5 uses PyTorch for regression, classification, CNNs, attention, and an optional character-GPT.
Spring 2025 at people.eecs.berkeley.edu/~jrs/189s25 is the only A3 self-study edition with notes, videos, HW1–7, code/data and past exams; Fall 2026 at eecs189.org/fa26 has a 27-lecture schedule but most materials are not yet released and the rotating site can 404 old URLs.
07-380 Fall 2026 is the first offering of CMU's new AI II, 26 lectures from logic, planning and optimization to probabilistic graphs and generative systems; Lec01 and Prop Logic are public, HW1-7, six quizzes and the final project are still TBD — an A2→A3 transition with the 07-280 bridge.
HW6 combines generalization, MLE/MAP, probabilistic learning, fairness metrics, and social impact in one written assignment about assumptions and tradeoffs.
The final written assignment combines ensembles, clustering, representation, and recommendation to test whether you can choose a learning paradigm from problem structure.
Spring 2026 publishes material for 27 lectures and nine homework bundles; outsiders can do the core work but cannot access Panopto, Piazza, Gradescope, or official homework solutions.
In 2026, CMU recombined its separate general-AI and SCS machine-learning introductions into the 07-280 → 07-380 sequence. This is a redistribution of content and prerequisites, not a pair of simple course renames.
CS50 AI is Harvard's most complete public entry point, but the Summer 2026 course still uses 2020 recordings and assignment assets while the rolling OCW projects have moved to other editions. CS181 Spring 2026 exposes current homework and notes without current recordings; CS182 Fall 2026 has not yet completed an offering.
Logistic regression turns a linear score into a Bernoulli probability with sigmoid; the gradient xⱼ(y-ŷ) follows directly from the log-likelihood chain rule.
Lambda Cloud provides on-demand GPU VMs and 1-Click Clusters; it offers direct AI compute environments rather than automatically solving training, serving, and MLOps.
Replicate abstracts GPUs behind versioned models, predictions, Cog, and deployments; integrators still own version pinning, async workflows, webhook verification, data persistence, and spending limits.
RunPod Pods fit interactive and persistent GPU work, while Serverless fits queued or load-balanced inference; choosing incorrectly mixes persistence, cold starts, and retry semantics.
The three things you need to self-study CS229 run on three different clocks. The lecture notes are 278 pages and were recompiled in August 2026. The newest problem sets you can download are from summer 2020. The self-assessment Stanford Online tells you to attempt before enrolling is a PDF created in 2008. Seventeen lectures from spring 2026 are public, and the last three are mislabeled.
Berkeley has no standalone undergraduate AI degree. A workable path builds on the CS BA or EECS BS foundation, enters through either CS188's broad AI curriculum or CS189's mathematical machine learning curriculum, then branches into deep learning, NLP, vision, or reinforcement learning. Many 2025–2026 courses are A3, but the newest class, the newest stable URL, and the best self-study edition are not always the same.
CMU's current BSAI now runs through 07-280 and 07-380 before branching into an NLP/vision core and four AI clusters, but 07-380 does not debut until Fall 2026. The residual Spring 2026 materials for 07-280 and the complete 10-301/601 site already support self-study; retired 15-281 remains a useful legacy route.
MIT has offered Course 6-4, a formal BS in Artificial Intelligence and Decision Making, since 2022. For an outside learner, however, the current degree requirements, the 2025–2026 course sites, and the best OCW editions rarely line up. A workable route follows 6-4's programming, algorithms, linear algebra, and probability foundation, then selects among 6.S191, 6.3900, 6.4110, 6.7960, vision, and robotics according to what is actually public.
Every lecture in CS109's Summer 2026 offering ships with an official LLM Learning Guide — six concepts, a Learn prompt and a Test me prompt for each, written week by week across the quarter for a total of 23 PDFs. The same course's honor code Rule 4 forbids asking an LLM to solve your homework, and 65% of the grade sits in proctored exam rooms. Those two facts are halves of one design.
The finale recaps the CS161 toolbox and points toward LP duality, Reed–Solomon coding, and ML-assisted algorithms. Officially, this lecture has slides but no notes.
ML fundamentals interviews don't test formula memorization — they test whether you can explain concepts intuitively and hold up under follow-up questions. High-frequency topics: the practical meaning of bias-variance tradeoff, the selection logic for L1/L2 regularization, why cross-entropy beats MSE for classification, SGD vs. Adam tradeoffs, and how precision/recall priorities differ by scenario.
AI Engineer interviews go beyond ML — big tech emphasizes system design and coding, startups look for end-to-end delivery, and AI-native companies test LLM engineering depth. Strategy: identify your target company types first, then allocate prep time across six dimensions (ML fundamentals, system design, LLM applications, coding, paper reading, and behavioral).
Google's Professional ML Engineer exam guide was rewritten in 2026: Vertex AI is renamed Gemini Enterprise Agent Platform throughout, so older study material no longer matches the product names in the questions. This guide uses the official six-section weighting as its skeleton, listing what each section tests, which official materials cover it, and what to build — plus a study schedule whose reasoning is spelled out. Official specs: $200, two hours, 50–60 multiple-choice and multiple-select questions, two-year validity, 3+ years of industry experience recommended including 1+ year on Google Cloud.
Andrew Ng demonstrates error analysis on a deep researcher: columns are the pipeline stages, rows are 10 to 100 queries, you only look at the ones that went badly, and you mark each cell where something broke. The percentages don't have to sum to 100%. He says it takes three or four hours and saves weeks of going the wrong direction — and the fraction of people who actually do it is far below 100%.
Andrew Ng walks a face-recognition door system through the entire project lifecycle, and the whole lecture has one thesis: speed. He gives teams a two-day deadline, on the reasoning that 'time spent preparing data should be commensurate with the time it takes to train the model once.' It closes on a line: my job is to build something that actually works, and that is not the same as building something that works on the test set.
ArduPilot's ModeReason enum is the exhaustive list of on-board autonomous decisions: of 56 values, 43 are the aircraft deciding for itself, 21 of those because it detected something dangerous, and exactly one — SOARING_THERMAL_DETECTED — because it found an opportunity. As for end-to-end, PX4 mainline's mc_nn_control has a 10 KB tensor arena and a Kconfig default of n; ArduPilot mainline has none.