RNNs propagate hidden states through time; gradient products can shrink or grow. LSTM gates control cell-state retention and updates, improving long-range signal flow without guaranteeing stability. Self-attention shortens paths between positions, while dense attention still has quadratic sequence-length cost.
Representation learning compresses raw data into a 'useful' vector: autoencoders force a meaningful latent space via reconstruction, VQ discretizes it into a codebook, and self-supervision turns 'mask-and-reconstruct' into free supervision.
Similarity-based representation learning does not reconstruct input; it directly shapes latent geometry: pull same-class representations together, push different ones apart. InfoNCE turns this into 'spot the positive among negatives', and alignment / uniformity give it interpretable metrics.
Take a net to infinite width and its random-init output becomes a Gaussian process (NN–GP); its training dynamics freeze into the Neural Tangent Kernel (NTK). This theory analyzes nets and, in reverse, guides us to design the 'right inductive bias'.
Generative models learn the data distribution p(x). Density models model probability directly, energy models use an unnormalized potential + sampler, GANs let a discriminator force realistic samples, autoregressive predicts the next token step by step, and diffusion dodges tricky maximum-likelihood via 'add noise then learn to denoise'.
The core of VAE is ELBO + reparameterization: log p(x) is replaced with E_q[log p(x|z)] − KL(q(z|x)‖p(z)); the encoder outputs μ/σ and z = μ + σ⊙ε (ε ~ N(0,1)) makes sampling differentiable. Training = reconstruction + KL in tension, which gives rise to β-VAE, posterior collapse, VQ-VAE, and related fixes.
The key to conditional generation is 'feed y into the model': cGAN concatenates y into G/D; cVAE passes y to both encoder and decoder; in diffusion, Classifier Guidance uses gradients from an external classifier to push samples toward a class, while Classifier-Free Guidance trains conditional + unconditional together and linearly combines them at inference — the latter is the standard weapon behind Stable Diffusion and Imagen.
OOD failure is not a bug, it's the i.i.d. assumption breaking: covariate shift (image style changes), label shift (class proportions change), concept shift (a word's meaning changes) each need different responses; the most common cause is the model latching onto spurious correlations (using grass as a cue for cows); IRM and domain randomization try to fix this in training data structure, test-time adaptation fixes it at inference.
Transfer learning's core insight is 'features learned on big data are good general-purpose representations': freeze the backbone and train only a linear head when downstream data is tiny; full fine-tune when data is plentiful; reach for LoRA / adapter when compute is tight. SimCLR and MAE removed the need for upstream labels and pushed downstream quality another notch.
ResNet's skip connections solve degradation, enabling 100+ layer nets; EfficientNet compound scales depth/width/resolution; ConvNeXt absorbs Transformer design to reclaim CV crown.
The business post (order 27) of the Sandbox Series. How Modal's sandbox grew from a feature to a $300M–$1B annualized run rate standalone product line, pricing and packaging strategies, and implications for build-vs-buy decisions.
The security post (order 24) of the Sandbox Series. Compliance requirements for regulated industries (finance, healthcare) using sandboxes — SOC 2, HIPAA, GDPR, ISO 27001, EU AI Act — audit process, data sovereignty, and cross-border restrictions.
The cost post (order 26) of the Sandbox Series. Compares wall-clock vs active-CPU billing models, cost differences across isolation layers, and how to estimate monthly cost and avoid hidden fees.
The business post (order 28) of the Sandbox Series. Sandbox lifecycle management: pause/resume, snapshot/hibernation, resume concepts, implementation, and applicable scenarios for long-running tasks.
The security post (order 23) of the Sandbox Series. How tool calls, file operations, and network requests inside sandbox execution build auditable trails, trace-level visibility, immutable logs, SIEM integration, and the delegation-chain identity gap.
The cloud landscape post (order 7) of the Sandbox Series. Summarizes positioning, pricing, cold start, isolation strength, and applicable scenarios for E2B, Daytona, Runloop, Vercel, Cloudflare, Modal, AWS Bedrock AgentCore, Alibaba OpenSandbox, Azure Container Apps — with a selection decision matrix.
The overview entry point (order 0) of the Sandbox Series. Explains at which isolation layer untrusted agent code should run, which provider to pick, and how to configure permissions, credentials, and network boundaries; technical details live in eight sub-series, this post is the map only.
The motivation post (order 1) of the Sandbox Series. Letting AI agents execute untrusted code carries direct risks: data exfiltration, privilege escalation, lateral movement, supply-chain poisoning, persistent backdoors. Sandboxing is not an optional add-on — it's the first line of defense.
HW0 checks CS181 prerequisites in four problems — y=Xw solvability, optimizing an objective, reasoning about randomness, and OLS in Python. The problem that slows you down most is the gap to patch before HW1.
HW1 uses an 800,000-year ice-core temperature dataset across four problems: kNN and kernel regression, a geometric proof of least squares, basis-function regression, and a probabilistic derivation of ridge and LASSO, ending with a coordinate-descent LASSO implementation.