Lectures
- Lecture 1: Introduction, Motivation and Applications of Deep Generative Models, Course Logistics (pptx)
- Lecture 2: Probability basics: Random Variables, CDF/PDF, Multivariate Gaussian, Asymptotics
- Lecture 3: Estimation Basics: Maximum Likelihood, MAP Estimation, Kullback-Leibler Divergence
- Lecture 4: Shallow Generative Models, Gaussian Mixture Model, Expectation Maximization (EM) Algorithm
- Lecture 5: Autoregressive Models: Introduction, NADE, RNADE
- Lecture 6: Autoregressive Models: WaveNet (slides by Vassilis Tsiaras)
- Lecture 7: Normalizing Flows: Definition, Properties, Change of Variables, Planar Flows
- Lecture 8: Normalizing Flows (cont.): Coupling Flows, Real NVP, Masked Autoregressive and Inverse Autoregressive Flows
- Lecture 9: Variational Autoencoders: Latent Variable Models, Variational Inference, ELBO, Reparameterization Trick
- Lecture 10: Variational Autoencoders (cont.): ELBO, Reparameterization Trick, beta-VAE, Posterior Copllapse
- Lecture 11: Energy-based Models: Introduction, Partition Function, Boltzmann Machines, Contrastive Divergence
- Lecture 12: Energy-based Models (cont.): Langevin MCMC, Score Matching, Noise Contrastive Estimation
- Lecture 13: Diffusion Probabilistic Models: Forward/Backward Process, ELBO, Denoising
- Lecture 14: Diffusion Probabilistic Models (cont.): Accelerated Sampling, Latent Space and Conditional Diffusion Models
- Lecture 15: Diffusion Probabilistic Models (cont.): GLIDE, CLIP, DALLE2, Imagen, Vision Applications
- Lecture 16: Generative Adversarial Nets: Learning by Comparison, Vanilla GAN, Discriminator, Generator, DCGAN
- Lecture 17: Generative Adversarial Nets (cont.): f-GAN, Wasserstain GAN, CycleGAN, StarGAN
Tutorials
- Tutorial 1: Python basics: numpy, pandas, matplotlib, pytorch
- Tutorial 2: PyTorch basics: multivariate Gaussian, data loader, NN intro
- Tutorial 3: Preparatory for 1st Homework
- Tutorial 4: Multilayer Perceptron, GPU Training, 2D CNNs, Feature Visualization, Weight Normalization, Skip & Residual Connections
- Tutorial 5: Recurent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Gated Linear Units (GRUs)
- Tutorial 6: PGFPlots, Transformer Theory (Attention Is All You Need), PyTorch's Transformer, Custom Transformer Implementation
- Tutorial 7: Implementing basic Normalizing Flows
- Tutorial 8: Implementing basic Variational Autoencoders
- Tutorial 9: Implementing basic Denoising Diffusion Models