Home
MIT 6.7960
Deep Learning
Lecture 1: Introduction to Deep Learning
Lecture 2: How to Train a Neural Net
§
Gradient-Based Optimization
§
Bells and Whistles
§
Gradient Difficulties
§
Backpropagation
Lecture 3: Approximation Theory
Lecture 4: Architecture: ConvNets
Lecture 5: Architectures: Sequence Modeling
§
Sequence Modeling
§
Causal CNNs
§
Recurrent Neural Networks (RNNs)
§
Long Short-Term Memory (LSTM)
§
Deep RNNs
§
Advantages and Disadvantages
§
Memory
Lecture 6: Transformers
§
CNN Limitations
§
Tokens
§
Query-Key-Value Attention
§
Self-Attention
§
Transformer Architecture
§
Positional Encodings
§
Autoregressive Models (GPT, Claude)
Lecture 7: Generalization Theory
Lecture 9: Representation Learning: Introduction and Methods
§
Representations
§
Unsupervised Learning: Auxiliary Targets
§
Reconstruction-Based Methods
§
Autoencoders
§
Similarity-Based Methods: Contrastive Learning