Basic information:
Course description
This course aims at covering the basics of modern deep neural networks. In specific, the first part will introduce the fundamental concepts in neural networks including network architecture, activation function, loss, optimization, gradient and initializations etc. Then, the second part will describe specific types of different deep neural networks such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) and attention-based Transformer, as well as their applications in computer vision and natural language processing. In the final part we will briefly discuss some recent advanced topics in deep learning including graph neural networks, unsupervised representation learning, deep reinforcement learning, generative adversarial networks (GANs), etc. In this course, the hands-on practice of implementing deep learning algorithms (in Python) will be provided via homeworks and course project.
Textbooks
We will have required readings from the following textbook:
Announcements
Links
Paper review list
Important: Read the requirements (click here) for paper review. Here is a review example for your reference. (Paper review lists will be gradually added.)| # | Due Date | Theme | Novel Representative Papers for each theme |
|---|---|---|---|
| 1 | Feb 6 | Foundations of Modern Deep Learning | A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. NeurIPS, 2012. |
| 2 | A. Paszke et al., "PyTorch: An imperative style, high-performance deep learning library," in Proc. NeurIPS, 2019. | ||
| 3 | K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. ICLR, 2015. |
Schedule will be updated as needed by instructor to accomodate student learning.
| Date | Lecture | Reading | Note |
|---|---|---|---|
| Weeks 1–6: Foundations + Basic Neural Networks | |||
| Jan. 13 | Introduction to the course, overview of deep learning | Chapter 1: Introduction | |
| Jan. 20 | Supervised learning | Chapter 2: Supervised learning | Paper Review List 1 out |
| Jan. 27 | Shallow neural networks (SNN) | Chapter 3: Shallow neural networks | |
| Feb. 3 | Deep neural networks | Chapter 4: Deep neural networks | Paper Review List 2 out |
| Feb. 10 | Loss functions + Optimization: Fitting Models (SGD, Momentum, Adam) | Chapter 5: Loss functions + Chapter 6: Fitting models | |
| Feb. 17 | Training Deep Networks: Gradients, backprop, initialization, optimization, evaluation | Chapter 7: Gradients and initialization + Chapter 8: Measuring performance | Team Assignment Completed (2/20) |
| Weeks 7–15: Modern Architectures (CNN → ResNet → RNN → Transformers → GNN → Generative Models) | |||
| Feb. 24 | Convolutional Neural Networks (CNNs Part 1) | Chapter 10: Convolutional networks (10.1-10.3) | |
| Mar. 3 | CNNs Part 2 (Downsampling, Upsampling, Applications) + Regularization | Chapter 10: Convolutional networks (10.1-10.3) + Chapter 9: Regularization (9.1 - 9.2) | |
| March 9 - 15, 2026 | Spring break (No Class) | Project Proposal Due (3/15) | |
| Mar. 17 | Residual Networks (ResNets) | Chapter 11: Residual networks | |
| Mar. 24 | Recurrent Neural Networks (RNNs) – sequence modeling, vanishing gradients, LSTM/GRU intro. | Deep Learning by Goodfellow et al : Ch 10 | |
| Mar. 31 | Transformers – self-attention, encoder/decoder, applications | Chapter 12: Transformers | |
| Apr. 14 | Advanced Topic: Graph Neural Networks | Chapter 13: Graph neural networks | |
| Apr. 21 | Advanced Topic: GANs or Diffusion Models – forward & reverse process | Chapter 18: Diffusion models | |
| Apr. 28 | (Tentative) Why Deep Learning Works, Ethics + Course Wrap-Up | Chapter 20: Why deep learning works + Chapter 21: Deep learning and ethics | Project Report (4/28) |
| May 4 - 8 | Finals Week | ||