
COL 865: Special Topics in Computer Applications  Deep Learning
Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)
Class Timings/Venue:
Slot H. Time (Modified): Monday  2:00 pm  3:20 pm. Thursday  12:00 noon  1:20 pm.
 Slot H. Time: Monday, Wednesday  11:00 am  11:50 pm. Thursday  12:00 noon  12:50 pm.
 Venue: Bharti 305.
Office Hours:
Teaching Assistants:
Ankit Anand (csz138105 AT cse), Akshay Gupta (cs5130275 AT cse)
Announcements
 [October 2, 2017] Check out the updated grading policy!
 [September 22, 2017] We are planning to use Microsoft Azure Credits for part of the assignments in the course. Thanks to Microsoft for the credits!
 [September 22, 2017] Course Webpage is finally up! Yay!!
Obective:
This course is meant to be the first graduate level course in deep learning. Deep Learning is
an emerging area of Machine Learning which has revolutionized the progress in the field during
last few years with applications found in NLP, Vision and Speech to name a few domains. This
course is intended to give a basic overview of the mathematical foundations of the field, and
present the standard techniques/arhitectures which become basis for more advanced ones. About a
3rd of the course will focus on latest research in the area through research paper dicussions.
Without an implementation, no deep learning class can be complete. Students will get to implement
some of the architectures (CNNs/LSTMs etc.) on a GPU to test on large datasets. Students will also
likely get some experience with cloud computing facilities such as Microsoft Azure and/or other
HPC systems.
For 201718 Sem I offering: We plan to use Data Science Virtual Machine (DSVM) service provided
by Microsoft Azure so that students can collaborate with each other for building deep learning
models on large amount of data.
Content: Basics: Introduction. Multilayered Perceptrons. Backpropagation. Regularization:
L1L2 Norms. Dropouts. Optimization: Challenges. Stochastic Gradient Descent. Advanced Optimization
Algorithms. Convolutional Neural Networks (CNNs). Advanced Architectures for Vision. Recurrent Neural
Networks. Long Short Term Memory (LSTMs). Gated Recurrent Units (GRUs). Attention. Word Vectors.
Generative Adversarial Networks (GANs). Deep Reinforcement Learning. More Recent Advances in
the field.
WeekWise Schedule
Week  Topic  Book Chapters  Class Notes/ Supplementary Material
 1  Introduction, Motivation  

2  Mulitlayered Perceptrons, Backpropagation  Goodfellow et al. Chapter 6 

3  Regularization Techniques  Goodfellow et al. Chapter 7 

4,4.5  Optimization  Goodfellow et al. Chapter 8 

5  Convolutional Neural Networks (CNNs)  Goodfellow et al. Chapter 9 

6  CNNs  Advanced Architectures 
Slides 

7  Recurrent Neural Networks (RNNs)  Goodfellow et al. Chapter 10 

8  LSTMs, GRUs  Goodfellow et al. Chapter 10 

8.5  Word2Vec 
Slides 
9  Generative Adversarial Learning (GANs)  

10  Research Paper Discussions  

11  Deep Reinforcement Learning  

12  14  Research Paper Discussions  

Guidelines & Detailed Schedule
Chapters 1  5, Goodfellow et al. 
References
 Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville. 2016.
Assignment Submission Instructions
 You are free to discuss the problems with other students in the class. But the final solution/code that you
produce should come through your individual efforts.
 Required code should be submitted using Moodle Page.
 Honor Code: Any cases of copying will be awarded a zero on the assignment. Additional
penalities will be imposed based on the severity of copying. Any copying cases run the chances of being
escalated to the Department/DISCO.
 Late policy:
You will lose 20% of the score for every late day in submission. Maximum of two late
days are allowed for any given assignment. You are allowed a total of 2 buffer days for the two
programming assignments. There is no penalty if your submission stays withing the limit of the 2 buffer days
(total). Any delay beyond 2 days will result in a zero on your submission(s).
Assignments

Assignment
1: Implement AlexNet on a Subset of classes in the Imagenet Dataset. Implement an LSTM Cell (tentative) [Weight : 10%]. Due: Sunday October 15, 11:50 pm.

Assignment 2. Open ended implementation of a deep learning system. Details to be decided.
We are planning to use Microsoft Azure Credits for part of the above assignments. Thanks to Microsoft for the credits!
Grading Secheme (Tentative)
Assignment 1  10% 
Assignment 2  15% 
Class Presentation + Reviewing  15% (Presentation:6% + Reviewing:9%) 
Minor 1/Assignment 0  6% 
Minor 2  20% 
Major  34% 
