CSL864: Special Topics in Artificial Intelligence: Probabilistic Graphical Models



General Information

Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)

Class Timings/Venue:
  • Time: Tue,Thu, Fri. 11:00 - 12:00
  • Venue: Bharti 204

Office Hours:
  • Time: Tuesday 12:30 pm to 1:15 pm
  • Venue: Bharti 209


Teaching Assistant: Dinesh Khandelwal (email: csz138294 AT cse.iitd.ac.in)

Announcements

  • [Mon Mar 31]: Assignment 2 (Part B) is out (details below)!
  • [Sat Mar 15]: Minor 2 Rescheduled! New Date/Time: Wed Mar 26, 2 pm - 3 pm. Venue: Bharti 501.
  • [Sun Mar 9]: Assignment 2 (Part A) is now out (see below)!
  • [Sun Feb 16]: Assignment 1 has been updated (see below)!
  • [Sun Jan 26]: Assignment 1 is out! Details below.
  • [Sun Jan 26]: Class Timing have changed! We will run the class in the regular F slot hours now.
    Tue, Thu,Fri: 11:00 - 12:00.
  • [Sat Jan 18]: In Class Quiz: Tuesday January 28. Duration: Half an Hour (Approx). Syllabus: Material Covered up to the Class Preceding Jan 28.
  • The Class Timings and the Venue has been finalized now! Check above for details.
  • [Sat Jan 4]: Next Class will be held on Tuesday Jan 7 from 2 pm to 3:30 1:30 pm to 3:00 pm.

Course Objective and Content

Obective: This course is meant to be the first graduate level course in the area of Probabilistic Graphical Models (PGM). PGMs have emerged as a very important research field during last decade or so with wide range of applications including Computer Vision, Information Retrieval, Natural Language Processing, Biology and Robotics. This course aims to provide students with a comprehensive overview of PGMs. The course content will include introduction to Probabilistic Graphical Models, directed and undirected representations, inference and learning algorithms and practical applications. This course is also meant to provide the required background for pursuing research in this area.

Content: Basics: Introduction. Undirected and Directed Graphical Models. Bayesian Networks. Markov Networks. Exponential Family Models. Factor Graph Representation. Hidden Markov Models. Conditional Random Fields. Triangulation and Chordal Graphs. Other pecial Cases: Chains, Trees. Inference: Variable Elimination (Sum Product and Max-Product). Junction Tree Algorithm. Forward Backward Algorithm (for HMMs). Loopy Belief Propagation. Markov Chain Monte Carlo. Metropolis Hastings. Importance Sampling. Gibbs Sampling. Variational Inference. Learning: Discriminative Vs. Generative Learning. Parameter Estimation in Bayesian and Markov Networks. Structure Learning. EM: Handling Missing Data. Applications in Vision, Web/IR, NLP and Biology. Advanced Topics: Statistical Relational Learning, Markov Logic Networks.

Note: All the topics above may not be covered in the course.

Week-Wise Schedule

WeekTopic Book ChaptersClass Notes/
Supplementary Material
1 Introduction, Basics KF Chapter 1, 2 Introduction
2 Bayesian Networks KF Chapter 3 Bayes Net-1
Bayes Net-2
3 Markov Networks KF Chapter 4 Markov Network-1
Markov Network-2
4,5 Factor Graph Representation, HMMs, CRFs,
Expoential Family
KF Chapter 4,8 Factor Graphs
Log Linear Models

Hidden Markov Models
Condiation Random Fields
6,7 Exact Inference: Variable Elimination KF Chapter 9 Variable Elimination-1
Variable Elimination-2
8,9,10 Exact Inference: Junction Tree Algorithm.
Belief Propagation (Loopy or not)
KF Chapter 10,11 Clique Tree Message Passing-1
Clique Tree Message Passing-2
Cique Tree Message Passing-3
Loopy Belief Propagation
Max-Product Belief Propagation
11,12 Sampling Based Approximate Inference:
MCMC, Metropolis Hastings, Gibbs Sampling,
Importance Sampling
KF Chapter 12 Sampling Based Inference - Basics
Forward Sampling, Likelihood Weighting, Importance Sampling
Importance Sampling
Markov Chain Monte Carlo
Markov Chain Monte Carlo-1
Gibbs Sampling, Metropolis Hastings
Gibbs Sampling - Additional Notes
13 Learning: Overview, Learning in Bayesian
Networks, Learning in Markov Networks
KF Chapter 16,17,19 Parameter Estimation, Expectation Maximization
14 Advanced Topics/Applications/Revision

Scribed Notes

Template: Latex Source File, Sample Pdf File

Additional Reading

Review Material

Topic Notes
Probability prob.pdf
Borrowed from Andrew Ng's
Machine Learning Course at Stanford

References

  1. Probabilistic Graphical Models: Principles and Techniques. Daphne Koller and Nir Friedman. First Edition, MIT Press, 2009.
  2. Learning in Graphical Models. Michael Jordan (ed.). MIT Press, 1998. Collection of Papers.
  3. Probabilistic Reasoning in Intelligent Systems. Judea Pearl. Morgan Kaufmann, 1988.

Other Places where a Similar Course is Offered

Assignment Submission Instructions

  1. You are free to discuss the problems with other students in the class. You should include the names of the people you had a significant discussion with in your submission.
  2. All your solutions should be produced independently without referring to any discussion notes or the code someone else would have written.
  3. All the non-programming solutions should be submitted using a hard copy. If you are writing by hand, write legibly.
  4. Required code should be submitted using Sakai Page.
  5. You should archive all your submission (code) in one single zip file. This zip file should be named as "yourentrynumber_firstname_lastname.zip". For example, if your entry number is "2008anz7535" and your name is "Nilesh Pathak", your submission should be named as "2008anz7535_nilesh_pathak.zip
  6. Honor Code: Any cases of copying will be awarded a zero on the assignment. An additional penalty of 5 points will also be imposed (on the total course points out of 100). More severe penalties may follow.
  7. Late Policy: You will lose 20% for each late day in submission. Maximum of 2 days late submissions are allowed.

Assignments

  • Assignment 2
    • Part B. Due Date: Sunday April 27, 11:50 pm. (On Sakai).
    • Part A. Due Date: Thursday April 17 (In Class).
  • Assignment 1. Updated Feb 21.

      Due Date:
    • Theoretical Questions: Tue Feb 25 (in Class).
    • Implementation Questions: Wed Feb 26. 11:50 pm.

    Grading Policy

    Assignments (2) 12% each
    Quizzes (2) 3% each
    Minors (2) 15% each
    Major 40%