CS 536: Course Description

This is a graduate course in supervised learning. The course will cover the theory and practice of methods and problems such as point estimation, naive Bayes, decision trees, nearest neighbor, linear classfication and regression, kernel methods, learning theory, cross validation and model selection, boosting, optimization, graphical models, semi supervised learning, reinforcement learning, deep nets etc.

Staff and Office Hours

  • [Instructor: Pranjal Awasthi, Office hours: Monday 2-3pm, Hill 448]
  • [TA: Yan Zhu, Office hours: Friday 10-11am, CBIM]
  • Textbooks

    There is no required textbook for the course. The lecture notes will serve as the primary reference. The following books are a good source for additional reference

  • Machine Learning and Pattern Recognition by Chris Bishop
    [link]
  • Machine Learning: a Probabilistic Perspective by Kevin Murphy
    [link]
  • The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman
    [link]
  • Grading

  • ~5 Homeworks (40%)
  • In class midterm(30%) (On March 10th)
  • Final Project (30%) (Details coming later...)
  • >90 A [85,90] B+ [80,85] B [75,80] C+ [70,75] C <70 F

    Homework Policy

    All homeworks must be typeset in LaTex and follow the [ACM formatting guidelines]. Homeworks must be submitted via sakai by 6pm EST of the due date. Late homeworks are not accepted. Due to the large size of the class, we will not accomodate any requests for regrading. The TA's decision in all such matters will be final. You are encouraged to discuss the homework problems provided you have spent enough time(> 24 hrs) thinking about the solution yourself. A discussion is meant to be a collaborative effort to help everyone involved understand the problem better. Asking for solutions is not considered a collaborative effort. In the end, you must write all the solutions in your own words. You must also write the names and Rutgers id's of the people that you discussed the homework problems with. We will follow the [Rutgers academic policy on cheating].

    Homeworks

  • [Homework 0](No need to submit)
  • [Homework 1][Solutions] [Data]
  • [Homework 2][Solutions] [Data]
  • [Homework 3][Solutions]
  • [Homework 4]
  • Final Project

    Lectures

    Date Subject Reading
    Jan 21 Point Estimation[Slides] http://www.stat.cmu.edu/~larry/=stat705/Lecture1.pdf http://www.stat.cmu.edu/~larry/=stat705/Lecture2.pdf http://www.stat.cmu.edu/~larry/=stat705/Lecture7.pdf
    Jan 25 Point Estimation and Naive Bayes[Slides] http://www.stat.cmu.edu/~larry/=stat705/Lecture7.pdf http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf (See Section 5) http://www.cs.columbia.edu/~mcollins/em.pdf
    Jan 28 Decision lists and Decision trees[Slides] Bishop book, Chapter 14.4
    Decision Tree chapter by Mitchell
    Decision Tree chapter by Nilsson
    Feb 1 Decision trees continued[Slides] Bishop book, Chapter 14.4
    Decision Tree chapter by Mitchell
    Decision Tree chapter by Nilsson
    Feb 4 Perceptrons[Slides] http://www.stat.cmu.edu/~cshalizi/350/lectures/25/lecture-25.pdf
    http://www.cs.columbia.edu/~mcollins/courses/6998-2012/notes/perc.converge.pdf
    Feb 8 Support Vector Machines[Slides] Bishop Chapter 7
    Feb 11 Support Vector Machines Continued[Slides] SVM Tutorial by Burges (See Section 3)
    Feb 15 Kernel Methods[Slides] Bishop Chapter 6 (See 6.1-6.3)
    A nice overview of Kernel methods
    A more detailed discussion of Kernel methods
    Feb 18 Linear Regression[Slides] Bishop Chapter 3
    Hastie, Tibshirani, Friedman Chapter 3
    Feb 22 Ridge Regression[Slides] Bishop Chapter 3
    Hastie, Tibshirani, Friedman Chapter 3
    Feb 25 Lasso[Slides] Bishop Chapter 3
    Hastie, Tibshirani, Friedman Chapter 3
    Feb 29 Statistical Learning Theory[Slides] Notes by Nina Balcan
    More notes by Nina Balcan
    March 3 Statistical Learning Theory II[Slides] Notes by Nina Balcan
    More notes by Nina Balcan
    More notes
    March 7 Uniform Convergence and Rademacher bounds[Slides] A nice survey of statistical learning theory
    Notes by Rob Schapire on rademacher bounds
    March 21 Boosting[Slides] Survey article by Rob Schapire
    Boosting Tutorial
    March 24 Graphical Models[Slides] Murphy Book, Chapter 19
    Notes by Kevin Murphy
    March 28 Belief Propagation[Slides] Murphy Book, Chapter 20
    Notes on junction tree algorithm
    March 31 Learning in Graphical Models[Slides] Murphy Book, Chapter 26
    More detailed notes on Chow Liu ALgorithm
    April 4 EM Algorithm[Slides] Murphy Book, Chapter 11
    Notes on EM
    April 7 Convex Optimization[Slides] Book by Boyd and Vandenberghe, Chapter 3 and 9
    April 11 Gradient Descent[Slides] Book by Boyd and Vandenberghe, Chapter 9
    Notes on stochastic gradient descent
    April 14 Semi Supervised Learning[Slides] Tutorial on Active Learning
    Another survey on active learning
    Tutorial on co-training and semi supervised learning
    April 18 Reinforcement Learning[Slides] Online book by Sutton and Barto. See chapters 1-4, and chapter 8
    April 21 Deep Learning[Slides] Online book by Bengio. See chapter 6
    April 25 Deep Learning[Slides] Online book by Bengio. See chapters 7 and 8