This is an advanced graduate course in the theory of machine learning. The course is ideal for graduate students and senior undergraduates who are theoretically inclined and want to know more about related research challenges in the field of machine learning. A tentative list of topics include the following:
Statistical learning theory, VC theory and PAC model
Online learning, bandit algorithms, connections to game theory
Theory of regression, classification, kernel methods
Graphical Models and high dimensional learning
Unsupervised and semi supervised Learning
Theory of convex and non-convex optimization
Each student is expected to prepare scrbe notes for 2 lectures. In addition, there will be a final project. The final project will involve either thinking about an open problem or reading and summarizing 2-3 related papers in a topic of your interest. The final project can involve programming but needs to have a significant theoretical component. The project will be graded on the basis of a final report.
Each student is expected to produce high quality lecture notes for 2 classes. Failure to do so will automatically result in a C grade. The purpose of each scribe note is to document a given lecture from start to finish including proofs and details. The notes should be such that anyone who might have missed the lecture can easily understand what was covered. Of course, it is not easy to produce such notes on the first try. Typically a student will email me his scribe notes and then I will provide feedback on how to improve it. Once done, the notes will be put up online on the course webpage. The first draft of the notes will be due within a week of the lecture. The latex template for the scribe notes is here.