This is an advanced graduate course in the theory of machine learning. The course is ideal for graduate students and senior undergraduates who are theoretically inclined and want to know more about related research challenges in the field of machine learning. A tentative list of topics include the following:
Statistical learning theory, VC theory and PAC model
Online learning, bandit algorithms, connections to game theory
SVMs and kernel methods
Theory of deep learning
Each student is expected to prepare scribe notes for 2 lectures. In addition, there will two homeworks and a final project. The final project will involve either thinking about an open problem, reading and summarizing 2-3 related papers in a topic of your interest. The final project can involve programming but needs to have a significant theoretical component.
Each student is expected to produce high quality lecture notes for 2 classes. Failure to do so will automatically result in a C grade. The purpose of each scribe note is to document a given lecture from start to finish including proofs and details. The notes should be such that anyone who might have missed the lecture can easily understand what was covered. Of course, it is not easy to produce such notes on the first try. Typically a student will email me his scribe notes and then I will provide feedback on how to improve it. Once done, the notes will be put up online on the course webpage. The first draft of the notes will be due within a week of the lecture. The latex template for the scribe notes is here.