- Course Number: 01:198:461
- Instructor: Karl Stratos
- Course Type: Undergraduate
- Semester 1: FALL
- Credits: 4
This course is a systematic introduction to machine learning, covering theoretical as well as practical aspects of the use of statistical methods. Topics include linear models for classification and regression, support vector machines, regularization and model selection, and introduction to deep learning. Application examples are taken from areas like computer vision, natural language processing, information retrieval and others.
Machine learning is a vast, fast-changing field. The course is designed to provide a solid starting point by focusing on timeless technical foundations. Students will learn fundamental concepts in the field such as generalization and overfitting, bias-variance tradeoff, Bayes optimality, and error decomposition. Students will also learn how to implement key techniques in the field from scratch including gradient descent, kernel trick, boosting, and expectation maximization. The course content will be delivered by a combination of written assignments that require rigorous proofs and self-contained Python Juypter Notebooks that require completing missing portions and empirically investigating the behavior of models on datasets. The course will reinforce learning with a series of short quizzes throughout the semester.
The course content is designed to be accessible to all SAS students regardless of their majors who have an adequate background in linear algebra, calculus, probability, and programming.
- Instructor Profile: Stratos, Karl
- Prerequisite Information:
Although the course has CS 206 as one of the pre-requisites, which requires M 152 (Calculus II), it does not require students to have an in-depth knowledge of integration. Thus, this prerequisite can be replaced by alternative classes on probability in Mathematics or Statistics.
- Expected Work: To assess if students have a suitable technical background, there will be a short entrance quiz on linear algebra, calculus (differentiation only), and probability. To make this serious but not too impactful on the grade, it will count as 5% of the final grade. To assess that students have acquired basic literacy in all the concepts, tools, and techniques they are taught, they will be given 3 short quizzes periodically through the semester. There will be five assignments each given 2 weeks to complete. The assignments are designed to be challenging but rewarding. Collaboration is allowed and encouraged as long as students (1) write their own solutions entirely on their own, and (2) specify names of student(s) they collaborated with in their writeups. The students will need to digest and reformulate the lecture content in order to successfully complete the assignments. Discussion about assignment problems on Canvas is encouraged. There will be a final exam to assess the student’s ability to put together the concepts and tools they have learned in the course. The quizzes and the written assignments will provide sufficient preparation for
- Exams: Grade Evaluation Entrance quiz 5% Quizzes 15% Assignments 50% Final Exam 30%
The course is focused on basic principles of machine learning. In particular, students must implement everything from scratch (i.e., using Python/NumPy only, no Scikit/PyTorch/TensorFlow). This will require a true understanding of core machine learning techniques, which are mostly used as black box tools in existing CS courses such as 210 (Data Management in Data Science) and 439 (Introduction to Data Science). Thus the course can serve as a natural follow-up for students who have used these techniques in 210 or 439 and want to understand their underlying principles more deeply. Another major distinction from 210 and 439 is that the course has no data preparation or analysis components. The course assumes that data is given and is only concerned with representation, modeling, estimation, and model selection for general prediction problems. It will thus reinforce and complement existing classes and they will eventually lead to a track for a Data Science major.