Course Details

This is an introductory course to deep learning. The course will cover theories, principles, and practices of traditional neural networks and modern deep learning. The topics of the course are structured into four-fold: (i) Fundamentals of Machine Learning, (ii) Neural Networks, (iii) Modern Deep Learning, and (iii) Applications and Advanced Topics.

For the fundamentals of machine learning, we will briefly review basic concepts and principles of machine learning to prepare for the learning of the main topics of the course. In the second part, the course will cover foundational topics of neural networks. This includes topics such as feed-forward neural networks, backpropagation, recurrent neural networks, and LSTMs. Next, we will cover the modern deep learning methodologies. This includes advanced CNN architectures for computer vision, attention mechanisms, transformers, etc. Lastly, we will also touch some advanced topics such as deep generative models, deep reinforcement learning, graph neural networks, etc.. This advanced topics can be changed depending on the instructor. 

An important learning goal of the course is to learn practical implementation skills along with the above concepts, principles, and methodologies. For this, the course will have a series of programming assignments and quizzes. Through the final project, students will have a chance to use all learned knowledge and skills to solve practical problems through team collaboration.

The course content is designed to be accessible to all SAS students regardless of their majors who have an adequate background in linear algebra, calculus, probability, and programming. Although the course has CS 206 as one of the pre-requisites, which requires M 152 (Calculus II), it does not require students to have an in-depth knowledge of integration. Thus, this prerequisite can be replaced by alternative classes on probability in Mathematics or Statistics.  

Prerequisites: M 250 (Introductory Linear Algebra), CS 112 (Data Structures), CS 206 (Introduction to Discrete Structures II) OR M 477 (Mathematical Theory of Probability) OR S 379 (Basic Probability Theory)

The course objectives are (1) to understand the principles of deep learning and its capabilities and (2) to acquire practical skills to design, implement, and train practical deep learning systems. At the end of the course, students will have knowledge of the fundamentals of neural networks and modern deep learning. With this knowledge, the student will be able to use deep learning models or develop new architectures to solve practical real-world problems such as computer vision and natural language processing. In particular, students will become familiar with one of the most popular deep learning programing frameworks based on Python. The exposure to some research topics in the latter part of the course will also encourage students’ interest in research of this topic. Thus, students will be prepared to have a relevant experience for their next career either in industry or academia.