198:516 - Programming Languages and Compilers II
Announcements
- Optional lecutre moved to Tuesday, May 12
- Project 2 deadline extension: Tuesday, May 5
- Sample solution for homework 3 has been posted on sakai/Resources
UNTIL FURTHER NOTICE, ALL LECTURES AND CLASS RELATED
ACTIVITIES WILL BE DONE ONLINE.
THE UNIVERSITY IS CLOSED .
We will be using webex for our remote interactions.
Stay safe!
Description
In the last ten years, the computing landscape has changed
dramatically with the realization that Moore's law can only be
sustained
via parallelism because of CMOS technology's power dissipation properties. However, this approach also has its inherent limits. New paradigms are now under consideration such as approximate, reversible, quantum, or analog computing, with hundreds of millions of dollars in research funding dedicated specifically to quantum computing. In addition, new optimization goals have taken a more dominant role besides execution time, such as energy consumption, resource usage vs. QoS tradeoffs, reliability and robustness, or security and privacy.
The goal of this course is to expose students to existing (e.g. parallel) and emerging programming paradigms (e.g. reversible, quantum, and approximate) to get a better understanding of their foundations, applicability, and limitations. We will discuss the different required tool chains, consisting of languages, compilers, optimizers, debuggers, and runtime systems. To solve hard optimization problems, students will learn to use different forms of integer programming formulations (gurobi) to compute optimal solutions, either as part of an evaluation infrastructure, or as part of the compiler/optimizer. After taking this class, students will be able to assess the effectiveness of the new paradigms for different application domains, and the efforts needed to build a tool chain for each paradigm and relevant optimization objectives.
The class will consist of lectures, small projects (including writing a quantum program and executing it on an actual quantum computer), and student presentations on selected topics. Grades will be based on class participation, presentations, and a final project. There will be no midterm or final exam.
Prerequisites
Some background in compilers and programming languages.
Staff
- Ulrich (Uli) Kremer
(uli@cs.rutgers.edu)
Office: CoRE 318
Office hours: Friday 2:00-3:00pm or by appointment
- Grader Akshat Kumar (ak1648@scarletmail.rutgers.edu)
Office: TBA
Office hours: TBA
Sakai
Please post questions regarding the projects and homeworks in
our sakai group.
Do not send these questions to me directly. Thanks.
Tentative Syllabus
- Introduction
- Energy optimizations: OpenMP, CUDA, project to measure energy
consumption (multi-core ARM boards with GPUs)
- Approximation / configuration management: KDG (knob dependence graph),
gurobi, project to solve large optimization problem using gurobi
- Reversible computing: Fundamentals (Landauer's Principle), adiabatic
circuits, reversible programming languages
- Quantum computing: Fundamentals, QC supremacy, current programming
tools for QC, project using IBM's quantum computer
- Different optimization objectives: Reliability and robustness,
security and privacy
- Other paradigms: analog computing, DNA computing, neural networks
Lectures
Tuesday/Friday noon-1:20 p.m., Hill 482 (Busch Campus)
- January 21, 2020 ---
Overview, syllabus, and introduction to power and energy management
(introduction)
Power/energy/thermal limit for Moore's law; first and second law of
thermodynamic; reversible and quantum computing; approximate computing
- January 24, 2020 ---
Compiler optimizations
(Overview
of Compiler Optimizations 1)
General structure of optimizing compilers; performance
optimizations
- January 28, 2020 ---
Compiler optimizations (cont.)
(Overview
of Compiler Optimizations 2)
General structure of optimizing compilers; intermediate
representations; levels of abstraction; optimization interactions
- January 31, 2020 ---
Compiler optimizations for performance, power, energy, and thermal
(Overview
of Power/Energy/Thermal Optimizations 1)
Performance, power, energy, and
thermal as distinct optimization goals; introduction to
power/energy optimizations
Readings :
- February 4, 2020 ---
Dynamic voltage and frequency scaling
(DFVS-1)
Compiler-time optimizations for CPU DVFS with soft deadlines
(part 1)
Readings :
-
Compiler-Directed Dynamic Frequency and Voltage Scheduling
, C-H Hsu, U. Kremer, and M. Hsiao.
International Workshop on Power-Aware Computing Systems
(PACS'00), November 2000
-
Dynamic Voltage and Frequency Scaling for Scientific
Applications , C-H Hsu and U. Kremer.
DCS-TR-447, Rutgers University, August 2001
-
The Design, Implementation, and Evaluation of a Compiler
Algorithm for CPU Energy Reduction
, C-H Hsu and U. Kremer.
Conference on Programming Languages Design and Implementation (PLDI'03), June 2003
- February 7, 2020 ---
Dynamic voltage and frequency scaling
(DFVS-2)
Compiler-time optimizations for CPU DVFS with soft deadlines
(part 2)
Readings :
-
Compiler-Directed Dynamic Frequency and Voltage Scheduling
, C-H Hsu, U. Kremer, and M. Hsiao.
International Workshop on Power-Aware Computing Systems
-
Dynamic Voltage and Frequency Scaling for Scientific
Applications , C-H Hsu and U. Kremer.
DCS-TR-447, Rutgers University, August 2001
-
The Design, Implementation, and Evaluation of a Compiler
Algorithm for CPU Energy Reduction
, C-H Hsu and U. Kremer.
Conference on Programming Languages Design and Implementation (PLDI'03), June 2003
- February 11, 2020 ---
Resource hibernation and computation offloading
(Hibernation-Offloading)
Readings :
- February 14, 2020 ---
Basic concepts of approximation; semi-lattice and lattice; data flow framework example
(Approximation-1).
- February 18, 2020 ---
Reaching Definitions (RD) data flow framework; worklist algorithm; Model, data, and computation approximation
(Approximation-2).
- February 21, 2020 ---
Approximation wrap-up; introduction to concurrent execution,
dependence, vectorization vs. parallelization
(Parallel-1).
- February 25, 2020 --- no class,
attend faculty talk of Yipeng Huang at 10:30am in CoRE 301
instead.
Title: Emerging Architectures for Humanity's Grand Challenges
- February 28, 2020 ---
Vectorization vs. parallelization, iteration space, loop-carried
vs. loop-independent dependencies, valid transformations
(Parallel-2).
Readings :
-
Practical Dependence Testing ,
G. Goff, K. Kennedy, C-W Tseng
Conference on Programming Languages Design and Implementation
(PLDI 1991), June 1991
- March 3, 2020 ---
OpenMP and CUDA
(OpenMP-CUDA).
First project, OpenMP parallel loops with scheduling directives;
introduction to CUDA
- March 6, 2020 ---
CUDA parallel programming abstractions
(CUDA-2).
- March 10, 2020 ---
Distance and direction vectors, loop carried / loop independent dependences, valid
loop interchange
(Parallel-3).
Readings :
-
Practical Dependence Testing ,
G. Goff, K. Kennedy, C-W Tseng
Conference on Programming Languages Design and Implementation
(PLDI 1991), June 1991
- March 13, 2020 --- cancelled due to
coronavirus related closure of Rutgers
- March 14 to March 22 --- Spring break, no classes
- March 24, 2020 ---
Look-carried/loop-independent dependencies, dependence testing
(Parallel-4).
Readings :
-
Practical Dependence Testing ,
G. Goff, K. Kennedy, C-W Tseng
Conference on Programming Languages Design and Implementation
(PLDI 1991), June 1991
- March 27, 2020 ---
Dependence testing, SIV and ZIV,
simple vectorization algorithm
(Parallel-5).
Readings :
-
Practical Dependence Testing ,
G. Goff, K. Kennedy, C-W Tseng
Conference on Programming Languages Design and Implementation
(PLDI 1991), June 1991
- March 31, 2020 ---
Advanced vectorization algorithm; hard compiler and runtime
system problems (NP-completeness proof)
(Parallel-6
and introduction to hard problems).
Readings :
- April 3, 2020 ---
Hard compiler and runtime system problems, linear integer
programming, gurobi, solving compiler/runtime problems using gurobi
(Hard Problems and Gurobi).
Readings :
- April 7, 2020 ---
Functional and cloning redundancy, RSDG, configuration space,
optimal configuration selection
(Redundancy-1).
- April 10, 2020 ---
RAPIDS prototype system overview, dynamic reconfiguration,
project#2
(Redundancy-2).
- April 14, 2020 ---
Modified project #2; Introduction to reversible computing, second law of
thermodynamics, Landauer's limit, adiabatic circuits
(Reversible-1).
Readings :
- April 17, 2020 ---
Research paper presentations
(Presentations-1).
- Polyhedral analysis and optimizations (Shaleen); slides
available on sakai/Resources
- Intermittent energy optimizations (Philip); slides
available on sakai/Resources
- April 21, 2020 ---
Reversible programming languages, Janus language; mathematical
foundations of quantum computing
(Reversible-2 Quantum-1).
Readings :
- April 24, 2020 ---
Quantum computing, superposition, quantum gates, entanglement
(Quantum-2).
- April 28, 2020 ---
Research paper presentations
- JouleGuard (Fei); slides
available on sakai/Resources
- Uncertain< T > (Hari); slides
available on sakai/Resources
- Sarana (Xiao); slides
available on sakai/Resources
- May 1, 2020 ---
Research paper presentations
(Presentations-3).
- DNA computing (Thomas); slides
available on sakai/Resources
- Neuromorphic computing (Jonathan); slides
available on sakai/Resources
- Quantum computing status (Jiapeng); slides
available on sakai/Resources
- May 12, 2020 (optional) ---
Quantum computing
(Quantum-3).
Study Homeworks
- Homework
1
New submission deadline: Friday, February 21, 2020 at 11:59pm.
Please submit a single pdf file through sakai.
- Homework
2
New submission deadline: Tuesday, March 3, 2020 at 11:59pm.
Please submit a single pdf file through sakai.
- Homework
3
Submission deadline: Thursday, April 9, 2020 at 11:59pm.
Please submit a single pdf file through sakai.
Projects
- Power/energy/performance tradeoffs: CPU-OpenMP-CUDA
NEW Deadline: Monday, April 6, 2020, at 11:59pm ; submission through sakai.
- How to use Gurobi to solve hard compiler problems: Solving Hard Problems
To download the provided project code, select
proj2.tar
New deadline: Tuesday, May 5, 2020, at 11:59pm ; submission through sakai.
How to Survive Graduate School, Keep the Big Picture in Mind, and
More