Events Feed

 Start Date: 12 Oct 2021; Start Time: 11:00AM - Title: Understanding Human Memory as a Constrained Optimization Problem Bio: Dr. Zhang is an Assistant Professor in the Department of Psychology and is also affiliated with the Department of Computer Science and the Rutgers Center for Cognitive Science. Dr. Zhang received her Ph.D. in 2019 from Carnegie Mellon University jointly in the Machine Learning Department under the School of Computer Science, and the Center for the Neural Basis of Cognition. She completed additional postdoctoral training in the Princeton Neuroscience Institute before joining Rutgers. She was a visiting researcher in the Department of Artificial Intelligence at the University of Groningen in 2016 and interned as a research scientist at Facebook Reality Labs in 2018.   Speaker: Abstract: How can methods from machine learning be brought to bear on cognitive psychology’s classic questions about mental representations and cognitive processes? The literature on human memory has primarily focused on identifying a set of behavioral patterns that are consistently shown across experiments, but with relatively little concern for why humans demonstrate these patterns in the first place. In contrast, my research explains human memory behavior by understanding how close it is to optimal behavior, by adapting the theory of rational analysis from psychology literature. This creates an opportunity to draw inspiration from the best examples of intelligent systems in computer science.Humans live a resource constrained existence. We can only process a fraction of our experiences at one time, and we store a shockingly small subset of these experiences in the memory for later use. Properly identifying optimal behavior requires taking into account the limitations and constraints of human cognition. In this talk, I will discuss a few directions I have taken to understand what the solutions are to different constrained-optimization memory problems, their evidence over human empirical data, and how we can leverage the optimal solutions as guidance in developing technology to improve human memory. Location: Via Zoom Committee: Start Date: 18 Oct 2021; Start Time: 12:30PM - 02:00PM Title: Improving the Efficiency of Kinodynamic Planning with Machine Learning Bio: Speaker: Abstract: Kinodynamic motion planning is characterized by the lack of a steering function (i.e., a local planner) for the underlying robotic system. This talk will first survey efforts to improve the efficiency of state-of-the-art, sampling-based kinodynamic planners through data-driven methods. It will then present a proposed direction for improving the path quality and computational efficiency of such planners when applied to vehicular navigation. Given a black-box dynamics model for the vehicle, a reinforcement learning process is trained offline to return a low-cost control that reaches a local goal state (i.e., a waypoint) in the absence of obstacles. By focusing on the system's dynamics and not knowing the environment, this process is data-efficient and takes place once for a robotic system. In this way, it can be reused in different environments. Then, the proposed sampling-based planner generates online local goal states for the learned controller in an informed manner to bias towards finding a high-quality solution trajectory fast. The planner also maintains an exploratory behavior for cases where the guidance from the machine learning process is not effective. The results show that the proposed integration of learning and planning can produce higher quality paths than standard, sampling-based kinodynamic planning with random controls in fewer iterations and computation time. Location: 1 Spring Street, Room SPR-319 & via Zoom Committee: Prof. Kostas Bekris (Chair) Prof. Abdeslam Boularias Prof. Jingjin Yu Prof. Amélie Marian Start Date: 19 Oct 2021; Start Time: 11:00AM - 12:30PM Title: Fairness and Bias in Algorithmic Decision-Making Bio: Jon Kleinberg is the Tisch University Professor in the Departments of Computer Science and Information Science at Cornell University. His research focuses on the interaction of algorithms and networks, the roles they play in large-scale social and information systems, and their broader societal implications. He is a member of the National Academy of Sciences and National Academy of Engineering, and the recipient of MacArthur, Packard, Simons, Sloan, and Vannevar Bush research fellowships, as well awards including the Harvey Prize, the Nevanlinna Prize, and the ACM Prize in Computing. Speaker: Abstract: As algorithms trained via machine learning are increasingly used as a component of screening decisions in areas such as hiring, lending, and education, discussion in the public sphere has turned to the question of what it means for algorithmic classification to be fair to different groups. We consider several of the key fairness conditions that lie at the heart of these debates, and discuss recent research on trade-offs and interventions through the lens of these conditions. We also explore how the complexity of a classification rule interacts with its fairness properties, showing how natural ways of approximating a classifier via a simpler rule can lead to unintended biases in the outcome.The talk will be based on joint work with Jens Ludwig, Sendhil Mullainathan, Manish Raghavan, and Cass Sunstein. Location: Via Zoom Committee: Start Date: 19 Oct 2021; Start Time: 02:00PM - 03:30PM Title: Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images Bio: Speaker: Abstract: Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. Here, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. Location: Remote via Zoom Committee: Prof. Vladimir Pavlovic (Chair) Prof. Dimitris Metaxas Prof. Sungjin Ahn Prof. David Pennock Start Date: 21 Oct 2021; Start Time: 11:00AM - Title: How to identify anomalies accurately and privately Bio: Jaideep Vaidya is a Professor in the MSIS Department at Rutgers University. He received the B.E. degree in Computer Engineering from the University of Mumbai, the M.S. and Ph.D. degree in Computer Science from Purdue University. His general area of research is in security, privacy, data mining, and data management. He has published over 190 technical papers in peer-reviewed journals and conference proceedings, and has received several best paper awards from the premier conferences in data mining, databases, digital government, security, and informatics. He is an ACM Distinguished Scientist, and IEEE Fellow, and is the Editor in Chief of the IEEE Transactions on Dependable and Secure Computing. Speaker: Abstract: In the current digital age, data is continually being collected by organizations and governments alike. While the goal is to use this data to derive insight and improve services, the ubiquitous collection and analysis of data creates a threat to privacy. In this talk, we examine the problem of private anomaly identification. Anomaly detection is one of the most fundamental data analysis tasks, and is useful in applications as far ranging as homeland security, to medical informatics, to financial fraud. However, many applications of outlier detection such as detecting suspicious behavior for counter-terrorism or anti-fraud purposes also raise privacy concerns. We conclusively demonstrate that differential privacy (the de facto model for privacy used today) is inherently incapable of solving this problem. We then present a new notion of privacy, called Sensitive Privacy, that protects the vast majority of records that are or could be normal, while still enabling accurate identification of records that are anomalous. Given the widespread impact of COVID-19, we also present some results from a recent NSF funded effort to perform privacy-preserving crowdsensing of COVID-19, from the context of hotspot detection. Location: Via Zoom Committee: Start Date: 26 Oct 2021; Start Time: 11:00AM - Title: Tensor-structured dictionary learning: theory and algorithms Bio: Anand D. Sarwate is an Associate Professor in the Department of Electrical and Computer Engineering at Rutgers, the State University of New Jersey. He received a B.S. degree in Electrical Science and Engineering and a B.S. degree in Mathematics from MIT in 2002, an M.S. in Electrical Engineering from UC Berkeley in 2005 and a PhD in Electrical Engineering from UC Berkeley in 2008. Prof. Sarwate received the NSF CAREER award in 2015, and the Rutgers Board of Governors Research Fellowship for Scholarly Excellence in 2020. His interests are in information theory, machine learning, and signal processing, with applications to distributed systems, privacy and security, and biomedical research. Speaker: Abstract: Existing and emerging sensing technologies produce multidimensional, or tensor, data. The traditional approach to handling such data is to “flatten” or vectorize, the data. It is well-known that this ignores the multidimensional structure and that this structure can be exploited to improve performance. In this talk we study the problem of dictionary learning for tensor data, where we want to learn an efficient representations with low rank. A naïve approach to this problem would fail due to the large number of parameters to estimate in the dictionary. However, by looking at dictionaries that admit a low rank factorization the problem becomes tractable. We characterize how hard the statistical problem of estimating these dictionaries is and provide novel algorithms for learning them. Joint work with Z. Shakeri (EA), M. Ghassemi (JP Morgan Research), and W.U. Bajwa (Rutgers) Location: Via Zoom Committee: Start Date: 03 Nov 2021; Start Time: 11:00AM - 01:00PM Title: Towards Efficient and Reliable Skeleton-Based Human Pose Modeling Bio: Speaker: Abstract: Understanding human behaviors by deep neural networks has been a central task in computer vision due to its wide application in our daily life. Existing studies have explored various modalities for learning powerful feature representations of human poses, such as RGB frames, optical flows, depth images, and human skeletons. Among them, skeleton-based pose representation has received increasing attention in recent years thanks to its action-focusing nature, compactness, and domain-invariant property. However, prevalent skeleton-based algorithms are typically inefficient in network parameters or training data, but also unreliable in human action forecasting problems. In this dissertation, we explore the benefits and challenges of skeleton-based human action modeling and offer novel solutions to achieve efficient and reliable model performance in human action estimation, recognition, and generation tasks.In the first part of this dissertation, we tackle the problem of model as well as data efficiency in human pose understanding. Given the meaningful topological structure carried by human skeletons, we show that capturing the relationships between joints in the skeleton of a human body by graph neural networks leads to an efficient network architecture that outperforms state of the art while using 90% fewer parameters. Then we present a novel representation learning method to disentangle pose-dependent as well as view-dependent factors from human poses based on mutual information maximization. Empirically, we show that the resulting pose representations can be used for different action recognition scenarios where training data are limited.As the second part of this dissertation, we explore the structure-driven paradigms to make long-term predictions of future human actions by explicitly using skeletons as structural conditions. Such hierarchical strategies are built upon multi-stage generative adversarialnetworks, and typically lead to more robust and reliable predictions than previous appearance-driven ones. To avoid inherent compounding errors in recursive pixel-level prediction, we first estimate high-level structure in the input frames and then predict how that structure evolves in the future. Through developing specialized network architectures, we are able to capture the high-level structure of actions efficiently while preserve temporal coherence, thereby benefiting long-term future forecasting. Location: Via Zoom Committee: Prof. Dimitris N. Metaxas (Advisor) Prof. Mubbasir Kapadia Prof. Hao Wang Prof. Xiaolei Huang (External member, Penn State University) Start Date: 16 Nov 2021; Start Time: 11:00AM - 12:15PM Title: Learning and Using Knowledge for Text Bio: Karl Stratos is an assistant professor in the Computer Science Department at Rutgers University. His research centers on unsupervised representation learning and knowledge-intensive language processing. He completed a PhD in computer science from Columbia University in 2016. During PhD, he was advised by Michael Collins and also worked closely with Daniel Hsu. After PhD, he was a senior research scientist at Bloomberg LP (2016-2017) and a research assistant professor at Toyota Technological Institute at Chicago (2017-2019). Speaker: Abstract: Two key problems in automatic text understanding are (1) how to learn high-level representations that capture useful knowledge from noisy unlabeled data, and conversely (2) how to use existing knowledge resources to robustly handle unknown facts. In this talk, I will present our recent works along the two thrusts. The first is AMMI, a general framework for learning discrete structured latent variables from noisy signals by adversarially maximizing mutual information (ICML 2020). The second is a theoretical and empirical investigation of the use of "hard" negative examples in noise contrastive estimation (NAACL 2021). The third is EntQA, a new paradigm for entity linking that reduces the task as inverse open-domain question answering and fundamentally solves the dilemma of having to predict mentions without knowing their entities (under review). We achieve new state-of-the-art results in document hashing, zero-shot entity retrieval, and entity linking on numerous datasets. Location: Via Zoom Committee: Start Date: 23 Nov 2021; Start Time: 11:00AM - Title: Rethinking Modern Storage and Memory Management Bio: Sudarsun Kannan is an assistant professor at Rutgers University, where he leads the Rutgers Systems Lab. His research group works at the intersection of hardware and software, building operating systems and system software for next-generation memory and storage technologies. Results from his work have appeared at premier operating systems and architecture venues. Sudarsun's work has also resulted in patents related to nonvolatile memory and resource management. Before joining Rutgers, he was a postdoctoral research associate at Wisconsin-Madison and graduated with an M.S. and Ph.D. from Georgia Tech. Speaker: Abstract: The last decade has seen a rapid hardware innovation to develop ultra-fast and heterogeneous storage and memory technologies critical for accelerating data-intensive and intelligent applications. Unfortunately, today's monolithic operating systems (OS) that are the backbone for managing storage and memory heterogeneity continue to be an Achilles heel and fail to exploit hardware innovations. In the first part of the talk, I will discuss our approach to designing scalable storage solutions through a cross-layered disaggregation of software layers across the host and the storage hardware (OSDI '20). I will also briefly outline techniques to improve storage reliability and accelerate data recovery after failures (FAST '21). In the second part of the talk, I will focus on the need for new OS principles and abstractions for managing memory heterogeneity and their resulting impact on applications (ASPLOS '21). Finally, I will conclude the talk by outlining unsolved challenges and prospective future directions. Location: Via Zoom Committee: Start Date: 30 Nov 2021; Start Time: 11:00AM - 12:30PM Title: Processing Massive Datasets via Sublinear Algorithms: New Challenges and Opportunities Bio: Sepehr Assadi is an Assistant Professor of Computer Science at Rutgers University. His primary research interests are in theoretical foundations of processing massive datasets and in particular streaming and sublinear algorithms and lower bounds for massive graph problems. He received a Ph.D. in Computer Science from University of Pennsylvania in 2018 and spent a year as a postdoctoral researcher at Princeton University, before joining Rutgers. Sepehr is a recipient of NSF CAREER award, Google Research Scholar award, EATCS Distinguished Dissertation Award, ACM-EATCS Principles of Distributed Computing Dissertation Award, Rubinoff Dissertation Award, and several best paper awards at theoretical computer science conferences including SODA, SPAA, and DISC. Speaker: Abstract: With the emergence of massive datasets across various application domains, there is a rapidly growing need in designing algorithms that can process these massive inputs efficiently. The challenges of data processing at this scale are inherently different from those of traditional algorithm design. For instance, the sheer size of these datasets no longer allows one to assume fast random access to the input, or even store the entire input in one place to begin with. In this talk, I will give an overview of my research on developing algorithms that follow the rigorous standards of traditional algorithm design, while explicitly accounting for the practical challenges of processing massive inputs. As an illustrative example, I will focus on my research on algorithms for graph coloring problems over massive graphs. Graph coloring is one of the most fundamental problems in graph theory with a wide range of applications in computer science. I will describe my earlier work in obtaining the first set of efficient algorithms for coloring massive graphs across multiple computational models in a unified way. I will then talk about my recent work on exploring the inherent limitations of graph coloring on massive graphs as well as surprising connections established in this line of work to problems beyond graph coloring such as correlation clustering. Location: Via Zoom Committee: Start Date: 07 Dec 2021; Start Time: 11:00AM - 12:30PM Title: Leveraging kernel extensions for safe and efficient networking Bio: Srinivas Narayana is an Assistant Professor in the Department of Computer Science at Rutgers University. His research goal is to enable developers to implement novel and flexible packet-processing applications, with guarantees of safety and high performance. To achieve this goal, he applies compiler and formal methods technology in domain-specific ways to network software and hardware. Srinivas received his M.A/Ph.D. in Computer Science from Princeton University in 2016 and a B.Tech from Indian Institute of Technology Madras in 2010. Srinivas completed a post-doc at Massachusetts Institute of Technology in 2018. Srinivas's research has been recognized with the best paper award at the 2017 ACM SIGCOMM conference, a Facebook research award, and grants from the National Science Foundation and the Network Programming Institute. Speaker: Abstract: Extended Berkeley Packet Filter (eBPF) is a mechanism that emerged recently in Linux to extend the functionality of the operating system kernel. eBPF allows users to download their code into the kernel and execute it at specific points of attachment within the kernel---for example, the network device driver. To mitigate security risks from running untrusted user code, the kernel implements software fault isolation through static program analysis, encapsulated into an in-kernel component called the verifier. Due to its trifecta of flexibility, safety, and performance in a familiar Linux environment, eBPF code is already widely deployed on production systems. However, programming networking applications in eBPF presents several new challenges. Optimizing compilers need to incorporate safety into their optimizations. Developers must wrestle with the arcane rules of the verifier to prove the safety of their code. The verifier's static analysis algorithms contained critical bugs, resulting in disastrous security consequences. This talk will cover my recent work (SIGCOMM'21, CGO'22) that addresses some of these challenges by combining networking with formal methods.Joint work with Qiongwen Xu, Harishankar Vishwanathan, Michael Wong, Tanvi Wagle, Anirudh Sivaraman, Matan Shachnai, and Santosh Nagarakatte. Location: Via Zoom Committee: Start Date: 08 Dec 2021; Start Time: 09:00AM - Title: An integrated platform for joint simulation of occupant-building interactions Bio: Speaker: Abstract: Several approaches exist for simulating building properties (e.g. temperature, noise) and human occupancy (e.g. movement, actions) in an isolated fashion, providing limited ability to represent how environmental features affect human behavior and vice versa. To systematically model building-occupant interactions, several requirements must be met, including the modelling of (a) interdependent multi-domain phenomena ranging from temperature and sound changes to human movement, (b) high-level occupant planning and low-level steering behaviors, (c) environmental and occupancy phenomena that unfold at different time scales, and (d) multiple strategies to represent occupancy using established models. In this work, we propose an integrated platform that satisfies the aforementioned requirements thus enabling the joint simulation of building-occupant interactions. To this end, we combine the benefits of a model-independent, discrete-event, general-purpose framework with an established crowd simulator. Our platform provides insights on a building’s performance while accounting for alternative design features and modelling strategies. Location: Via Zoom Committee: Prof. Mubbasir Kapadia Prof. Vladimir Pavlovic Prof. Mridul Aanjaneya Prof. Eric Allender Start Date: 08 Dec 2021; Start Time: 09:00AM - 10:00AM Title: Unsupervised Learning of Structured Representation of the World Bio: Speaker: Abstract: Intelligent agents would like to build representations of the world that are well-suited for planning actions and achieving goals in the physical world. Because the environments are diverse and always changing, providing supervision to train the representation learning systems makes them hard to scale and unviable. In this work, we explore unsupervised representation learning methods, based on variational auto-encoders, that do not simply return a monolithic compression of the input observation but rather provide a significantly richer representation in the following respects: (i) our encoder, taking only a few observations of a 3D scene as input, produces a representation that contains information about the full 3D scene. (ii) It organizes the representation as a set of object vectors and decomposes these vectors further into what' and where', thus opening the possibility for symbolic processing downstream. (iii) As the agent observes a dynamic environment, our representation method, using past observations and the learned knowledge of the dynamics, infers the state of invisible objects and unseen viewpoints of the scene. (iv) Lastly, our representation provides, not just a point estimate of the inferred environment state, but rather a belief state and thus embraces the randomness of the physical world. In achieving these characteristics, we develop novel models that advance the state of the art. To do this, we build on existing approaches for neural modeling of stochastic processes, recurrent state-space modeling, sequential Monte Carlo, and unsupervised object-centric representation learning. Location: Via Zoom Committee: Hao Wang Karl Stratos Mario Szegedy (assigned by the department) Sungjin Ahn (Advisor) Start Date: 08 Dec 2021; Start Time: 10:00AM - 12:00PM Title: Scenario Generalization and its Estimation in Data-driven Decentralized Crowd Modeling Bio: Speaker: Abstract: In the context of crowd modeling, we propose the notion of scenario generalization, which is amacroscopic view of the performance of a decentralized crowd model. Based on this notion, firstly, weaim to answer the question that how a training paradigm and a training domain (source) affect thescenario generalization of an imitation learning model when applied to a different test domain (target).We evaluate the exact scenario generalizations of models built on combinations of imitation learningparadigms and source domains. Our empirical results suggest that (i) Behavior Cloning (BC) is better thanGenerative Adversarial Imitation Learning (GAIL), (ii) training samples in source domain with diverseagent-agent and agent-obstacle interactions are beneficial for reducing collisions when generalized tonew scenarios.Secondly, we note that although the exact evaluation of scenario generalization is accurate, it requirestraining and evaluation on large datasets, coupled with complex model selection and parameter tuning.To circumvent this challenge by estimating the scenario generalization without training, we proposed aninformation-theoretic inspired approach to characterize both the source and the target domains. Itestimates the Interaction Score (IS) that captures the task-level inter-agent interaction difficulty of targetscenario domain. When augmented with Diversity Quantification (DQ) on the source, the combined ISDQscore offers a means to estimating the source to target generalization of potential models. Variousexperiments verify the efficacy of ISDQ in estimating the scenario generalization, compared with the exactscenario generalizations of models trained with imitation leaning paradigms (BC, GAIL) and reinforcementlearning paradigm (proximal policy optimization, PPO). Thus, it would enable rapid selection of the bestsource-target domain pair among multiple possible choices prior to training and testing of the actualcrowd model. Location: Via Zoom Committee: Prof. Petros Faloutsos (York University) Prof. Yongfeng Zhang Prof. Mubbasir Kapadia Prof. Vladimir Pavlovic (chair) Start Date: 08 Dec 2021; Start Time: 04:00PM - 06:00PM Title: Learning of Networks Dynamics: Mobility, Diffusion, and Evolution Bio: Speaker: Abstract: Within network science and network theory, traditional social network analysis is mainly based on static networks. Network dynamics analysis takes interactions of social features and temporal information into account, appearing to be an emergent scientific field. Different from the social networks, they consider larger, dynamic, multi-mode, multi-plex networks, and may contain varying levels of uncertainty. Due to the heterogeneity of networks, agent-based modeling and other forms of simulations are often used to explore how networks evolve and adapt as well as the impact of interventions on those networks. In this thesis, we discuss two aspects of this topic. In the first aspect, we consider the mobility properties in the social networks, i.e., human trajectories. A human trajectory is a sequence of spatial-temporal data from an individual. The data mining of human trajectories can help us improve a lot of real-world applications. However, with the increasing size of the human trajectory dataset, it brings higher challenges to our analysis. At the same time, for the human trajectory, we still lack of a comprehensive understanding of the relationship among the human trajectories, such as commonality and individuality. Thus, we focused on the statistic tools to measure the similarity between the human trajectories. Different from the traditional geometric similarity measures, such as Hausdorff distance and Frechet distance, we proposed three novel partial similarity measures, which are more suitable to the human trajectories. They can also reduce the storage requirement and save computation time. Based on these similarity measures, we designed a more advanced unsupervised clustering algorithm, integrating with the conformal prediction framework. It performs better in varied trajectory datasets compared with the classical clustering algorithms. In the second aspect, we investigate two problems in the social interactions of the social networks. For the first problem, the previous works mainly focused on the information spreading speed in the static social networks, which can be regarded as the diffusion phenomenon. However, taking mobility into the consideration, the social networks become dynamic and the interactions between different individuals happen over time. The heterogeneity of mobility plays an important role in the diffusion process. We utilize the human trajectory dataset to simulate the physical interactions among individuals to view this diffusion process. At the same time, we also investigate some targeted interventions' impact on the social networks. In the second problem, the opinion evolution process in the social networks is discussed. We proposed a co-evolution model, including the opinion dynamics and social tie dynamics, to investigate the community structure and structural balance in the final state with rigorous theoretical analysis. The simulations also are utilized to validate the communities form process. Location: Via Zoom Committee: Prof. Jie Gao (advisor, chair) Prof. Hao Wang Prof. Peng Zhang Prof. Feng Luo Prof. Joseph S.B. Mitchell (external member, Stony Brook University) Start Date: 10 Dec 2021; Start Time: 01:00PM - Title: Toward Structured Plan Exploration for Multi-Object Rearrangement Bio: Speaker: Abstract: With practical applications in both industrial automation and home automation, object rearrangement is a topic of interest in the broader area of task and motion planning. In this problem, a robot arm moves objects one at a time from current positions to specified goal positions. To avoid collisions, some objects need to be moved before others or even be temporarily displaced to open space. Therefore, to compute feasible rearrangement plans, one needs to not only determine the ordering of object manipulations but also allocate clearings for temporary object displacements. In this talk, we investigate this problem with different objectives. We first present structural properties of the problem, which include upper and lower bounds of objectives. Second, we introduce fast algorithms for object rearrangement in different setups. Extensive experiments show that our algorithms outperform state-of-the-art methods in both computation time and solution quality under various practical scenarios. Location: Via Zoom Committee: Jingjin Yu Kostas Bekris Abdeslam Boularias Aaron Bernstein Start Date: 14 Dec 2021; Start Time: 03:00PM - Title: Scalable machine learning algorithms for multi-armed bandits and correlation clustering Bio: Speaker: Abstract: Massive datasets appear frequently in modern machine learning applications. The situation poses a pressing challenge to design provably efficient large-scale learning algorithms. One of the most promising directions to tackle this challenge is to explore sublinear algorithms for machine learning, i.e. the algorithms that can work with resources substantially smaller than the size of the input they operate on. In this talk, I will discuss sublinear machine learning algorithms for two well-known problems, namely multi-armed bandits (MABs) and correlation clustering. For the multi-armed bandits problem, I will discuss the space efficiency for best-arm identification under the streaming model. The optimal sample complexity to identify the best arm has been known for almost two decades. However, not much was known about the space complexity of the algorithms. To study the space complexity, we introduced the streaming multi-armed bandits model. We proved that, perhaps surprisingly, there exists an algorithm that achieves the asymptotically optimal sample complexity and only uses a space of a single arm. More recently, we extended the results by showing several lower bounds that characterize the necessary assumptions for instance-sensitive sample complexity in streaming MABs. For the correlation clustering problem, I will discuss recent results on sublinear time and space algorithms. This, in particular, includes algorithms that can solve the problem quadratically faster than even reading the input -- assuming standard query access to the data -- or with quadratically smaller space than needed to store the input using a single-pass stream over the data. Location: Via Zoom Committee: Prof. Sepehr Assadi Prof. Jie Gao Prof. Peng Zhang Prof. Mubbasir Kapadia Start Date: 16 Dec 2021; Start Time: 09:00AM - Title: Predicting Long-Term Crowd Flow in Built Environments Bio: Speaker: Abstract: Predicting the concerted movements of crowds in built environments, whether at the scale of a house or a campus with several buildings, is a key requirement for crowd and disaster management, architectural design, and urban planning. It is particularly important to study this movement with large crowds and environments over long time frames to understand the influence of the environment on the crowd, e.g. in terms of congestion. However, this comes at a prohibitive logistical cost when studying real humans and a prohibitive computational cost when studying simulations, neither of which meet the demands of practitioners. We therefore propose the first framework for instantaneously forecasting the full movement of a human/agent crowd using solely the initial state of the crowd and environment. Since the spatial dimensions of environments can vary widely, we have developed fixed-size crowd scenario representations to effectively direct initial state information into convolutional neural network architectures. Experimental results indicate that after training models exclusively on synthetic data, the models generalize to never-before-seen real environments and crowds. Location: Via Zoom Committee: Prof. Mubbasir Kapadia (advisor) Prof. Vladimir Pavlovic Prof. Dimitris Metaxas Prof. David Pennock Start Date: 20 Dec 2021; Start Time: 12:00PM - 02:00PM Title: A GPU Binary Analysis Framework for Memory Performance and Safety Bio: Speaker: Abstract: General-Purpose Graphics Processing Units (GPUs) have attained popularity for their massive concurrency. But with their relative infancy, as well as a prevalence of closed-source and proprietary technology, GPU software has not undergone the degree of optimization that CPU software has. In this work, we focus on NVIDIA's GPUs and the CUDA framework, but our techniques may be generalized to other GPU platforms. We develop a compiler which targets the low-level SASS assembly, rather than the high-level source code or the intermediate-level PTX assembly. This allows for a degree of tuning and modification not possible at higher levels. We reconstruct program information such as the control-flow graph and call graph, thereby permitting data flow analysis. Thanks to extensive reverse-engineering, our compiler retains compatibility with numerous versions of the CUDA framework and several generations of NVIDIA GPUs. We are able to improve memory performance with optimizations not available in the proprietary compiler. We perform memory-allocation across the multiple types of available on-chip memory, making full use of available resources including registers, scratchpad memory, and the L1 data cache. This further allows us to tune occupancy - the number of threads allowed to be simultaneously active. We perform static and dynamic occupancy tuning, in order to find an effective balance between concurrency and resource contention. Our compiler can also be applied toward improvement of memory safety. We use it to implement dynamic taint tracking, a technique previously used on CPUs to identify sensitive data as it spreads through memory. By analyzing and modifying the low-level assembly, we minimize tracking overhead, track memory resources not visible to the programmer, and erase sensitive data before it has opportunity to leak. We evaluate our compiler across a number of benchmarks on NVIDIA devices of the Fermi, Kepler, Maxwell, and Pascal architectures. We demonstrate that our resource allocation and occupancy tuning provide significant improvement in both speed and energy usage. We additionally show that our GPU-specific optimizations for taint tracking can enormously reduce overhead. Location: Via Zoom Committee: Professor Zheng Zhang (advisor, chair) Professor Ulrich Kremer Professor Manish Parashar Professor Chen Ding (external member, University of Rochester) Start Date: 21 Dec 2021; Start Time: 09:00AM - 10:30AM Title: ABCinML: Anticipatory Bias Correction in Machine Learning Bio: Speaker: Abstract: Static models (i.e., train once, deploy forever) of machine learning (ML) rarely work in practical settings. Besides fluctuations in accuracy over time, they are likely to suffer from biases based on the poor representations or past injustices coded in human judgements. Thus, multiple researchers have begun to explore ways to maintain algorithmic fairness over time. One line of work focuses on”dynamic learning” i.e., retraining after each batch, and the other on ”robustlearning” which tries to make the algorithms robust across all possible future challenges. Robust learning often yields to (overly) conservative models and ”dynamic learning” tries to reduce biases soon after, they have occurred. We propose an anticipatory ”dynamic learning” approach for correcting the algorithm to prevent bias before it occurs. Specifically, we make use of anticipations regarding the relative distributions of population subgroups (e.g., relative ratio of maleand female applicants) in the next cycle to identify the right parameters for an importance weighing fairness approach. Results from experiments over multiple real-world datasets suggest that this approach has a promise for anticipatory bias correction. Location: Via Zoom Committee: Vivek K. Singh (Research advisor) David M. Pennock Amélie Marian Srinivas Narayana Start Date: 21 Dec 2021; Start Time: 11:00AM - 01:00PM Title: Natural Language Understanding through Emotions and Stylistic Elements Bio: Speaker: Abstract: In the context of text understanding, computational methods are used to study how humans utilize stylistic elements (visual and rhetorical) in addition to language to express emotions and opinions. In this dissertation, I first explore to what extent distributional semantic models can capture the intensity of emotions in lexical items. Using human judgment as gold standard, I employ language models for word-level emotion intensity prediction and present a technique to attain an emotional database for more fine-grained and more accurate word-level emotion predictions. The results indicate that: 1) the sentiment score of words can improve distributional models for emotion classification, and 2) language models perform better in practice in comparison to the state-of-the-art lexicons. Next, I present a statistical analysis of the role of emojis in text as a visual element that conveys emotion. I show the strong connection between the emoji and emotions, and the syntactic role of emojis in the text. The empirical results illustrate that emojis are used mainly to intensify the emotion in a sentence; however, in some cases, they replace a word or phrase in the sentence or signal contrast in a sarcastic context. The last part of the dissertation is an analysis of stylistic and rhetorical elements in standard and poetic Persian text. I present a corpus of Persian literary text mainly focusing on poetry, covering the 7th to 21st century annotated for century and style, with additional partial annotation of rhetorical figures. I present several computational experiments to analyze poetic styles, authors, and periods, as well as context shifts over time. Location: Via Zoom Committee: Dr. Gerard de Melo (advisor) Dr. Matthew Stone Dr. Yongfeng Zhang Dr. Smaranda Muresan(outside member) Start Date: 23 Dec 2021; Start Time: 09:00AM - Title: Generative Scene Graph Networks Bio: Speaker: Abstract: Human perception excels at building compositional hierarchies of parts and objects from unlabeled scenes that help systematic generalization. Yet most work on generative scene modeling either ignores the part-whole relationship or assumes access to predefined part labels. In this work, we propose Generative Scene Graph Networks (GSGNs), the first deep generative model that learns to discover the primitive parts and infer the part-whole relationship jointly from multi-object scenes without supervision and in an end-to-end trainable way. We formulate GSGN as a variational autoencoder in which the latent representation is a tree-structured probabilistic scene graph. The leaf nodes in the latent tree correspond to primitive parts, and the edges represent the symbolic pose variables required for recursively composing the parts into whole objects and then the full scene. This allows novel objects and scenes to be generated both by sampling from the prior and by manual configuration of the pose variables, as we do with graphics engines. We evaluate GSGN on datasets of scenes containing multiple compositional objects, including a challenging Compositional CLEVR dataset that we have developed. We show that GSGN is able to infer the latent scene graph, generalize out of the training regime, and improve data efficiency in downstream tasks. Location: Via Zoom Committee: Prof. Sungjin Ahn (Advisor) Prof. Karl Stratos Prof. Hao Wang Prof. Dong Deng Start Date: 13 Jan 2022; Start Time: 11:00AM - Title: Rethinking Curriculum Design for a Scalable CS Education Bio: Prof. Andy Gunawardena is an Associate Teaching Professor of Computer Science at Rutgers University. Prior to joining Rutgers in 2018, he served as an Associate Teaching Professor of Computer Science at Carnegie Mellon University (CMU) from 1998-2013 and as a Lecturer in Computer Science at Princeton University from 2013-2018. A dedicated CS educator and an evangelist for technology in education, he has been a PI and Co-PI of multiple grants from National Science Foundation, Hewlett Packard, Microsoft, Gates Foundation and Qatar foundation to research, build, deploy technological infrastructures to support student learning and measure their impact. He co-led the CMU’s multi-million-dollar measuring learning initiative in 2010 and is a co-founder of Classroom Salon and cubits.ai platforms that have been widely used by instructors and students worldwide. He holds 3 patents (CMU, Princeton) and has co-authored 2 textbooks in Computational Linear Algebra published by Springer-Verlag (1998) and Cengage (2003). Speaker: Abstract: Number of students taking CS courses have increased dramatically in recent times. At the same time, the availability of human resources to manage CS courses have decreased. The traditional teaching methods have not changed, and student overall educational experiences seem to have diminished over time and continue to do so. Today, instructors hardly understand the individual needs of the students they are responsible for. There are also significant disruptive pressures in higher education due to prevalence of online content and opportunities. Attracting and retaining women and under-represented groups continue to be a challenge. Students demand that instructors teach what is relevant for them to get jobs in industry and/or to pursue higher education opportunities. Addressing these issues require significant efforts in course innovation and long-term dedication. In this talk, we will address the efforts to revamp 3 introductory CS courses at Rutgers (CS 111, CS 112, CS 205) as well as challenges in designing, implementing, and delivering the popular CS 439 Introduction to Data Science course at Rutgers. We will argue that it is important to rethink what we teach, how we teach it, and how we measure student outcomes. We will also argue for the need to centralize the management of large enrollment courses and the need to create more data-driven measuring instruments to help scale student learning in large courses. Finally, we discuss some initial efforts to increase outreach into NJ communities colleges (CC) and provide better transfer opportunities for a diverse group of students from CCs. Location: Via Zoom Committee: Start Date: 03 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Designing Emotionally-Intelligent Digital Humans that Move, Express, and Feel Like Us! Bio: Aniket Bera is an Assistant Research Professor at the Department of Computer Science. His core research interests are in Affective Computing, Computer Graphics (AR/VR, Augmented Intelligence, Multi-Agent Simulation), Autonomous Agents, Cognitive modeling, and planning for intelligent characters. His work has won multiple awards at top Graphics/VR conferences. He has previously worked in many research labs, including Disney Research and Intel. Aniket's research has been featured on CBS, WIRED, Forbes, FastCompany, Times of India, etc. Speaker: Abstract: The creation of intelligent virtual agents (IVAs) or digital humans is vital for many virtual and augmented reality systems. As the world increasingly uses digital and virtual platforms for everyday communication and interactions, there is a heightened need to create human-like virtual avatars and agents endowed with social and emotional intelligence. Interactions between humans and virtual agents are being used in different areas including, VR, games and story-telling, computer-aided design, social robotics, and healthcare. Designing and building intelligent agents that can communicate and connect with people is necessary but not sufficient. Researchers must also consider how these IVAs will inspire trust and desire among humans. Knowing the perceived affective states and social-psychological constructs (such as behavior, emotions, psychology, motivations, and beliefs) of humans in such scenarios allows the agents to make more informed decisions and navigate and interact in a socially intelligent manner. In this talk, I will give an overview of our recent work on simulating intelligent, interactive, and immersive human-like agents who can also learn, understand and be sentient to the world around them using a combination of emotive gestures, gaits, and expressions. Finally, I will also talk about our many ongoing projects which use our AI-driven IVAs, including intelligent digital humans for urban simulation, mental health and therapy applications, and social robotics. Location: Via Zoom Committee: Start Date: 04 Feb 2022; Start Time: 04:00PM - 05:00PM Title: Predicting Crowd Egress and Environment Relationships to Support Building Design Optimization Bio: Speaker: Abstract: During the building plan conceptual design process, the crowd behavior of pedestrians is considered an important evaluation factor. The pedestrian crowd simulation models are generally adapted for contemporary built environment design and evaluation. However, in most cases, the processing time is limited by the computation resources. To optimize calculation, we developed a new methodology revealing the relationship between crowd egress and the built environment utilizing the neural network approach. With the proposed crowd simulating methodology, the computing precision and simulation speed is optimized significantly. A novel layout optimization method based on the neural network approach was tested in a case study at Metropolitan Museum. The automatically generated building plan, focusing on a safe evacuation plan, presented a professionally reasonable result, which also achieves comparable performance to the expert design result. Location: Via Zoom Committee: Mubbasir Kapadia Vladimir Pavlovic Mridul Aanjaneya Sepehr Assadi Start Date: 07 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Learning with Strategic Agents Bio: Fang-Yi is currently a Postdoctoral fellow at Harvard School of Engineering and Applied Sciences. He received the Ph.D. in Computer Science from the University of Michigan, and worked as a Postdoctoral research fellow at the School of Information at the University of Michigan. His research is broadly situated at the interface between machine learning, artificial intelligence, and economics. His recent work focuses on machine learning with strategic agents. Speaker: Abstract: In the age of artificial intelligence, properly deployed AI technology can transform society at every level, from individuals making decisions to institutes developing better policies. To achieve this, we need to understand the interaction between AI and society, such as how society provides AI inputs and how AI's outputs may affect society when strategic behavior is possible. I will highlight two examples of my work about this interaction: 1) Information elicitation: How can we elicit and aggregate high-quality information from strategic agents? Using variational statistics, I design peer prediction mechanisms that reward strategic agents for truthful reports even without verification. 2) Performative prediction: Can standard learning algorithms converge in supervised learning when the outcome distribution (strategically) responds to our predictive models? With techniques in dynamical systems, I show that the learning algorithm can converge to the global stable and optimal point when the learning rate is small enough, but exhibits Li-Yorke chaos when the algorithm is not cautious in the learning rates or has an overwhelming influence on the data distribution. Location: Via Zoom Committee: Start Date: 08 Feb 2022; Start Time: 10:30AM - Title: Putting BPC into Practice: Charting a Course to Aligning your Computing Demographics with those of your University Bio: Professor Carla E. Brodley is the Dean of Inclusive Computing at Northeastern University, where she serves as the founding Executive Director for the Center for Inclusive Computing, a national initiative funded by Pivotal Ventures, to increase the representation of women graduating with degrees in computing. Dr. Brodley served as Dean of Khoury College from 2014-2021. During her tenure as Dean, the representation of women majoring in computing increased from 19% to 32%. Prior to joining Northeastern, she was a professor of the Department of Computer Science and the Clinical and Translational Science Institute at Tufts University (2004-2014) and on the faculty of the School of Electrical Engineering at Purdue University (1994-2004). A fellow of the ACM, AAAI and AAAS, Dr. Brodley’s interdisciplinary machine learning research led to advances not only in computer science, but in other areas including remote sensing, neuroscience, digital libraries, astrophysics, content-based image retrieval of medical images, computational biology, chemistry, evidence-based medicine, and predictive medicine. Dr. Brodley’s numerous leadership positions include having served as program co-chair of the International Conference on Machine Learning, co-chair of AAAI, co-chair of CRA-WP, and associate editor of JAIR, JMLR, and PAMI. She was a member of the DSSG, and she served on the boards of International Machine Learning Society, the AAAI Council, and ISAT. She is currently on the CRA Board of Directors, and on the Board of Trustees of the Jackson Laboratory. Speaker: Abstract: For the last two decades professors, non-profits, philanthropists, NSF and other agencies have been working to broaden participation in computing (BPC) in higher-ed. Progress has been made, but often it is incremental and takes place in small pockets. At the same time, booming enrollments, college budget models, and other institutional factors frequently stand in the way of implementing systemic changes and work at cross purposes to BPC efforts. Launched in 2019 with funding from Pivotal Ventures LLC, an investment and incubation company created by Melinda French Gates, the Center for Inclusive Computing (CIC) is working in partnership with colleges and universities across the country to increase the representation of women – of all races and ethnicities – in computing. Today, the CIC is engaged with 22 partner schools – Rutgers among them – with the goal of identifying and removing their specific institutional barriers. Partner schools receive grant funding as well as technical advising and data collection and visualization. In her talk, Dr. Carla Brodley, the CIC’s Executive Director and former dean of Northeastern’s Khoury College of Computer Sciences, will explore the most common institutional barriers the CIC is seeing across its portfolio, and will dig into the barriers most relevant to Rutgers. She will discuss the concrete measures that can be taken to address barriers, such as creating a BA in computing, making CS1 a general education requirement, handling the distribution of prior computing experience in the intro sequence, and making CS+X actually work. Dr. Brodley will make the case that, if we want to move the national needle, we must pay close attention to issues of how to attract students to computing once they are already at university, in addition to the traditional focus on retention. As part of the talk, Dr. Stone will describe the specific actions Rutgers is taking to remove institutional barriers to BPC. Location: Via Zoom Committee: Start Date: 10 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Learning to address novel situations through human-robot collaboration Bio: Dr. Tesca Fitzgerald is a Postdoctoral Fellow in the Robotics Institute at Carnegie Mellon University. Her research is centered around interactive robot learning, with the aim of developing robots that are adaptive, robust, and collaborative when faced with novel situations. Before joining Carnegie Mellon, Dr. Fitzgerald received her PhD in Computer Science at Georgia Tech and completed her B.Sc at Portland State University. She is an NSF Graduate Research Fellow (2014), Microsoft Graduate Women Scholar (2014), and IBM Ph.D. Fellow (2017). Speaker: Abstract: As our expectations for robots' adaptive capacities grow, it will be increasingly important for them to reason about the novel objects, tasks, and interactions inherent to everyday life. Rather than attempt to pre-train a robot for all potential task variations it may encounter, we can develop more capable and robust robots by assuming they will inevitably encounter situations that they are initially unprepared to address. My work enables a robot to address these novel situations by learning from a human teacher’s domain knowledge of the task, such as the contextual use of an object or tool. Meeting this challenge requires robots to be flexible not only to novelty, but to different forms of novelty and their varying effects on the robot’s task completion. In this talk, I will focus on (1) the implications of novelty, and its various causes, on the robot’s learning goals, (2) methods for structuring its interaction with the human teacher in order to meet those learning goals, and (3) modeling and learning from interaction-derived training data to address novelty. Location: Via Zoom Committee: Start Date: 11 Feb 2022; Start Time: 10:00AM - 11:00AM Title: Synthesizing Safe and Efficient Kernel Extensions for Packet Processing Bio: Speaker: Abstract: Extended Berkeley Packet Filter (BPF) has emerged as a powerful method to extend packet-processing functionality in the Linux operating system. Users can write a BPF program and attach it in the kernel at specific hooks (e.g., network device driver) to process packets. To ensure safe execution (e.g., crash-free) of a user-developed BPF program in kernel context, Linux uses an in-kernel static checker. A BPF program is allowed to execute only if it is proved safe by the checker. However, developing high-performance BPF programs is not easy because every optimization must respect the checker’s intricate safety rules. Even small performance optimizations to BPF code (e.g., 5% gains) must be meticulously hand-crafted by expert developers. We present K2, a program-synthesis-based compiler that automatically optimizes BPF bytecode with formal correctness and safety guarantees. K2 produces code with 6–26% reduced size, 1.36–55.03% lower average packet-processing latency, and 0–4.75% higher throughput (packets per second per core) relative to the best clang-compiled program, across benchmarks drawn from production systems. K2 incorporates several domain-specific techniques to make synthesis practical by accelerating equivalence-checking of BPF programs by 6 orders of magnitude. Location: Via Zoom Committee: Prof. Srinivas Narayana (advisor) Prof. Santosh Nagarakatte Prof. He Zhu Prof. Ahmed Elgammal Start Date: 14 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Secure Multi-Party Computation: from Theory to Practice Bio: Peihan Miao is an assistant professor of computer science at the University of Illinois Chicago (UIC). Her research interests lie broadly in cryptography, theory, and security, with a focus on secure multi-party computation. She received her Ph.D. from the University of California, Berkeley in 2019 and her B.S. degree from Shanghai Jiao Tong University in 2014. Before joining UIC, she was a research scientist at Visa Research. Speaker: Abstract: Encryption is the backbone of cybersecurity. While encryption can secure data both in transit and at rest, in the new era of ubiquitous computing, modern cryptography also aims to protect data during computation. Secure multi-party computation (MPC) is a powerful technology to tackle this problem, which enables distrustful parties to jointly perform computation over their private data without revealing their data to each other. Although it is theoretically feasible and provably secure, the adoption of MPC in real industry is still very much limited as of today, the biggest obstacle of which boils down to its efficiency.My research goal is to bridge the gap between the theoretical feasibility and practical efficiency of MPC. Towards this goal, my research spans both theoretical and applied cryptography. In theory, I develop new techniques for achieving general MPC with the optimal complexity, bringing theory closer to practice. In practice, I design tailored MPC to achieve the best concrete efficiency for specific real-world applications. In this talk, I will discuss the challenges in both directions and how to overcome these challenges using cryptographic approaches. I will also show strong connections between theory and practice. Location: Via Zoom Committee: Start Date: 15 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Automated Analysis and Implementation for Modern Networks Bio: Mina Tahmasbi Arashloo is a presidential post-doctoral fellow at the computer science department of Cornell University, working with Nate Foster and Rachit Agarwal. Her research focuses on programmable computer networks, specifically on designing and developing automated processes to program and reason about modern networks. In doing so, she brings in techniques from other computer science disciplines such as formal methods, programming languages, and hardware design. She received her Ph.D. from Princeton University, where she was advised by Jennifer Rexford, and her B.Sc. from Sharif University of Technology. She has been named a Rising Star in Networking and Communications by N2Women in 2021 and has received the ACM SIGCOMM Doctoral Dissertation Award. Speaker: Abstract: Modern computer networks are complex distributed systems, with thousands of heterogeneous software and hardware components working together to deliver traffic from sources to destinations. They serve online services that demand much more than basic network connectivity, asking for certain levels of performance, security, and reliability from the network. Meeting these growing expectations requires running increasingly sophisticated functionality in networking software and hardware. Yet, we still lack proper tools and techniques to analyze and implement these advanced functionalities and, instead, mostly rely on manual error-prone processes to ensure that the network can fulfill the requirements of the various applications it has to serve. In this talk, I will discuss how my research helps bridge this gap by automating the analysis and implementation of such advanced network functionality. First, I will focus on transport-layer algorithms. These algorithms are key to providing high performance for different classes of traffic but are quite challenging to implement on high-speed network hardware due to their inter-dependent complex operations. I will present a hardware architecture that can be programmed, with modest development effort, to execute new transport algorithms at 100Gbps. Next, I will focus on automated reasoning about network performance. Network performance depends on packet-level interactions between different traffic flows, making it easy to overlook corner cases that can cause performance problems. I will demonstrate how to use formal methods to automatically generate traffic patterns that lead to poor performance in a given network. Finally, I will discuss my future plans on using rigorous and automated tools and techniques to create networks that are robust and explainable. Location: Via Zoom Committee: Start Date: 17 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Optimization Opportunities in Human-in-the-loop Systems Bio: Senjuti Basu Roy is the Panasonic Chair in Sustainability and an Associate Professor in the Department of Computer Science at the New Jersey Institute of Technology. Her research focus lies on the intersection of data management, data exploration, and AI, especially enabling human-machine analytics in scale. Senjuti has published more than 70 research papers in high impact data management and data mining conferences and journals. She is the tutorial co-chair of VLDB 2023, The Web Conference 2022, has served as the Mentorship co-chair of SIGMOD 2018, PhD workshop co-chair of VLDB 2018, and has been involved in organizing several international workshops and meetings. She is a recipient of the NSF CAREER Award, a PECASE nominee, and one of the 100 invited early career engineers to attend the National Academy of Engineering’s 2021 US Frontiers of Engineering Symposium. Speaker: Abstract: An emerging trend is to leverage an under-explored and richly heterogeneous pool of human knowledge inside machine algorithms, a practice popularly termed as human-in-the-loop (HIL) process. A wide variety of applications, starting from query processing to text translation, feature engineering, or even human decision making in complex uncertain environments stand to benefit from such synergistic man-machine collaboration. This talk will discuss our ongoing projects, recent research results, and impacts that investigate a variety of optimization opportunities inside such HIL systems, considering the roles and responsibilities of three key stakeholders - humans (workers), machines (algorithms), and platforms (online infrastructure where the work takes place). Following that, the talk will briefly discuss how this ongoing research is well aligned in the context of the future-of-work. Location: Via Zoom Committee: Start Date: 17 Feb 2022; Start Time: 02:00PM - Title: Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention Bio: Speaker: Abstract: We propose a Dynamic Graph-Based Spatial-Temporal Attention (DG-STA) method for hand gesture recognition. The key idea is first to construct a fully-connected graph from a hand skeleton. Then, the node features and edges are automatically learned via a self-attention mechanism that performs in both spatial and temporal domains. We further propose to leverage the spatial-temporal cues of joint positions to guarantee robust recognition in challenging conditions. We carry out extensive experiments on benchmarks (DHG-14/28 and SHREC’17) and prove the superior performance of our method compared with the state-of-the-art methods. Location: Via Zoom Committee: Prof. Dimitris Metaxas (advisor) Prof. Hao Wang Prof. Konstantinos Michmizos Prof. Shiqing Ma Start Date: 21 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Enabling the Communication of Physical Experiences Bio: Jun Nishida is a postdoctoral fellow at the University of Chicago's Computer Science Department, advised by Prof. Pedro Lopes. He received a Ph.D. in Human Informatics at the University of Tsukuba, Japan in 2019. He is interested in exploring interaction techniques where people can communicate their physical experiences to support each other, with applications in the fields of rehabilitation, education, and design. To this end, he engineers wearable interfaces that share bodily cues across people by means of electrical muscle stimulation, exoskeletons, virtual/augmented reality systems, along human factors. He has received ACM UIST Best Paper Award, ACM CHI Best Paper Honorable Mention Award, Microsoft Research Asia Fellowship Award, and Forbes 30 Under 30 Award among others. He has worked as a Ph.D. fellow at Microsoft Research Asia and as a research assistant at Sony Computer Science Laboratories. | http://junnishida.net Speaker: Abstract: While today’s tools allow us to communicate effectively with others via video and text, they leave out other critical communication channels, such as bodily cues. These cues are important not only for face-to-face communication but even when communicating forces (muscle tension, movement, etc), feelings, and emotions. Unfortunately, the current paradigm of user interfaces used for communication between two users is rooted only in symbolic and graphical communication, leaving no space to add these additional and critical modalities such as touch, forces, etc.This is precisely the research question I tackle in my work: how can we also communicate our physical experience across people?In this talk, I introduce how I have engineered wearable devices that allow for sharing physical experiences across users, such as between a physician and a patient, including people with neuromuscular diseases and even children. These custom-built user interfaces include virtual reality systems, exoskeletons, and interactive devices based on electrical muscle stimulation. I then investigated how we can extend this concept to support interactive activities, such as product design, through the communication of one's bodily cues. Lastly, I discuss how we can further explore the possibilities enabled by a user interface that communicates more than audio-visual cues and the roadmap for using this approach in new territories, such as enabling more empathic communication. Location: Via Zoom Committee: Start Date: 22 Feb 2022; Start Time: 10:30AM - 12:00PM Title: Protecting User Security and Privacy in Emerging Computing Platforms Bio: Yuan Tian is an Assistant Professor of Computer Science at the University of Virginia. Before joining UVA, she obtained her Ph.D. from Carnegie Mellon University in 2017 and interned at Microsoft Research, Facebook, and Samsung Research. Her research interests involve security and privacy and its interactions with computer systems, machine learning, and human-computer interaction. Her current research focuses on developing new computing platforms with strong security and privacy features, particularly in the Internet of Things and mobile systems. Her work has real-world impacts as countermeasures and design changes have been integrated into platforms (such as Android, Chrome, Azure, and iOS), and also impacted the security recommendations of standard organizations such as the Internet Engineering Task Force (IETF). She is a recipient of Google Research Scholar Award 2021, Facebook Research Award 2021, NSF CAREER award 2020, NSF CRII award 2019, Amazon AI Faculty Fellowship 2019, CSAW Best Security Paper Award 2019, and Rising Stars in EECS 2016. Her research has appeared in top-tier venues in security, machine learning, and systems. Her projects have been covered by media outlets such as IEEE Spectrum, Forbes, Fortune, Wired, and Telegraph. Speaker: Abstract: Computing is undergoing a significant shift. First, the explosive growth of the Internet of Things (IoT) enables users to interact with computing systems and physical environments in novel ways through perceptual interfaces (e.g., microphones and cameras). Second, machine learning algorithms collect huge amounts of data and make critical decisions on new computing systems. While these trends bring unprecedented functionality, they also drastically increase the number of untrusted algorithms, implementations, interfaces, and the amount of private data processed by them, endangering user security and privacy. To regulate these security and privacy issues, privacy regulations such as GDPR (General Data Protection Regulation), CCPA (California Consumer Privacy Act) went into effect. However, there is a huge gap between the desired high-level security/privacy/ethical properties (from regulations, specifications, users’ expectations) and low-level real implementations.To bridge the gap, my work aims to change how platform architects design secure systems, assist developers by detecting security and privacy violation of implementations and build usable and scalable privacy-preserving systems. In this talk, I will present how my group designs principled solutions to ensure modern and emerging computing platforms' security and privacy. In this talk, I will introduce two developer tools we build to detect security and privacy violations. Using the tools, we found large numbers of policy violations in healthcare voice applications and security property violations in IoT messaging protocol implementations. Additionally, I will discuss our recent work on scalable privacy-preserving machine learning. Location: Via Zoom Committee: Start Date: 28 Feb 2022; Start Time: 10:30AM - 12:00PM Title: When Second Order Methods Shine: Big Batches, Bayes, and Bilevel Bio: Guodong Zhang is a PhD candidate in the machine learning group at the University of Toronto, advised by Roger Grosse. His research lies at the intersection between machine learning, optimization, and Bayesian statistics. In particular, his research focuses on understanding and improving algorithms for optimization, Bayesian inference, and multi-agent games in the context of deep learning. He has been recognized through the Apple PhD fellowship, Borealis AI fellowship, and many other scholarships. In the past, he has also spent time at Institute for Advanced Study of Princeton and industry research labs (including DeepMind, Google Brain, and Microsoft Research). Speaker: Abstract: Many challenges in modern machine learning involve the three fundamental problems of optimization, Bayesian inference, and multi-player games. In this talk, I will discuss how the use of second-order information – e.g., curvature or covariance – can help in all three problems, yet with vastly different roles in each. First, I will present a noisy quadratic model, which qualitatively predicts scaling properties of a variety of optimizers and in particular suggests that second-order optimization algorithms would extend perfect scaling to much bigger batches. Second, I will show how we can derive and implement scalable and flexible Bayesian inference algorithms from standard second-order optimization algorithms. Third, I will describe a novel second-order algorithm that finds desired equilibria and saves us from converging to spurious fixed points in two-player sequential games (i.e. bilevel optimization) or even more general settings. Finally, I will conclude how my research would pave the way towards intelligent machines that can learn from experience efficiently, reason about their own decisions, and act in our interests. Location: Via Zoom Committee: Start Date: 01 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Advancing Digital Safety for High-Risk Communities Bio: Allison McDonald (she/her) is a computer science PhD Candidate at the University of Michigan and a Research Fellow at the Center on Privacy & Technology at Georgetown Law. Her research interests lie in the intersection of security, privacy, and human-computer interaction, with a particular emphasis on how technology exacerbates marginalization and impacts digital safety. Her work has been recognized with Best Paper Awards at the USENIX Security Symposium, IEEE Security & Privacy Symposium, and the ACM Conference on Human Factors in Computing Systems (CHI). Before beginning her PhD, Allison was a Roger M. Jones fellow at the European University Viadrina studying international human rights and humanitarian law. She has a BSE in computer science and a BS in German from the University of Michigan. Speaker: Abstract: Security and privacy research has led to major successes in improving the baseline level of security for the general population. Nevertheless, privacy and security tools and strategies are not equally effective for everyone—many high-risk communities face security, privacy, and safety risks that are not well addressed by current solutions. To advance digital safety for high-risk users, my work uses a broad range of methods to investigate the digital safety needs and challenges for high-risk users, quantify the impact of government regulation and corporate policy on safety, and inform the design technical of and procedural interventions that support safety for all.In this talk, I will discuss two studies in detail that showcase the opportunities of taking an interdisciplinary approach to supporting digital safety for high-risk communities such as sex workers, undocumented immigrants, and survivors of intimate partner violence. First, I will discuss findings from an in-depth qualitative interview study on the security needs and practices of sex workers in Europe, highlighting their safety needs as well as technical and policy challenges that impede their safety. Then, I will describe a large-scale global measurement study on geoblocking, which reveals corporate and legal policies that are contributing to the fragmentation of Internet access worldwide. I will further provide an overview of my future research agenda, which will leverage both qualitative and quantitative methods to inform policy and technical design. Location: Via Zoom Committee: Start Date: 03 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Building Robots that Humans Accept Bio: Christoforos Mavrogiannis is a postdoctoral Research Associate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Prof. Siddhartha Srinivasa. His interests lie at the intersection of robotics, human-robot interaction, and artificial intelligence. His research often draws insights from algebraic topology and dynamical systems, tools from machine learning, planning and control, and inspiration from social sciences. He is a full-stack roboticist, passionate about real-world deployment of robot systems, and extensive benchmarking with users. He has been a best-paper award finalist at the ACM/IEEE International Conference on Human-Robot Interaction (HRI), and selected as a Pioneer at the HRI and RSS conferences. He has also led open-source initiatives (Openbionics, MuSHR), for which he has been a finalist for the Hackaday Prize and a winner of the Robotdalen International Innovation Award. His work has received coverage from many media outlets including Wired, IEEE Spectrum, GeekWire, RoboHub, and the Hellenic Broadcasting Corporation. Christoforos holds M.S. and Ph.D. degrees from Cornell University, and a Diploma in mechanical engineering from the National Technical University of Athens. Speaker: Abstract: Robotics has transformed sectors like manufacturing and fulfillment which now rely on robots to meet their goals. Conventionally, these robots operate in isolation from humans to ensure safety and efficiency. Lately, there have been efforts towards bringing robots closer to humans to assist in everyday-life tasks, enhance productivity, and augment human capabilities. Despite these efforts, robotic technology has not reached widespread acceptance outside of factories; robot autonomy is often not robust, producing new problems that outweigh its benefits for users. Inspired by theories of technology acceptance, my research strives to develop highly functional, safe, and comfortable robots that humans accept. In this talk, I argue that the path towards acceptance requires imbuing robots with a deeper understanding of how users perceive and react to them. To motivate this perspective, I will share insights on robot navigation in dynamic environments, a fundamental task with many crucial applications ranging from collaborative manufacturing to warehouse automation and healthcare. I will describe a human-inspired algorithmic framework for crowd navigation, highlighting how mathematical abstractions of multiagent behavior enable safe, efficient, and positively perceived robot motion across a series of extensive empirical studies involving real robots and human subjects. Inspired by field-deployment challenges, I will then present a data-driven framework that enables robots to recover from failure via bystander help without overloading users. I will conclude with future directions on the development of shared and full robot autonomy that explicitly reasons about human perceptions to produce safe, trustworthy, and comfortable robot behavior. Location: Via Zoom Committee: Start Date: 07 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Design and Formally Verify Post-Quantum Cryptography. Bio: Xiong (Leo) Fan is a cryptography researcher at Algorand. He obtained his Ph.D. from Cornell University and then spent one year as a postdoc at University of Maryland. During his doctoral study, he has been interned at Simons Institute, Yahoo! Labs, Bell Labs and IBM TJ Watson Research Center. His research interests are cryptography and its intersections with formal verification and hardware acceleration. Speaker: Abstract: The use of cryptography is ubiquitous in our daily life. However, the development of faster quantum computers can break today’s crypto-systems. Therefore, it is imperative that we develop and deploy post-quantum cryptography, before scalable quantum computers become a reality.My research focuses on designing and formally verifying cryptographic primitives based on post-quantum assumptions. My approach combines cryptography and formal methods, aiming to bring provable security to real-world applications. In this talk, I will discuss how to design a post-quantum secure encryption scheme that provides fine-grained access control over encrypted data. Further, I will describe a system called AutoLWE, capable of mechanizing security proofs of lattice-based cryptosystems. Location: Via Zoom Committee: Start Date: 10 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Towards post-quantum cryptography: complexity and protocols Bio: Katerina Sotiraki is currently a post-doctoral researcher at the EECS Department of UC Berkeley working with Alessandro Chiesa and Raluca Ada Popa. She received her PhD from the EECS Department at MIT where she was advised by Vinod Vaikuntanathan. She works on cryptography, complexity theory, and secure computation, with focus on cryptography based on lattices. Speaker: Abstract: The advent of quantum computers places many widely usedcryptographic protocols at risk. In response to this threat, thefield of post-quantum cryptography has emerged. The most broadlyrecognized post-quantum protocols are related to lattices. Beyondtheir resistance to quantum attacks, lattices are instrumental toolsin cryptography due to their rich mathematical structure. In thistalk, I will present my work on understanding the complexity oflattice problems and on constructing lattice-based cryptographicprotocols useful in practical scenarios. First, I will present anoptimal construction for worst-case collision-resistant hash functionsbased on a lattice problem. Second, I will show the firstlattice-based construction of cryptographic proofs with minimalcommunication and zero-knowledge for any language in NP. Location: Via Zoom Committee: Start Date: 21 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Efficient and Ethical Data Processing -- introducing new information-theoretic tools Bio:  Sumegha Garg is a Michael O. Rabin postdoctoral fellow in theoretical computer science at Harvard University. She received her Ph.D. in Computer Science from Princeton University in 2020, advised by Mark Braverman. She uses her background in computational complexity theory to answer fundamental questions in applied areas such as learning, data streaming, and cryptography. In particular, she is interested in determining the limits of space-efficient computing and establishing the foundations of responsible computing. Her awards include Siebel Scholarship (2019-20), Microsoft Dissertation Grant (2019) and Rising Star in EECS (2019). Speaker: Abstract: Data-driven algorithms have seen great success in a wide range of domains, from product recommendations to cyber-security and healthcare. The ever-expanding use of these algorithms brings in new efficiency and ethical concerns. Firstly, when analyzing massive amounts of data, memory is increasingly becoming a bottleneck computational resource. Secondly, as data-driven decision-making systems are being used to make important decisions about humans, they raise a host of fairness concerns and fears for potential discrimination.In this talk, I'll give an overview of my efforts to understand these concerns using modern mathematical tools from computational complexity theory and information theory. I'll then dig deeper into two results that illustrate the efficacy of information-theoretic techniques in quantifying memory usage as well as characterizing challenges in algorithmic fairness. First, I'll introduce the coin problem – given independent coin tosses, detect which way a coin is biased – and show that this problem is at the heart of hardness results for many data streaming problems. I’ll go on to describe new techniques we developed to determine the memory needed for solving the coin problem. Second, I'll show how informativeness of predictions on a subpopulation plays a key role in connecting various conflicting approaches to algorithmic fairness. Location: Via Zoom Committee: Start Date: 22 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Reasoning and Learning in Interactive Natural Language Systems Bio: Alane Suhr is a PhD Candidate in the Department of Computer Science at Cornell University, advised by Yoav Artzi. Her research spans natural language processing, machine learning, and computer vision, with a focus on building systems that participate and continually learn in situated natural language interactions with human users. Alane’s work has been recognized by paper awards at ACL and NAACL, and has been supported by fellowships and grants, including an NSF Graduate Research Fellowship, a Facebook PhD Fellowship, and research awards from AI2, ParlAI, and AWS. Alane has also co-organized multiple workshops and tutorials appearing at NeurIPS, EMNLP, NAACL, and ACL. Previously, Alane received a BS in Computer Science and Engineering as an Eminence Fellow at the Ohio State University. Speaker: Abstract: Systems that support expressive, situated natural language interactions are essential for expanding access to complex computing systems, such as robots and databases, to non-experts. Reasoning and learning in such natural language interactions is a challenging open problem. For example, resolving sentence meaning requires reasoning not only about word meaning, but also about the interaction context, including the history of the interaction and the situated environment. In addition, the sequential dynamics that arise between user and system in and across interactions make learning from static data, i.e., supervised data, both challenging and ineffective. However, these same interaction dynamics result in ample opportunities for learning from implicit and explicit feedback that arises naturally in the interaction. This lays the foundation for systems that continually learn, improve, and adapt their language use through interaction, without additional annotation effort. In this talk, I will focus on these challenges and opportunities. First, I will describe our work on modeling dependencies between language meaning and interaction context when mapping natural language in interaction to executable code. In the second part of the talk, I will describe our work on language understanding and generation in collaborative interactions, focusing on continual learning from explicit and implicit user feedback. Location: Via Zoom Committee: Start Date: 24 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Advanced Cryptography Bio: Rishab Goyal is a postdoctoral researcher in the Cryptography and Information Security group at the MIT Computer Science and AI Laboratory. His primary interests are cryptography and computer security, with a special focus on developing provably-secure systems with advanced cryptographic capabilities and post-quantum security. Prior to MIT, he was an Apple Research Fellow at the Simons Institute for the Theory of Computing. He received his Ph.D. from UT Austin, advised by Brent Waters, and B.Tech. from IIT Delhi. His Ph.D. was supported by the IBM Ph.D. Fellowship and the Graduate Dean's Prestigious Fellowship at UT Austin. And, his doctoral dissertation on building traitor tracing systems was awarded the Bert Kay Dissertation Award for best doctoral thesis at UT Austin. Speaker: Abstract: Rapid scientific innovations have exposed the growing tension between functionality and security in emerging technologies. This is further exacerbated by the looming threat of quantum computers. The goal of advanced cryptography is to build secure systems that simultaneously address these concerns while equally prioritizing efficiency. In my research, I develop provably-secure advanced cryptographic systems by relying on the hardness of well-studied mathematical problems related to integer lattices, number theory, noisy parity learning, and more.In this talk, I will describe some of the advanced systems that I have designed which include traceable encryption, program obfuscation, aggregate signatures, decentralized proof systems, and multi-user encryption systems. The focus of the talk will be on my work in building advanced encryption systems that solved the 25-year-old open problem of traitor tracing. Traitor tracing has great practical impact in protecting from insider corruption and providing accountability alongside privacy in a wide array of applications. It has connections to many other areas such as differential privacy, software watermarking and leasing, and quantum copy protection. I will also introduce new techniques in lattice-based cryptography which have been instrumental for proving post-quantum security of such advanced encryption systems. Finally, I will speak about the impact advanced cryptography will have in influencing public policy, and how it provides science as a tool for legislators to protect society from all kinds of emerging threats. Location: Via Zoom Committee: Start Date: 24 Mar 2022; Start Time: 02:00PM - 03:00PM Title: Understanding Event Processes in Natural Language Bio: Muhao Chen is an Assistant Research Professor at the Department of Computer Science, USC, where he directs the Language Understanding and Knowledge Acquisition (LUKA) Lab. His research focuses on minimally supervised data-driven machine learning for natural language understanding, structured data processing, and knowledge acquisition from unstructured data. His work has been recognized with an NSF CRII Award, a best student paper award at ACM BCB, and a best paper award nomination at CoNLL. Muhao obtained his PhD degree from UCLA Department of Computer Science in 2019, and was a postdoctoral fellow at UPenn prior to joining USC. Additional information is available at https://muhaochen.github.io/ Speaker: Abstract: Human languages evolve to communicate about events happening in the real world. Therefore, understanding events plays a critical role in natural language understanding (NLU). A key challenge to this mission lies in the fact that events are not just simple, standalone predicates. Rather, they are often described at different granularities, temporally form event processes, and are directed by specific central goals in a context. This talk covers recent advances in event process understanding in natural language. In this context, I will first introduce how to recognize the evolution of events from natural language, then how to solve fundamental problems of event process completion, intention prediction and membership prediction, and how knowledge about event processes can benefit various downstream NLU and machine perception tasks. I will also briefly present some open problems in this area, along with a system demonstration. Location: Via Zoom Committee: Start Date: 25 Mar 2022; Start Time: 03:00PM - Title: Neural Logic Reasoning and Applications Bio: Speaker: Abstract: Recent years have witnessed the success of deep neural networks in many research areas. The fundamental idea behind the design of most neural networks is to learn similarity patterns from data for prediction and inference, which lacks the ability of cognitive reasoning. However, the concrete ability of reasoning is critical to many theoretical and practical problems. On the other hand, traditional symbolic reasoning methods do well in making logical inference, but they are mostly hard rule-based reasoning, which limits their generalization ability to different tasks since different tasks may require different rules. In this work, we propose a Neural Logical Reasoning (NLR) framework to integrate the power of deep learning and logical reasoning. NLR is a dynamic modularized neural architecture that learns basic logical operations such as AND, OR, NOT as neural modules, and conducts propositional logical reasoning through the logical structured network for inference. Experiments show that our approach achieves state-of-the-art performance in various application scenarios. Moreover, we utilize a neural architecture search strategy to allow the model to learn the adaptive logical neural architectures automatically which brings flexibility to our framework. Location: Via Zoom Committee: Dr. Yongfeng Zhang (Chair, advisor) Dr. He Zhu (Rutgers, Computer Science Dept.) Dr. Hao Wang (Rutgers, Computer Science Dept.) Dr. Qingyao Ai (External member from the University of Utah) Start Date: 29 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Intuitive Robot Shared-Control Interfaces via Real-time Motion Planning and Optimization Bio: Speaker: Abstract: My research focuses on making robots intuitive to control and work alongside for as many people as possible, specifically in areas where people are understaffed or overworked such as nursing, homecare, and manufacturing. In this talk, I will overview numerous robot shared-control interfaces I have developed to be intuitive and easy-to-use, even for novice users, by blending users’ inputs with robot autonomy on-the-fly. I will highlight novel motion planning and motion optimization methods that enable these interfaces by quickly synthesizing smooth, feasible, and safe motions that effectively reflect objectives specified by the user and robot autonomy signals in real-time. I will comment on my ongoing and future work that will push the potential of these technical methods and physical robot systems, all striving towards broad and motivating applications such as remote homecare, tele-nursing, and assistive technologies. Location: Via Zoom Committee: Start Date: 29 Mar 2022; Start Time: 01:00PM - 03:00PM Title: Disentangled Generative Models and their Applications Bio: Speaker: Abstract: Generative models are fascinating for their ability to synthesize versatile visual, audio, and textual information from mere noise. This generative process usually requires a model to perceive and compress high-dimensional data into a compact, low-dimensional latent space, where each dimension encodes valuable semantic variations in the original data space. Disentangling generative models makes them more fun to play with. This thesis studies the unsupervised disentangling of the latent space in GANs focused on the image domain and further extended to multi-modalities (image captioning and text-to-image synthesis). Derived from disentanglement, this thesis also covers studies on model interpretability and human-controllable data synthesis. First, we work on general-purpose unsupervised disentanglement. A novel GAN-based disentanglement framework with One-Hot Sampling and Orthogonal Regularization (OOGAN) is proposed. Second, we tackle a more specific task: disentangling coarse and fine level style attributes for GAN. We design a Vector-Quantized module for better pose-identity disentanglement and a novel joint-training scheme merging GAN and Auto-Encoder. Lastly, we study two applications taking advantage of a better disentangled GAN with mutual information learning, which are text-and-image mutual-translation and sketch-to-image generation. Location: Via Zoom Committee: Ahmed Elgammal (elgammal@cs.rutgers.edu) Vladimir Pavlovic (vladimir@cs.rutgers.edu) Yongfeng Zhang (yongfeng.zhang@rutgers.edu) Xiaohui Shen (shenxiaohui@bytedance.com). Start Date: 31 Mar 2022; Start Time: 10:30AM - 12:00PM Title: Democratizing Haptics Bio: Hasti Seifi is an assistant professor in the Department of Computer Science at the University of Copenhagen. Previously, she was a postdoctoral research fellow at the Max Planck Institute for Intelligent Systems in Germany. She received her Ph.D. in computer science from the University of British Columbia in 2017, her M.Sc. from Simon Fraser University in 2011, and her B.Sc. from the University of Tehran in 2008. Her research interests lie at the intersection of human-computer interaction, haptics, and social robotics. Hasti has published her work in top-tier venues such as the ACM CHI Conference, IEEE Transactions on Haptics (ToH), IEEE World Haptics Conference, and ACM Transactions on Human-Robot Interaction (THRI). Her work was recognized by an NSERC postdoctoral fellowship (2018), the EuroHaptics best Ph.D. thesis award (2017), and a Maria Klawe award for her efforts in computer science diversity and outreach (2017). Her research has drawn the attention of major tech companies such as Meta Reality Labs, Microsoft Research, and Immersion Corporation (the world's largest haptics company). Hasti serves the HCI and haptics communities in various roles such as the program chair of EuroHaptics 2022 and the associate chair for IEEE WHC, ACM CHI, and ACM UIST conferences. Speaker: Abstract: Haptics, the science and technology of programmable touch experiences, is increasingly used to improve functionality in virtual reality, robotics, and wearable applications. Investments by large companies such as Meta, Apple, and Google are helping to drive growth in the field. Yet, haptic technology is complex and specialized. Thus, current progress is limited to achievements by a small group of experts and lab-based studies that do not scale to the diverse needs and preferences of users. My goal is to create a future where everyone can understand, access, and adapt touch technology for their needs. In this talk, I present my research toward this goal in the areas of haptic personalization, multisensory design, haptic technology collections, and social physical interactions with robots. Finally, I discuss the remaining steps toward a future where effective and fun virtual touch experiences are designed, tailored, and embraced by all. Location: Via Zoom Committee: Start Date: 04 Apr 2022; Start Time: 10:30AM - 12:00PM Title: Scalable and Provably Robust Algorithms for Machine Learning Bio: Yu Cheng is an assistant professor in the Mathematics department at the University of Illinois at Chicago. He obtained his Ph.D. in Computer Science from the University of Southern California. Before joining UIC, he was a postdoc at Duke University and a visiting member at the Institute for Advanced Study. His main research interests include machine learning, optimization, and game theory. Speaker: Abstract: As machine learning plays a more prominent role in our society, we need learning algorithms that are reliable and robust. It is important to understand whether existing algorithms are robust against adversarial attacks and design new robust solutions that work under weaker assumptions. The long-term goal is to bridge the gap between the growing need for robust algorithms and the lack of systematic understanding of robustness. In this talk, I will discuss the challenges that arise in the design and analysis of robust algorithms for machine learning. I will focus on three lines of my recent work: (1) designing faster and simpler algorithms for high-dimensional robust statistics where a small fraction of the input data is arbitrarily corrupted, (2) analyzing the optimization landscape of non-convex approaches for low-rank matrix problems and making non-convex optimization robust against semi-random adversaries, and (3) considering learning in the presence of strategic behavior where the goal is to design good algorithms that account for the agents' strategic responses. Location: Via Zoom Committee: Start Date: 05 Apr 2022; Start Time: 09:00AM - 10:00AM Title: Generative Neurosymbolic Machines Bio: Speaker: Abstract: Reconciling symbolic and distributed representations is a crucial challenge that can potentially resolve the limitations of current deep learning. Remarkable advances in this direction have been achieved recently via generative object-centric representation models. While learning a recognition model that infers object-centric symbolic representations like bounding boxes from raw images in an unsupervised way, no such model can provide another important ability of a generative model, i.e., generating (sampling) according to the structure of learned world density. In this work, we propose Generative Neurosymbolic Machines, a generative model that combines the benefits of distributed and symbolic representations to support both structured representations of symbolic components and density-based generation. These two crucial properties are achieved by a two-layer latent hierarchy with the global distributed latent for flexible density modeling and the structured symbolic latent map. To increase the model flexibility in this hierarchical structure, we also propose the StructDRAW prior. In experiments, we show that the proposed model significantly outperforms the previous structured representation models as well as the state-of-the-art non-structured generative models in terms of both structure accuracy and image generation quality. Location: Via Zoom Committee: Prof. Sungjin Ahn (advisor)  Prof. Hao Wang  Prof. Dimitris Metaxas Prof. Sudarsun Kannan Start Date: 12 Apr 2022; Start Time: 09:00AM - 10:00AM Title: Factoring and Learning Algorithms for Low-depth Algebraic Circuits. Bio: Speaker: Abstract: Polynomial factoring and learning arithmetic circuits (a.k.a. circuit-reconstruction) are two fundamental problems in algebraic complexity. They are deeply connected to the questions of circuit lower bound and polynomial identity testing (PIT). In this dissertation, we present various new results on learning and factoring of low-depth circuits, while emphasizing these connections. Some of our main results are as follows:1. Sparse Polynomial Factoring: We study the problem of deterministic factorization of sparse polynomials. We show that if $f \in \Fn$ is a polynomial with $s$ monomials, with individual degrees of its variables bounded by $d$, then $f$ can be deterministically factored in time $s^{\poly(d)\log n}$. A crucial ingredient in our proof is a quasi-polynomial sparsity bound for factors of sparse polynomials of bounded individual degree. In particular we show if $f$ is an $s$-sparse polynomial in $n$ variables, with individual degrees of its variables bounded by $d$, then the sparsity of each factor of $f$ is bounded by $s^{\BigO({d^2\log{n}})}$. 2. Learning low-rank tensors and depth 3 multilinear circuits: We give new and efficient black-box reconstruction algorithms for some classes of depth-$3$ arithmetic circuits. As a consequence, we obtain the first efficient algorithm for computing the tensor rank and for finding the optimal tensor decomposition as a sum of rank-one tensors when the input is a {\em constant-rank} tensor.3. Learning depth-4 multilinear circuits over finite Fields: We present a deterministic algorithm for reconstructing multilinear $\Spsp(k)$ circuits, i.e. multilinear depth-$4$ circuits with fan-in $k$ at the top $+$ gate. For any fixed $k$, given black-box access to a polynomial $f \in \Fn$ computable by a multilinear $\Spsp(k)$ circuit of size $s$, the algorithm runs in time quasi-poly($n,s,\size{\F})$ and outputs a multilinear $\Spsp(k)$ circuit of size quasi-poly($n,s$) that computes $f$.4. Average-case learning for generalized depth-3 circuits: We design a (randomized) learning algorithm for \textit{generalized} depth-3 circuits. A circuit in this class is an expression of the type, $g_1(\ell_{11}, \ldots, \ell_{1m}) + \cdots + g_s(\ell_{s1}, \ldots, \ell_{sm}),$ where $g_i$'s are $m$-variate degree $d$ homogeneous polynomials and $\ell_{ij}$'s are linear forms in the variables $\vecx = (x_1,\ldots, x_n)$. Our work is in line with the ideas presented in \cite{KayalSaha19, GargKayalSaha20} where they use lower bound methods in arithmetic complexity to design average-case learning algorithms. Location: Via Zoom Committee: Shubhangi Saraf (chair) Eric Allender Sepehr Assadi Zeev Dvir (external) Start Date: 22 Apr 2022; Start Time: 04:00PM - 05:00PM Title: Instance Dependent Sample Complexity Bounds for Interactive Learning Bio: Kevin Jamieson is an Assistant Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington and is the Guestrin Endowed Professor in Artificial Intelligence and Machine Learning. He received his B.S. in 2009 from the University of Washington, his M.S. in 2010 from Columbia University, and his Ph.D. in 2015 from the University of Wisconsin - Madison under the advisement of Robert Nowak, all in electrical engineering. He returned to the University of Washington as faculty in 2017 after a postdoc with Benjamin Recht at the University of California, Berkeley. Jamieson's work has been recognized by an NSF CAREER award and Amazon Faculty Research award. His research explores how to leverage already-collected data to inform what future measurements to make next, in a closed loop. The work ranges from theory to practical algorithms with guarantees to open-source machine learning systems and has been adopted in a range of applications, including measuring human perception in psychology studies, adaptive A/B/n testing in dynamic web-environments, numerical optimization, and efficient tuning of hyperparameters for deep neural networks. Speaker: Abstract: The sample complexity of an interactive learning problem, such as multi-armed bandits or reinforcement learning, is the number of interactions with nature required to output an answer (e.g., a recommended arm or policy) that is approximately close to optimal with high probability. While minimax guarantees can be useful rules of thumb to gauge the difficulty of a problem class, algorithms optimized for this worst-case metric often fail to adapt to “easy” instances where fewer samples suffice. In this talk, I will highlight some my group’s work on algorithms that obtain optimal, finite time, instance dependent sample complexities that scale with the true difficulty of the particular instance, versus just the worst-case. In particular, I will describe a unifying experimental design based approach used to obtain such algorithms for best-arm identification for linear bandits, contextual bandits with arbitrary policy classes, and smooth losses for linear dynamical systems. Location: Via Zoom Committee: Start Date: 28 Apr 2022; Start Time: 04:00PM - Title: The Human in Human-Robot Interaction Bio: Dr. Matthew Gombolay is an Assistant Professor of Interactive Computing at the Georgia Institute of Technology. He received a B.S. in Mechanical Engineering from the Johns Hopkins University in 2011, an S.M. in Aeronautics and Astronautics from MIT in 2013, and a Ph.D. in Autonomous Systems from MIT in 2017. Gombolay’s research interests span robotics, AI/ML, human-robot interaction, and operations research. Between defending his dissertation and joining the faculty at Georgia Tech, Dr. Gombolay served as technical staff at MIT Lincoln Laboratory, transitioning his research to the U.S. Navy and earning an R&D 100 Award. His publication record includes a best paper award from the ACM/IEEE International Conference on Human-Robot Interaction, and the American Institute for Aeronautics and Astronautics, a finalist for best paper at the Conference on Robot Learning, and a finalist for best student paper at the American Controls Conference. Dr Gombolay was selected as a DARPA Riser in 2018, received 1st place for the Early Career Award from the National Fire Control Symposium, and was awarded a NASA Early Career Fellowship.                 This talk is organized by the Rutgers SOCRATES project. Speaker: Abstract: New advances in robotics and autonomy offer a promise of revitalizing final assembly manufacturing, assisting in personalized at-home healthcare, and even scaling the power of earth-bound scientists for robotic space exploration. Yet, in real-world applications, autonomy is often run in the O-F-F mode because researchers fail to understand the human in human-in-the-loop systems. In this talk, I will share exciting research we are conducting at the nexus of human factors engineering and cognitive robotics to inform the design of human-autonomy interaction. In my talk, I will focus on our recent work (1) democratizing robot learning by formulating better models of heterogeneous and suboptimal human teachers; (2) explaining to those humans what the robot has learned to facilitate shared mental models; and (3) scaling the human-robot interaction to team-level coordination with graph neural networks. The goal of this research is to inform the design of autonomous teammates so that users want to turn – and benefit from turning – autonomy to the O-N mode. Location: Via Zoom Committee: Start Date: 29 Apr 2022; Start Time: 04:00PM - 05:00PM Title: Explaining the Decisions of AI Systems Bio: Adnan Darwiche is a professor and former chairman of the computer science department at UCLA. He directs the Automated Reasoning Group, which focuses on symbolic reasoning, probabilistic reasoning and their applications to machine learning. Professor Darwiche is Fellow of AAAI and ACM and recipient of the Lockheed Martin Excellence in Teaching Award. He is a former editor-in-chief of the Journal of Artificial Intelligence Research (JAIR) and author of "Modeling and Reasoning with Bayesian Networks," by Cambridge University Press.  Presented in association with the DATA-INSPIRE TRIPODS Institute. Speaker: Abstract: I will present a theory for reasoning about the decisions made by AI systems, particularly classifiers such as decision trees, random forests, Bayesian networks and some limited types of neural networks. The theory is based on “compiling" the input-output behavior of classifiers into discrete functions in the form of tractable circuits. At the heart of the theory is the notion of “complete reason” behind a decision which is extracted from a circuit-instance pair and can be used to answer many queries about the decision, including ones pertaining to explainability, robustness and bias. I will also overview developments on tractable circuits which provide the computational arm for employing this theory in practice and will briefly overview recent results on quantified Boolean logic which provide classifier-independent semantics of this theory that further broadens its applicability. Location: Presented via Zoom Committee: Start Date: 02 May 2022; Start Time: 02:00PM - 04:00PM Title: Beyond Instance-level Reasoning in Object Pose Estimation and Tracking for Robotic Manipulation Bio: Speaker: Abstract: This thesis deals with object pose estimation and tracking, and solve robot manipulation tasks. It aims to address uncertainty due to dynamics and generalize to novel object instances by reducing the dependency on either instance or category level 3D models. Robot object manipulation often requires reasoning about object poses given visual data. For instance, pose estimation can be used to initiate pick-and-drop manipulation and has been studied extensively. Purposeful manipulation, however, such as precise assembly or withinhand re-orientation, requires sustained reasoning of an object's state, since dynamic effects due to contacts and slippage, may alter the relative configuration between the object and the robotic hand. This motivates the temporal tracking of object poses over image sequences, which reduces computational latency, while maintaining or even enhancing pose quality relative to single-shot pose estimation. Most existing techniques in this domain assume instance-level 3D models. This complicates generalization to novel, unseen instances, and thus hinders deployment to novel environments. Even if instance-level 3D models are unavailable, however, it may be possible to access category-level models. Thus, it is desirable to learn category-level priors, which can be used for the visual understanding of novel, unknown object instances. In the most general case, where the robot has to deal with out-of-distribution instances or it cannot access category-level priors, object-agnostic perception methods are needed. Given this context, this thesis proposes a category-level representation, called NUNOCS, to unify the representation of various intra-class object instances and facilitate the transfer of category-level knowledge across such instances. This work also integrates the strengths of both modern deep learning as well as pose graph optimization to achieve generalizable object tracking in the SE(3) space, without needing either instance or category level 3D models. When instance-level object models are available, a synthetic data generation pipeline is developed to learn the relative motion along manifolds by reasoning over image residuals. This allows to achieve state-of-art SE(3) pose tracking results, while circumventing manual efforts in data collection or annotation. It also demonstrates that the developed solutions for object tracking provide efficient solutions to multiple manipulation challenges. Specifically, this thesis starts from a single-image object pose estimation approach that deals with severe occlusions during manipulation. It then moves to long-term object pose tracking via reasoning over image residuals between consecutive frames, while training exclusively over synthetic data. In the case of object tracking along a video sequence, the dependency on either instance-level or category-level CAD models is reduced via leveraging multi-view consistency, in the form of a memory-augmented pose graph optimization, to achieve spatial-temporal consistency. For initializing pose estimates in video sequences involving novel unseen objects, category-level priors are extracted by taking advantage of easily accessible virtual 3D model databases. Following these ideas, frameworks for category-level, task-relevant grasping, and vision-based, closed-loop manipulation are developed, which resolve complicated and high precision tasks. The learning process is scalable as the training is performed exclusively over synthetic data or through a robot's self-interaction process conducted solely in simulation. The proposed methods are evaluated first over public computer vision benchmarks, boosting the previous state-of-art tracking accuracy from 33.3% to 87.4% on the NOCS dataset, despite reducing dependency on category-level 3D models for training. When applied to real robotic setups, they significantly improve category-level manipulation performance, validating their effectiveness and robustness. In addition, this thesis unlocks and demonstrates multiple complex manipulation skills in open world environments. This is despite limited input assumptions, such as training solely over synthetic data, dealing with novel unknown objects, or learning from a single visual demonstration. Location: 1 Spring St, New Brunswick, NJ 08901 Rm 319 Committee: Kostas Bekris (advisor) Abdeslam Boularias Dimitris Metaxas Shuran Song (Columbia University - external) Start Date: 05 May 2022; Start Time: 10:00AM - 11:30AM Title: Characterizations of Experts Algorithms That Are Incentive Compatible Bio: Speaker: Abstract: Experts algorithms are online learning algorithms that combine the predictions of multiple experts to produce an aggregate prediction. These algorithms have strong accuracy guarantees: the loss of the aggregate is at most a small amount more than the loss of the best expert. Most analyses assume that the experts volunteer their best information completely and truthfully. Experts are treated like thermometers reporting temperatures, without any agency of their own. Recently, authors have explored new algorithms that take the experts’ motivations into account. They treat experts as rational, not altruistic: experts maximize their own influence on the algorithm and thus their own reputations as forecasters. In this paper, we look at many standard expert algorithms and ask what happens if experts act strategically. We prove several sufficient conditions under which standard expert algorithms are incentive compatible, meaning that the expert’s best strategy is to report their true belief about the realization of each event. Some conditions lead to incentive compatibility in the limit of the number of experts, while other more specific conditions produce incentive compatibility regardless of the number of experts and across all external parameters on the expert algorithms. We illustrate our main results through simulation. Location: Via Zoom Committee: David Pennock (advisor) Sepehr Assadi Yongfeng Zhang Desheng Zhang Start Date: 09 May 2022; Start Time: 04:00PM - Title: Improved Bounds for Distributed Load Balancing Bio: Speaker: Abstract: We consider the problem of load balancing in the distributed setting. The input is a bipartite graph of clients and servers where each client comes with a positive weight. The goal is to assign each client to one of its adjacent servers minimizing the weighted sum of clients assigned to any one server. This problem has a variety of applications and has been widely studied under several different names, including scheduling with restricted assignment, semi-matching, and distributed backup placement. We give the first LOCAL algorithm to compute an O(1)-approximation to the load balancing problem in polylog(n) rounds. In the CONGEST model, we give an O(1)-approximation algorithm in polylog(n) rounds when the client weights are uniform; for general weights, the approximation ratio is O(log(n)). Based on joint work with Sepehr Assadi and Aaron Bernstein. Location: Via Zoom Committee: Sepehr Assadi (Advisor) Aaron Bernstein Martin Farach-Colton Vladimir Pavlovic Start Date: 12 May 2022; Start Time: 02:30PM - 04:30PM Title: Interleaving Learning and Search for Solving Long-Horizon Episodic Robotic Planning Tasks Bio: Speaker: Abstract: Robots need to compute high-quality solutions or plans quickly to provide meaningful assistance in solving household and industrial tasks. However, computing high-quality, long-horizon plans are rather challenging. In my research, I attempt to harness the power of deep learning, reinforcement learning, and Monte Carlo tree search to tackle long-horizon robot planning tasks. This presentation will discuss our proposed solutions, employing the tools mentioned earlier that effectively solve two long-horizon robot manipulation problems: de-clutter and object retrieval. Location: Virtual Committee: Proffesor Jinjin Yu (Advisor) Professor Abdeslam Boularias Professor Kostas Bekris Professor Karthik Srikanta Start Date: 18 May 2022; Start Time: 10:00AM - Title: Machine Learning and Understanding Art Bio: Speaker: Abstract: Recent advances in machine learning on various computer vision tasks have shown the great potential for developing an AI system for art through the successful applications in prediction of style, genre, medium, attribution, school of art, etc. Beyond the categorical information, in this dissertation, a more fundamental level of artistic knowledge is pursued by developing machine learning systems for art principles. The art principles are to know how art is visually formed and identify what content in it and what symbolic meaning it has. The task necessitates fine-grained semantics to describe art, but art data does not generally accommodate the fine details. The scarcity of data annotation is a primary challenge in this machine learning study. Three research problems are explored; (1) first is to find principal semantics for style recognition. (2) second is to lay the groundwork for computational iconography, i.e. recognize content and discover the co-occurrence and visual similarities among the content in fine art paintings. (3) third is to quantify paintings with finite visual semantics from style through language models. In the system design, well-established knowledge and facts in art theory are leveraged, or general knowledge of art is integrated into the hierarchical architecture of deep-CNN as a numerical form after learning it from a corpus of art-texts through contemporary language models in Natural Language Processing. Language modeling is a practical and scalable solution requiring no direct annotation, but it is inevitably imperfect. This dissertation shows how deep learning's hierarchical structure and adaptive nature can create a stronger resilience to the incompleteness of the practical solution than other related methods. Location: Virtual Committee: Professor Ahmed Elgammal (Advisor) Professor Konstantinos Michmizos Professor Karl Stratos Professor Gennaro Vessio (University of Bari, Italy) Start Date: 24 May 2022; Start Time: 10:00AM - 11:30AM Title: Towards Fairer Recommender Systems through Deep Reinforcement Learning Bio: Speaker: Abstract: The issue of fairness in recommendation is becoming increasingly essential as Recommender Systems (RS) touch and influence more and more people in their daily lives. While most of the existing fairness-aware recommendation approaches do alleviate the impact of unfair recommendations, they also have two major drawbacks. First, most of them have been situated in a static or one-shot setting, where the protected groups of items are fixed, and the model provides a one-time fairness solution based on fairness-constrained optimization, which fails to consider the dynamic nature of the recommender systems, where attributes such as item popularity may change over time due to the recommendation policy and user engagement. Second, they mainly aim at solving a constrained optimization problem by imposing a constraint on the level of fairness while optimizing the main recommendation objective, which may significantly compromise the recommendation accuracy due to the inherent trade-off between fairness and utility. Regarding to these questions, we propose two fairness-aware recommendation frameworks leveraging the advantages of deep reinforcement learning to overcome the above drawbacks correspondingly. For the first, we tackle it by proposing a fairness-constrained reinforcement learning algorithm for recommendation, which models the recommendation problem as a Constrained Markov Decision Process (CMDP), so that the model can dynamically adjust its recommendation policy to make sure the fairness requirement is always satisfied when the environment changes. For the second, we propose a fairness-aware recommendation framework using multi-objective reinforcement learning (MORL), called MoFIR (pronounced “more fair”), which is able to learn a single parametric representation for optimal recommendation policies over the space of all possible preferences between fairness and utility. Location: Virtual Committee: Professor Yongfeng Zhang (Advisor) Professor Desheng Zhang Professor Jie Gao Professor He Zhu Start Date: 25 May 2022; Start Time: 10:00AM - 11:30PM Title: Counterfactual Explainable AI Bio: Speaker: Abstract: By providing explanations for users and AI system designers to facilitate better decision making and system understanding, explainable AI has been an important research problem. It is challenging because the most state-of-the-art machine learning models, which utilize deep neural networks, are non-transparent. In my research on Counterfactual Explainable AI, we take insights of counterfactual reasoning from causal inference to address this challenge. We first mathematically formulate the complexity and strength of explanations, and then propose a general model-agnostic counterfactual learning framework to seek simple (low complexity) and effective (high strength) explanations for the model decision. More specifically, counterfactual reasoning asks, “if A did not happen, will B happen?”. When applied in the machine learning field, it looks for a minimal change on the input such that the prediction will be different. Therefore, the changed factors are crucial for the original prediction made by the system, which constitute the counterfactual explanations. We formulate this idea as a machine learning optimization problem and generate faithful explanations. Meanwhile, we also design metrics based on counterfactual reasoning to quantitatively evaluate the necessity and sufficiency of the explanations. They are proved to be suitable for evaluating generated explanations in explainable AI, which is a challenging task in this field. We conduct a comprehensive set of experiments for different machine learning applications, such as recommendations and graph-based drug mutagenicity predictions, to show the effectiveness of the proposed counterfactual-based explainable models as well as the evaluation metrics. Location: Virtual Committee: Professor Yongfeng Zhang (Advisor) Professor Desheng Zhang Professor Dong Deng Professor Jie Gao Start Date: 25 May 2022; Start Time: 01:00PM - 03:00PM Title: Brain-informed Deep Learning of Human Movements with Neurophysiological Interpretations Bio: Speaker: Abstract: The accurate and reliable decoding of movement from non-invasive electroencephalography (EEG) is essential for informing therapeutic interventions ranging from neurorehabilitation robots to neural prosthetics. However, the main caveats of EEG, namely its low spatial resolution and ill-defined source localization, hinder its reliable decoding, despite progress in both statistical and the recent machine learning methods. In this talk, I will present our brain-informed deep learning solutions for the accurate and reliable decoding of movements from EEG with neurophysiological interpretations. First, I will present a 3-dimensional convolutional neural network (3D-CNN) that, when designed and trained upon the brain's constraints, can decode EEG to predict the movement primitives, namely the reaction time (RT), movement intent, and the direction of the movement. When validated on data acquired from in-house IRB-approved motor experiments, our proposed method outperforms the state-of-the-art deep networks by up to 6.74%. Next, I will show how the deep learning framework can be extended to assess the cognitive engagement (CE) of the subjects when performing a rehabilitation task. Specifically, our method uses i) a deep network that predicts the level of CE for two classes- cognitively engaged vs. disengaged with 88.13% accuracy; and ii) a novel sliding window method that predicts continuous levels of CE in real-time. Next, to widen the application domain of our method to portable brain-computer interfaces, I will present an energy-efficient neuromorphic solution to EEG decoding. Our neuromorphic solution, where a trained spiking neural network (SNN) is deployed on Intel's Loihi neuromorphic chip, achieves the same level of performance as the deep neural networks (DNN) while consuming 95% less energy per inference. Lastly, to guarantee the reliability of our solutions in real-world applications, I will present interpretation techniques that establish correspondence between the features learned by the artificial networks and the underlying neurophysiology. Overall, our approach demonstrates the importance of biological relevance in neural networks for accurate and reliable decoding of EEG, suggesting that the real-time classification of other complex brain activities may now be within our reach. Location: Via Zoom Committee: Dr. Konstantinos Michmizos (Chair) Dr. Dimitris N. Metaxas Dr. Vladimir Pavlovic Dr. Georgios D. Mitsis (McGill University) Start Date: 26 May 2022; Start Time: 03:00PM - Title: Biologically Inspired Spiking Neural Networks for Energy-Efficient Robot Learning and Control Bio: Speaker: Abstract: : Energy-efficient learning and control are becoming increasingly crucial for robots that solve complex real-world tasks with limited onboard resources. Although deep neural networks (DNN) have been successfully applied to robotics, their high energy consumption limits their use in low-power edge applications. Biologically inspired spiking neural networks (SNN), facilitated by the advances in neuromorphic processors, have started to deliver energy-efficient, massively parallel, and low-latency solutions to robotics. In this defense, I will present our energy-efficient neuromorphic solutions to robot navigation, control, and learning, using SNNs on Intel's Loihi neuromorphic processor. First, I will present a biologically constrained SNN, mimicking the brain's spatial system, solving the unidimensional SLAM problem while only consuming 1% of energy compared with the conventional filter-based approach. In addition, when extending the model to 2D environments by adding biologically realistic hippocampal neurons, the SNN formed cognitive maps in real-time and helped study the neuronal interconnectivity and cognitive functions. Next, I will show how the neuromorphic approach can be extended to high-level cognitive functions such as learning control policies. Specifically, I will present a reinforcement co-learning framework that jointly trains a spiking actor network (SAN) with a deep critic network using backpropagation to learn optimal policies for both mapless navigation and high-dimensional continuous control. Compared with state-of-the-art DNN approaches, our method results in up to 140 times less energy consumption during inference, while generating a superior successful rate on mapless navigation, and achieves the same level of performance on high-dimensional continuous control when using the population-coded spiking actor network (PopSAN). Lastly, I will present how these energy gains can further be extended to training through the development of a biologically plausible gradient-based learning framework on the neuromorphic processor. The learning method is functionally equivalent to the spatiotemporal backpropagation but solely relies on spike-based communication, local information processing, and rapid online computation, which are the main neuromorphic principles that mimic the brain. Overall, our work pushes the frontiers of SNN applications to energy-efficient robotic control and learning, and hence paves the way toward the introduction of a biologically inspired alternative solution for autonomous robots running on energy-efficient neuromorphic processors. Location: Via Zoom Committee: Konstantinos Michmizos (Advisor) Vladimir Pavlovic, Abdeslam Boularias, James Bradley Aimone (Sandia National Lab) Start Date: 06 Jun 2022; Start Time: 03:00PM - 04:30PM Title: Online List Labeling: Breaking the log2n Barrier Bio: Speaker: Abstract: The list labeling problem is a classical combinatorial problem with many algorithmic applications. There has long existed a gap between the lower and upper bounds in the most algorithmically interesting part of the problem's parameter space. We present our recent results, which narrow this gap for the first time in nearly 4 decades. Location: Virtual Committee: Professor Martin Farach-Colton Professor Sepehr Assadi Professor Aaron Bernstein Professor Qiong Zhang Start Date: 10 Jun 2022; Start Time: 11:00AM - Title: Near-Optimal Scalable Multi-Robot Path Planning : Algorithms, and Applications Bio: Speaker: Abstract: In Multi-Robot Path Planning (MRPP), given a graph and n robots with assigned starts and goals, we need to find a collision-free path for each robot connecting the start vertex and the goal vertex. It has many applications in video games, warehouse automation, formation control, and swarm robotics. Solving MRPP optimally in terms of makespan or sum-of-costs is NP-hard. Therefore, research on near-optimal solvers that have good scalability and acceptable suboptimality is an attractive topic. In this talk, we first propose a polynomial time algorithm with asymptotic expected sub-1.5 makespan optimality guarantee, which utilizes a novel Rubik Table result. The method scales up to 30,000+ robots on 300x300 grids. In addition, we extend the algorithm to three dimensional grids and apply it on large-scale drone swarms in both simulation and real drone experiments. Then we introduce two divide-and-conquer based heuristics that enhance the scalability of classical MRPP solvers while keeping 1.x optimality. Location: Virtual Committee: Professor Jingjin Yu Professor Kostas Bekris Professor Abdeslam Boularias Professor Peng Zhang Start Date: 14 Jun 2022; Start Time: 11:00AM - Title: Robot Imagination: Affordance-Based Reasoning about Unknown Objects Bio: Gregory S. Chirikjian received undergraduate degrees from Johns Hopkins University in 1988, and a Ph.D. degree from the California Institute of Technology, Pasadena, in 1992. From 1992 until 2021, he served on the faculty of the Department of Mechanical Engineering at Johns Hopkins University, attaining the rank of full professor in 2001. Additionally, from 2004-2007, he served as department chair. Starting in January 2019, he moved to the National University of Singapore, where he is serving as Head of the Mechanical Engineering Department, where he has hired 14 new professors so far. Chirikjian’s research interests include robotics, applications of group theory in a variety of engineering disciplines, applied mathematics, and the mechanics of biological macromolecules. He is a 1993 National Science Foundation Young Investigator, a 1994 Presidential Faculty Fellow, and a 1996 recipient of the ASME Pi Tau Sigma Gold Medal. In 2008, Chirikjian became a fellow of the ASME, and in 2010, he became a fellow of the IEEE. From 2014-15, he served as a program director for the US National Robotics Initiative, which included responsibilities in the Robust Intelligence cluster in the Information and Intelligent Systems Division of CISE at NSF. Chirikjian is the author of more than 250 journal and conference papers and the primary author of three books, including Engineering Applications of Noncommutative Harmonic Analysis (2001) and Stochastic Models, Information Theory, and Lie Groups, Vols. 1+2. (2009, 2011). In 2016, an expanded edition of his 2001 book was published as a Dover book under a new title, Harmonic Analysis for Engineers and Applied Scientists. Speaker: Abstract: Today’s robots are very brittle in their intelligence. This follows from a legacy of industrial robotics where robots pick and place known parts repetitively. For humanoid robots to function as servants in the home and in hospitals they will need to demonstrate higher intelligence, and must be able to function in ways that go beyond the stiff prescribed programming of their industrial counterparts. A new approach to service robotics is discussed here. The affordances of common objects such as chairs, cups, etc., are defined in advance. When a new object is encountered, it is scanned and a virtual version is put into a simulation wherein the robot imagines’’ how the object can be used. In this way, robots can reason about objects that they have not encountered before, and for which they have no training using. Videos of physical demonstrations will illustrate this paradigm, which the presenter has developed with his students Hongtao Wu, Meng Xin, Sipu Ruan, and others. Location: 1 Spring Street, Room 403 Committee: Start Date: 23 Jun 2022; Start Time: 12:00PM - Title: Semantic Modeling, Integration and Episodic Organization of Personal Digital Traces Bio: Speaker: Abstract: Memory plays a fundamental role in life and is critical to our everyday functioning. We use memories to maintain our personal identity, to support our relationships, to learn, and to solve problems. Today, a vast amount of tools supports digital capture of different aspects of people's lives. These tools produce a multitude of data objects, which we call Personal Digital Traces - PDTs, which can be used to help reconstruct people’s episodic memories and connect to their past personal events. This reconstruction may have several applications, from helping the recall of patients with neurodegenerative diseases to helping people remember past events and better manage their data and time if this information is used in activity-centric applications, like personal assistants.This dissertation takes steps towards supporting autobiographical memory by associating heterogeneous PDTs according to their higher-level purposes and usages and summarizing them into episodic narratives. We start by presenting a unified and intuitive conceptual modeling language whose novel features include the properties "who, what, when, where, why, how" applied uniformly to both personal digital traces and their corresponding atomic events/tasks that produce them. We then proceed by describing an ontology for prototypical higher level plans ("scripts") for common everyday events and show how families of related scripts can be defined by incrementally modifying more general scripts through inheritance. We then present a multiplayer web-based game called OneOfUs, which creates possible associated digital trace descriptions that can be produced by each of these activities/scripts through crowdsourcing techniques that act as evidence for the execution of such scripts. The game is able to automatically validate and assess knowledge at the time of the game, as well as dynamically learn new pieces of information, as it has a way of not neglecting uncommon answers through players' votes. For instantiating those scripts based on the lower level actions of which scripts are composed of, we present a bottom up merging algorithm that groups and relates several digital traces from many different sources into script instances (episodes) as well as a software architecture that supports systematic and declarative specification of evidence. This also utilizes a scoring scheme to account for the varied strength of evidence provided by PDTs or script steps.Finally, to evaluate the efficacy of our methodology, we designed and implemented YourDigitalSelf, an Android mobile device application that gathers and integrates personal digital traces into narratives. A thorough evaluation performed over real user's data collections shows that our approach is able to integrate and combine successfully different traces from different popular sources into coherent episodes/activities. In addition, we show evidence that our approach does augment user's memory of their past actions, thereby forms a powerful retrospective memory aid. Location: Virtual Committee: Prof. Amelie Marian (Chair) Prof. Alexander Borgida Prof. Gerard De Melo Prof. Chirag Shah, University of Washington Start Date: 12 Jul 2022; Start Time: 10:00AM - Title: HARDWARE-SOFTWARE TECHNIQUES FOR ACCELERATING SPARSE COMPUTATION Bio: Speaker: Abstract: Linear algebra kernels are widely used in various fields such as machine learning, data science, physical science, and graph analysis. Many of these applications work with sparse data (i.e., only a small fraction of data is non-zero). Sparse data are often stored in a compressed format (i.e., sparse format) that stores only the non-zero elements with additional metadata to identify where the non-zero elements are located. Using compressed formats eliminates the need to store and process zeros, making the storage and computation of sparse kernels more efficient.General purpose architectures, such as CPUs and GPUs, are not able to deliver the same performance for sparse linear algebra kernels as they do for dense versions. First, accessing non-zero elements in sparse format introduces many indirect and irregular memory accesses incompatible with SIMD and caching mechanisms used by CPUs and GPUs. In addition, Dennard scaling is obsolete and Moore's law is slowing down, ending the era in which general-purpose architectures become faster and more energy efficient transparently. This has led to a plethora of research into developing specialized hardware, such as FPGAs and ASICs to improve the performance and energy efficiency of these sparse kernels. A key strategy for the specialized hardware is to customize the sparse format (i.e., storage) according to the operation memory access pattern, the pattern of non-zero elements in the input (i.e., sparsity pattern), and the underlying hardware structures. This approach is effective if the operations and input sparsity patterns do not change. However, applications often perform various operations on sparse data. Additionally, the sparse inputs may frequently change for each execution, and each input may have a different sparsity pattern. When this happens, the performance of specialized hardware degrades because a reformatting step is required to convert the data into a format that is compatible with the hardware. The data reformatting can be expensive when it cannot be overlapped with the computation on the hardware or amortized over multiple application executions with the same input data. This dissertation presents a few hardware-software techniques that enhance the performance and energy efficiency of some of the most important sparse problems, including sparse matrix-vector multiplication (SpMV), sparse general matrix-matrix multiplication (SpGEMM), and sparse convolutional neural networks (CNNs). The key insight of our method is to use the software to reformat the sparse data into a hardware-friendly format, allowing the hardware to perform the computation with a high degree of parallelism. The software improves design flexibility by supporting multiple sparse formats, and the hardware improves performance and energy efficiency. We applied these hardware-software techniques to SpMV, SpGEMM, and sparse CNNs. These problems have different characteristics, such as different input densities and distinct input sparsity pattern features. The contribution of this dissertation can be summarized as follows. First, we present a synergistic CPU-FPGA system to accelerate SpMV and SpGEMM kernels. In our proposed design, the CPU reorganizes sparse data into a format suitable for the FPGA, and the FPGA computes with high parallelism using the preprocessed data. We develop an intermediate representation that allows the software to communicate regularized data and scheduling decisions to the FPGA. Besides, most of the CPU and FPGA execution are overlapped. Our approach can effectively handle sparse kernels with low input densities and sparsity patterns varying for each sparse input. Second, we present a hardware accelerator for sparse CNN inference tasks. We formulate the convolution operation as general matrix-matrix multiplication (GEMM) using an image to column (IM2COL) transformation. With a dynamically reconfigurable GEMM and a novel IM2COL hardware unit, our design can support various layers in CNNs with high performance. Besides, our design exploits sparsity in both weights and feature maps. We use the software to perform group-wise pruning followed by a preprocessing step that puts the pruned weights into our hardware-friendly sparse format for efficient and high performance computation. We built an ASIC and an FPGA prototype of our accelerator. Our design is faster and more energy efficient than CPUs and GPUs for sparse CNNs. Location: Virtual Committee: Professor Santosh Nagarakatte (Chair) Professor Richard P. Martin Professor Yipeng Huang Professor Joe Devietti Start Date: 14 Jul 2022; Start Time: 02:00PM - Title: Application of Deep Neural Networks to Automated Analysis of Hatching Lines for Artist Identification Bio: Speaker: Abstract: Automated analysis of art has gained interest in the computer science research community over the past decade, resulting in the onset of computer vision solutions for authentication and attribution of art. Attribution and authentication are two of the most crucial tasks in the domain of art. Several studies have approached these tasks with an emphasis on analyzing paintings in the visual spectrum; however, with a limited scope to a specific artist, limited datasets, or without rigorous evaluations of the robustness of such approaches. Contrary to paintings, the application of computer vision methodologies for authentication and attribution of drawings and prints has been only sparsely explored. In this dissertation, we introduce a novel approach for attribution and authentication of drawings and sketches at the visual spectrum by automated analysis of hatching lines. Hatching is a prevalent technique in drawing and printmaking, often used to introduce toning, shading, or illusion of light and volume. Since artists tend to add hatching lines spontaneously, we hypothesize that these areas of drawings could carry unique physical and unconscious characteristics of the artist, which can be used to identify the artist. We further hypothesize that such unique characteristics can be quantified by a computational model that will enable automatic attribution and authentication. We investigated the application of deep convolutional neural networks for detecting hatching lines and identifying artists based on their hatching characteristics. We conducted several experiments on different sets of drawings and prints by different artists from different eras, using different techniques and with various degrees of complexity. We concluded that we can indeed identify artists based solely on their hatching with an accuracy of 90-100% in most cases. Location: Virtual Committee: Professor Ahmed Elgammal (Chair) Professor Casimir Kulikowski Professor Abdeslam Boularias Professor Lior Shamir (Kansas State University) Start Date: 15 Jul 2022; Start Time: 10:00AM - 12:00PM Title: Non-neuronal Computational Principles for Increased Performance in Brain-inspired Networks Bio: Speaker: Abstract: Understanding the brain’s computational principles holds potential to lead to human-level machine learning and intelligence. Yet, current brain-inspired spiking neural network (SNN) methods assume that only neurons form the basis of computation in the brain, which contradicts increasing biological evidence associating the function of ubiquitous non-neuronal cells, known as astrocytes, to brain cognitive states, memory and learning. Interestingly, these information processing functions are closely linked to near-critical neuronal network dynamics, which are believed to underlie the brain’s impressive computational abilities. Inspired by this emerging data, in this talk I will present our efforts to understand the computational role of astrocytes in the brain and to translate our findings into brain-inspired neuron-astrocyte spiking networks. First, I will present our biologically plausible computational modeling approach to propose that astrocytes learn to detect and signal deviations in brain activity away from critical dynamics. Further, we suggest that astrocytes achieve this function by approximating the information content of their neuronal inputs. Translating these findings into a well established brain-inspired machine learning paradigm known as the liquid state machine (LSM), I will next present our proposed neuron-astrocyte liquid state machine (NALSM). Specifically, the NALSM uses a single astrocyte to organize recurrent (liquid) network dynamics at the critical regime where the LSM computational capacity peaks. We demonstrate that the NALSM achieves state-of-the-art LSM accuracy and does not need data-specific hand-tuning as do comparable LSM methods. Further, we demonstrate that the NALSM achieves comparable performance to fully-connected multilayer spiking neural networks trained via backpropagation, with a top accuracy of 97.61% on MNIST, 97.51% on N-MNIST, and 85.84% on Fashion-MNIST. Overall, our dual pronged approach leads to testable biological hypotheses about brain computation, which are then abstracted away and integrated directly into brain-inspired machine learning methods. Location: Virtual Committee: Dr. Konstantinos Michmizos (Chair), Dr. Casimir Kulikowsk Dr. Dimitris Metaxa Dr. Constantinos Siettos (University of Naples Federico II, Italy) Start Date: 19 Jul 2022; Start Time: 10:00AM - Title: FAST METHODS TO DETECT AND DEBUG NUMERICAL ERRORS WITH SHADOW EXECUTION Bio: Speaker: Abstract: The floating-point (FP) representation uses a finite number of bits to approximate real numbers in computer systems. Due to rounding errors, arithmetic using the FP representation may diverge from arithmetic with real numbers. For primitive FP operations, the rounding error is small and bounded. However, with the sequence of FP operations, rounding errors can accumulate. Such rounding errors can be magnified by certain operations. The magnified error can affect the program's control flow and the output compared to an execution with infinite bits of precision. It is challenging for users to identify such bugs, as in such cases the program does not crash but generates the incorrect output. Without any oracle in high precision, it is hard to differentiate between correct and incorrect output. Detecting such bugs in long-running programs becomes even more challenging. This dissertation proposes a fast, yet precise mechanism to detect and debug numerical errors in long-running programs. This dissertation makes the following contributions. First, we propose a selective shadow execution framework to detect and debug numerical errors. Our idea is to use shadow execution with high-precision computation for comprehensive numerical error detection. The FP variable is shadowed with a high precision value in this approach. On every FP computation, an equivalent high precision computation is performed. If there is a significant difference between FP computation and high precision computation, the error is reported to the user. To debug errors, we maintain the additional information about the instructions. We use additional information to generate the directed acyclic graph (DAG) of instructions showing the error propagation. The DAG of instructions helps the user identify the root cause of the error. Our prototype FPSanitizer for floating-point is an order of magnitude faster than the prior work. Second, we propose a novel technique to run shadow execution in parallel to reduce the overheads further. In our approach, the user specifies parts of the program that need to be debugged. Our compiler creates shadow execution tasks that mirror these specified regions in the original program but perform equivalent high precision computation. To execute the shadow tasks in parallel, we need to break the dependency between them by providing the appropriate memory state and input arguments. Moreover, to correctly detect the numerical errors in the original program, shadow tasks need to follow the same control flow as in the original program. Our key insight is to use FP values computed by the original program to start the shadow task from some arbitrary point. To ensure shadow tasks follow the same control flow as the original program, our compiler updates every branch instruction in the shadow task to use the branch outcomes of the original program. As a result, the original program and shadow tasks execute in a decoupled fashion and communicate via a non-blocking queue. Our prototype PFPSANITIZER is significantly faster than the FPSANITIZER.Finally, we propose an alternative lightweight oracle to reduce the overheads of shadow execution. Although parallel shadow execution effectively reduces overheads, it requires user input. Often, users may not know where the numerical bugs are present. This thesis proposes a fast shadow execution framework, EFTSANITIZER, that uses error-free transformations (EFTs) to detect anddebug numerical bugs comprehensively. EFTs are a set of algorithms that provide a mechanism to capture the rounding error using primitive FP instructions. For certain FP computations rounding error can be represented as an FP value. Based on this observation, EFTs transform FP computation to derive the rounding error. In our approach, we maintain the error with each memory location and compute the error for a sequence of FP operations using error composition with EFTs. EFTSANITIZER provides a trace of instructions that help users isolate and debug the root cause of the errors. In addition, EFTSANITIZER is an order of magnitude faster than FPSANITIZER. Location: Virtual Committee: Professor Santosh Nagarakatte (Chair) Professor Richard Martin  Professor Mridul Aanjaneya  Professor Sreepathi Pai (University of Rochester) Start Date: 08 Sep 2022; Start Time: 01:00PM - Title: Generative Modeling for Multimodal Data Fusion Bio: Speaker: Abstract: Humans can process multiple perspectives of the world and generate compound information from them to obtain a richer understanding of the world.In machine learning, generative models have been widely used to comprehend the world by learning the latent representation of the given data. Representation learning is a key step in the process of data understanding, where the goal is to distill interpretable factors associated with the data. However, representation learning approaches typically focus on data observed in a single modality, such as text, images, or video. In this dissertation, we develop generative models that can fuse multimodal data in order to have a comprehensive understanding of the tasks. First, we introduce a model that can learn the latent representation given multi-modality data based on the VAE framework. Specifically, we disentangle the latent space into modality-common (shared) and modality-specific (private) spaces. By considering the private latent factors aside from the shared latent factors of all modalities, the proposed model can achieve more precise cross-modal generation or retrieving from one modality to another modality. Moreover, by assuming the missing modality, we demonstrate that our model can be utilized for solving semi-supervised learning or zero-shot learning problems. We further study the importance of perceiving the world from multiple views through trajectory forecasting scenarios. The growing demand for autonomous vehicles is spurring numerous studies of behavior prediction. The existing works underestimate the context such as nearby environment and neighbors while predicting the target's future trajectories. We suggest combining such information, and generate the context-aware future trajectory prediction conditioned on the past trajectory based on the Conditional VAE. We show that the proposed model is able to achieve collision-free predictions from the surrounding environment or neighbors in both real and simulated data. Location: Virtual Committee: Professor Vladimir Pavlovic, Chair Professor Mubbasir Kapadia Professor Dimitris Metaxas Professor Kris Kitani (Carnege Mellon University) Start Date: 12 Sep 2022; Start Time: 10:00AM - 12:00PM Title: Learning Compositional Robotic Manipulation Tasks from Unlabeled Data Bio: Speaker: Abstract: Researchers have been seeking intelligent robotic systems that can accomplish complex tasks autonomously with very little human effort. With the recent progress from both planning algorithms and learning based methods, many low-level primitives can be fulfilled with remarkable quality. This thesis aims at the next step, compositional manipulation tasks. A compositional manipulation task consists of multiple sub-tasks and each sub-tasks can be completed by low-level manipulation, it requires a high-level reasoning to schedule sub-tasks. In order to reduce labeling effort, we concentrate on self-supervised learning methods. To be specific, we study reinforcement learning algorithms and imitation learning algorithms. Reinforcement learning algorithms explore in the environment and find optimal behavior based on rewards. As the only feedback is a reward and there is no direct information about sub-tasks provided, we utilize first causality in training predictive model. Another proposed method is to construct a finite-state machine for describing transitions between sub-tasks and include it in the policy input. Another line of works covered by this thesis are imitation learning methods where the robot is given a collection of human demonstrations rather than the reward. These demonstrations always present a complete task rather than sub-tasks, labels on sub-tasks switching are never provided. We propose a decomposition of the policy, and maximization of demonstration trajectory likelihood based on this decomposition learns the sub-tasks switching autonomously. Finally, we investigate generalizing this manipulation algorithm to unseen objects by removing the requirement of semantic labels on objects. The proposed method describes objects by feature vectors including appearance and shape information, so similar objects share similar features. Location: Virtual Committee: Professor Abdeslam Boularias (Chair) Professor Jingjin Yu Professor Mridul Aanjaneya Professor Oliver Kroemer (Carnegie-Mellon University) Start Date: 12 Sep 2022; Start Time: 10:30AM - Title: A Case for Correctly Rounded Elementary Functions Bio: Santosh Nagarakatte is an Associate Professor and Undergraduate Program Director of Computer Science at Rutgers University. He obtained his PhD from the University of Pennsylvania in 2012. His research interests are in Hardware-Software Interfaces spanning Programming Languages, Compilers, Software Engineering, and Computer Architecture.  His group's research has been recognized with the NSF CAREER Award, two IEEE Micro Top Picks Paper Awards (2010 and 2013), five Distinguished Paper Awards (PLDI 2015, ICSE 2016, PLDI 2021, POPL 2022, and CGO 2022), SIGPLAN Research Highlights paper, CACM Research Highlight paper, 2018 ACM SIGPLAN John C Reynolds Outstanding Dissertation Award, Google Research Award, Intel Corporation Gifts, Facebook research award, and 2022 ACM SIGPLAN John C Reynolds Outstanding Dissertation Award.  Speaker: Abstract: This talk will provide an overview of the RLIBM project where we are building a  collection of correctly rounded elementary functions for multiple representations and rounding modes.  Historically, polynomial approximations for elementary functions have been designed by approximating the real value.In contrast, we make a case for approximating the correctly rounded result of an elementary function rather than the real value of an elementary function in the RLIBM project.   Once we approximate the correctly rounded result, there is an interval of real values around the correctly rounded result such that producing a real value in this interval rounds to the correct result. This interval is the freedom that the polynomial approximation has for an input, which is larger than the ones with the mini-max approach.  Using these intervals, we structure the problem of generating polynomial approximations that produce correctly rounded results for all inputs as a linear programming problem. The results from the RLIBM project makes a strong case for mandating correctly rounded results with any representation that has fewer than or equal to 32-bits. Read more about the RLIBM project at   https://people.cs.rutgers.edu/~sn349/rlibm/ Location: CoRE 301 + Virtual Committee: Start Date: 15 Sep 2022; Start Time: 09:00AM - 11:00AM Title: Multi-Dimensional Federated Learning In Recommender Systems Bio: Speaker: Abstract: A wide range of web services like e-commerce, job-searching, and target advertising heavily rely on recommender systems that finds products of interest to fulfills users' diverse and complicated demands. To better model the user preferences and provide satisfactory recommendations, there has been an increasing amount of research focusing on constructing more accurate and complete user representations that exploit the user profile and behavior history. Inevitably, this motion would induce privacy risks for users. This natural conflict between user privacy and recommendation accuracy has drawn lots of attention in recent years. Among these solutions, the most widely studied and verified method is the federated learning techniques. The general idea behind federated learning is maintaining users' critical data on the edge devices (e.g. mobile phones) which communicate only the model parameters to the central server.However, the standard federated learning system is designed for a single task with a simple learning objective. In reality, a user typically interacts with various applications everyday with heterogeneous intentions. In this defense, I will discuss the extended federated learning in a more complex but practical multi-objective setting, where multiple federated learning agents collaborate in an environment with multiple central servers and a number of distributed edge devices. Formally, this machine learning problem is regarded as a multi-objective federated optimization problem, and I identify several main challenges including the dimensional heterogeneity, conflicting objectives, multi-dimensional communication overhead. Then, I will illustrate the general solution framework and illustrate several techniques that could solve the aforementioned challenges. Location: Virtual Committee: Professor Amelie Marian (Chair) Professor  Yongfeng Zhang (Co-advisor) Professor Hao Wang Professor Qingyao Ai (University of Utah) Start Date: 19 Sep 2022; Start Time: 10:00AM - Title: Generative Model and Latent Space Based Medical Image Analysis and Applications Bio: Speaker: Abstract: The generative models have gained much attention in the computer vision community in many cases, such as domain adaptation, data augmentation, and image-to-image translation. These properties have been rapidly applied in some medical domains, but the generative model's potential is not fully explored. This dissertation studies new perspectives on the distributed generative models and latent space manipulation to address some of the most critical medical tasks: medical image private data sharing and cardiac motion analysis. First, we work on asynchronized distributed GAN(AsynDGAN) paradigm to learn the distribution across several private medical data centers and adopt the well-trained generator as a medical data provider for the future use of the downstream tasks. Further, I work on some real scenarios under the continuous learning(Life-long learning) settings with the distributed GAN with temporary discriminators(TDGAN). Such a method could prevent the model from catastrophic forgetting when continuously learning new incoming data. The multi-modality and missing-modality settings are also systematically analyzed. By using a multi-modality adaptive learning model and network(Modality Bank), the Modality Bank could auto-complete the missing modalities and generate multiple modality images simultaneously. We demonstrate that the AsynDGAN-related techniques could secure medical privacy while fully using these private data for machine learning applications. Secondly, the dissertation presents a framework for joint 2D cardiac segmentation and 3D volume reconstruction via a structure-specific generative method(DeepRecon). I will present the end-to-end latent-space-based framework that generates multiple clinically essential outcomes, including accurate image segmentation, synthetic high-resolution 3D image, and 3D reconstructed volume. Experimental results demonstrate the effectiveness of our approach on numerous fronts, including 2D segmentation, 3D reconstruction, and downstream 4D motion pattern adaption performance. And the motion adaptation method provides a unique tool to help cardiologists analyze cardiac motion functional differences between various cases. Overall, the approaches demonstrate the importance of the generative models for the newly emerging medical analysis domains for 3D reconstruction, motion analysis, and privacy data sharing. Location: Virtual Committee: Professor Dimitris N. Metaxas (Chair) Professor Konstantinos P. Michmizos Professor Karl Stratos Professor Xiaolei Huang(Penn State University) Start Date: 19 Sep 2022; Start Time: 10:30AM - Title: Towards Closing the Perception-Planning and Sim2Real Gaps in Robotics Bio: Kostas Bekris is an Associate Professor of Computer Science at Rutgers University in New Jersey. He is working in algorithmic robotics, where his group is developing algorithms for robot planning, learning and perception especially for robot manipulation and multi-robot problems. Applications include logistics and manufacturing with a focus on taking advantage of novel soft, adaptive mechanisms. His research has been supported by NSF, DHS, DOD and NASA, including a NASA Early Career Faculty award. He received his Ph.D in Computer Science from Rice University under the guidance of Prof. Lydia Kavraki. Kostas is Program Chair for the “Robotics: Science and Systems” (RSS) 2023 conference. Speaker: Abstract: Robotics is at the point where we can deploy complete systems across applications, such as logistics, service and field robotics. There are still critical gaps, however, that limit the adaptability, robustness and safety of robots, which lie at: (a) the interface of domains, such as perception, planning and learning, that must be viewed holistically in robotics, and (b) the sim2real gap, i.e., the deviation between internal models of robots’ AI and the real world. This talk will first describe efforts in tighter integration of perception and planning for vision-driven robot manipulation. We have developed high-fidelity, high-frequency tracking of rigid bodies’ 6D poses - without using CAD models or cumbersome human annotations - by utilizing progress both in deep learning and pose graph optimization. These solutions together with appropriate shared representations, tighter closed-loop operation and compliant mechanisms are unblocking the deployment of full-stack robot manipulation systems. This talk will provide examples of robust robotic packing, assembly under tight tolerances as well as constrained placement given a single demonstration that generalizes across an object category. The talk’s second part is motivated by tensegrity robots, which combine rigid and soft elements, to achieve safety and adaptability. They also complicate, however, modeling and control given their high-dimensionality and complex dynamics. This sim2real gap of analytical models has motivated us to look into reinforcement learning (RL) for controlling robot tensegrities, which allowed the development of new skills for them. RL applicability is limited, however, due to its high data requirements. Training RL in simulation is promising but is blocked again by the sim2real gap. For this reason, we are developing differential engines for tensegrity robots that reason about first-principles so as to be trained with few example trajectories from the real robot. They provide accurate-enough simulations to train a controller that is directly transferrable back to the real system. We report our first success in such a real2sim2real transfer for a 3-bar tensegrity robot. The talk will conclude with a brief discussion on how closing these gaps empowers the next step of developing robots that are socially cognizant and can be safely integrated into our society. Location: CoRE 301 + Virtual Committee: Start Date: 20 Sep 2022; Start Time: 11:30AM - 01:30PM Title: Projections, Extractors, and Streaming Lower Bounds Bio: Speaker: Abstract: See Above Location: Virtual Committee: Professor Swastik Kopparty, Chair (University of Toronto) Professor Eric Allender Professor Sepehr Assadi Professor Huacheng Yu (Princeton University) Start Date: 23 Sep 2022; Start Time: 11:00AM - Title: Closing the Reality Gap for Controlling Tensegrity Robots via Differentiable Physics Engines Bio: Speaker: Abstract: See above Location: Virtual Committee: Professor Kostas Bekris (Chair) Professor Mridul Aanjaneya Professor Abdeslam Boularias Professor Andrew Sabelhaus (Boston University) Start Date: 04 Oct 2022; Start Time: 11:00AM - Title: Hybrid robotics and implicit learning Bio: Michael Posa is an Assistant Professor in Mechanical Engineering and Applied Mechanics at the University of Pennsylvania. He leads the Dynamic Autonomy and Intelligent Robotics (DAIR) lab, a group within the Penn GRASP laboratory.  His group focuses on developing computationally tractable algorithms to enable robots to operate both dynamically and safely as they quickly maneuver through and interact with their environments, with applications including legged locomotion and manipulation. Michael received his Ph.D. in Electrical Engineering and Computer Science from MIT in 2017, where, among his other research, he spent time on the MIT DARPA Robotics Challenge team. He received his B.S. in Mechanical Engineering from Stanford University in 2007. Before his doctoral studies, he worked as an engineer at Vecna Robotics in Cambridge, Massachusetts, designing control algorithms for the BEAR humanoid robot. He has received the Best Paper award at Hybrid Systems: Computation and Control (HSCC) and been finalist awards at ICRA and IEEE Humanoids. He has also received Google Faculty Research Award in 2019 and the Young Faculty Researcher Award from the Toyota Research Institute in 2021. Speaker: Abstract: Machine learning has shown incredible promise in robotics, with some notable recent demonstrations in manipulation and sim2real transfer. These results, however, require either an accurate a priori model (for simulation) or a large amount of data. In contrast, my lab is focused on enabling robots to enter novel environments and then, with minimal time to gather information, accomplish complex tasks. In this talk, I will argue that the hybrid or contact-driven nature of real-world robotics, where a robot must safely and quickly interact with objects, drives this high data requirement. In particular, the inductive biases inherent in standard learning methods fundamentally clash with the non-differentiable physics of contact-rich robotics. Focusing on model learning, or system identification, I will show both empirical and theoretical results which demonstrate that contact stiffness leads to poor training and generalization, leading to some healthy skepticism of simulation experiments trained on artificially soft environments. Fortunately, implicit learning formulations, which embed convex optimization problems, can dramatically reshape the optimization landscape for these stiff problems. By carefully reasoning about the roles of stiffness and discontinuity, and integrating non-smooth structures, we demonstrate dramatically improved learning performance. Within this family of approaches, ContactNets accurately identifies the geometry and dynamics of a six-sided cube bouncing, sliding, and rolling across a surface from only a handful of sample trajectories. Similarly, a piecewise-affine hybrid system with thousands of modes can be identified purely from state transitions. I'll also discuss how these learned models can be deployed for control via recent results in real-time, multi-contact MPC. Location: 1 Spring street, New Brunswick, NJ – Room 403 + Virtual Committee:

Upcoming Events

 20 Sep 2022; - 11:30AM - 01:30PM Projections, Extractors, and Streaming Lower Bounds 23 Sep 2022; - 11:00AM - Closing the Reality Gap for Controlling Tensegrity Robots via Differentiable Physics Engines 04 Oct 2022; - 11:00AM - Hybrid robotics and implicit learning