AI and Optimization Research

Rutgers University

- Office: Hill 268
- Office Hours: Tuesday, 1-2, Friday 3-5, or by Appointment
- Current Lectures: CS 520, M 6:40-9:20 (TIL 246), W 10:20 - 1:20 (TIL 242)
- Email: cwcowan at cs dot rutgers dot edu

Over the last 25 years, an explosion of data and computational resources have pushed AI to impressive advancements in many areas. But ultimately data can only take you so far, and not everything can be understood through correlation and correlation of correlations. So where do we go next?

**General Topics**- Bandit Problems, Optimal Learning Theory
- Learning on Small to Modest Data Sets
- Reinforcement Learning
- Multi-Agent Coordination, Learning, Collaboration
- Stochastic Optimization
- Inference Under Uncertainty, Bayesian Inference
- Unsupervised, Structual Learning

**Specific Projects****Joint Bandit Problems**- In learning, actions are frequently treated as independent from one another: the results of taking an action are generally taken to say anything about the potential results of other actions, like operating two independent slot machines to figure out which one has a higher win rate. But given additional information about how the actions relate to one another, they potentially become informative about each other. If you knew for a fact that one of the slot machines yielded a jackpot 10% of the time and the other only 5% of the time, data on one would suggest the identity of the other. How can joint information be utilized to accelerate learning?**Improving Aggregate Performance with High Precision Actors**- We've all observed how one bad driver can ruin traffic for everyone else around them. Is the inverse possible? Can one exceptionally good driver, or at least a small team of high precision drivers, improve net traffic over all for everyone? How could these precision drivers be designed and organized for maximum benefit? In multi-agent, decentralized systems, can the actions of few benefit the many?

**Speculative Projects****Learning and the Theory of Mind**- One of the things that allows humans to be successful, socially, is a theory of mind: a model we use to understand the minds of and anticipate the actions of those around us. How could AIs be equipped with a theory of mind for each other, to facilitate action and learning in multi-agent settings?

- Accelerating the Computation of UCB and Related Indices for Reinforcement Learning: W. Cowan, M.N. Katehakis, D. Pirutinsky
- Asymptotically Optimal Sequential Experimentation Under Generalized Ranking: W. Cowan, M.N. Katehakis
- An Asymptotically Optimal Policy for Uniform Bandits of Unknown Support: W. Cowan, M.N. Katehakis
- Normal Bandits of Unknown Means and Variances: Asymptotic Optimality, Finite Horizon Regret Bounds, and a Solution to an Open Problem: W. Cowan, J. Honda, M.N. Katehakis
- Multi-Armed Bandits Under Generalized Depreciation and Commitment: W. Cowan, M.N. Katehakis
- Conley--Morse Databases for the Angular Dynamics of Newton's Method on the Plane: J. Bush, W. Cowan, S. Harker, and K. Mischaikow
- Detecting Wave Function Collapse Without Prior Knowledge: C.W. Cowan, R. Tumulka
- Epistemology of Wave Function Collapse in Quantum Physics: C.W. Cowan, R. Tumulka
- Can One Detect Whether a Wave Function Has Collapsed?: C.W. Cowan, R. Tumulka

Things below this point are old, potentially un-edited. But still good.

### Miscellaneous

### CS 674: Mathematical Topics in AI

- Syllabus
- Unconstrained Optimization Notes
- Constrained Optimization Notes
- Optimization: Barrier and Penalty Optimization Methods (Mathematica)
- Bandit and Online Learning Notes
- Online Learning: Dynamic Programming Solution for Two Coins (Mathematica)
- UCB Index Policies for Bandit Problems (Mathematica)
- Inference Notes

### CS 520: Introduction to Artificial Intelligence

- Syllabus
- Planning and the Wolf/Goat/Cabbage Problem
- Notes on Querying Probabilistic Knowledge
- Probabilistic Inference (Problems)
- Temporal Estimation Notes
- Notes on Value Iteration for Markov Decision Processes
- Notes on Clustering Algorithms (featuring Luna Rose)
- Supervised Learning on Simple Models
- Notes on Neural Networks and Error Propagation
- Review Notes for Midterm

### CS 512: Algorithms and Data Structures

- Divisibility and Modular Arithmetic (Problems)
- Notes on Computing Modular Inverses
- Notes on Euclid's Algorithm and Diophantine Equations (with Problems)
- Notes on Graph Search Algorithms
- Analyzing Structured Graphs (Problems)
- Notes on Vertex Ordering for Graph Traversals
- Huffman Encoding and the Cost of Information (Problems)
- Randomized Algorithms and Independent Sets (Problems)
- Notes on Linear Programs and Duality
- Set Covers (Problems)
- Review Notes for Midterm
- Review Notes for Final