I have a python reinforcement-learning demo (developed with Carlos Diuk) of the well-known taxi problem. It is available for download, but please send me mail if you try it out. We used it in an experiment for Carlos' dissertation and in a NIPS 2009 Tutorial on model-based reinforcement learning.
The "upod" group is creating Scratchable devices, everyday gadgets that can be programmed in a friendly and natural way.
I am serving as department chair July 2009--June 2012. We now have an archive of my Monday messages to the Rutgers CS community.
I've enjoyed creating various kinds of videos for teaching, research, and fun. You are welcome to check out my portfolio on my YouTube channel.
I declared myself an outspoken proponent of progressive values. My spouse and I used to have a little blog on The Huffington Post, but we don't remember how to submit new articles and we stopped.
My students and I created The Rutgers Laboratory for Real-Life Reinforcement Learning (or RL3). Check us out! We generally add one new member each year from among those admitted to the Rutgers CS Ph.D. program.
I have a page for publications, but I don't really like the formatting. Other papers are also online.
Here is a list of the PhD dissertations I have advised.
Spring 2010, I taught an honors course on Programming for the Masses to explore what the rest of the world might do if they could program. The spinoff project for Summer 2010 is "End user programming", or "upod" for "user programming of devices".
Spring 2009, I taught a graduate course on sequential decision making.
The lab also ran a light seminar on "A Machine Learning Approach to Solving Pitfall" (sakai only).
Fall 2008, I taught a CS intro course for non-majors, Great Insights in Computer Science (CS105). Some class members sent me words of support.
Carlos Diuk, Chris Mansley and I ran a light seminar on the topic of Bayesian Reinforcement Learning.
Fall 2007, I taught a CS intro course for non-majors, Great Insights in Computer Science (CS105), available from sakai. Lihong Li and I organized a light seminar on the topic of Planning in Learned Environments.
Spring 2007, I again taught a CS intro course for non-majors, Great Insights in Computer Science. Enrique Munoz de Cote and I also ran a seminar on the topic of Multiagent reinforcement learning.
Fall 2006, I taught Great Insights in Computer Science and helped Alex Strehl organize a Light seminar on reinforcement-learning theory.
I created a new course for non-majors, offered Spring 2006, Great Insights in Computer Science.
Fall 2005, I reprised my course on Learning and Sequential Decision Making.
Spring 2005, I taught Discrete Math (undergrad CS205) and co-organized the Social Reinforcement Learning light seminar with Matthew Stone.
Fall 2004, I taught Discrete Math (undergrad CS205) and Machine Learning (graduate CS536).
Spring 2004, I taught a course on Learning and Sequential Decision Making.
Fall 2003, I taught Machine Learning and ran a Learned Representations in AI Light Seminar.
Spring 2003, I ran a Learning Robots Reading Group.
Fall 2001, I taught Introduction to Artificial Intelligence at Princeton.
I was one of the organizers of our department's Yahoo! seminar series in Machine Learning with Michael Pazzani and Tina Eliasi-Rad (2010-2011).
I'm a big fan of Scratch and have used it for teaching and learning research. Here's my daughter Molly's scratch project page.
In the year 5770, I served as Ceremonial Leader ("lay rabbi") of the Congregation for Humanistic Judaism of Morris County. I also recently learned that there is a Humanist Chaplaincy at Rutgers.
I was one of the co-organizers of the Rutgers Yahoo! Machine Learning Seminar, Tuesdays at 11am (lunch provided), 2009-2010.
I was programme co-chair of ICML 2009 in Montreal.
I worked on the Journal of Machine Learning Research (action editor).
I participated in a panel as part of the opening of the NYC play, Universal Robots. The play won some nice awards!
I performed in HMS Pinafore Summer 2008.
I served on the organizing committee of the 2008 Reinforcement Learning Competition.
I served on the organizing committee for ICML/UAI/COLT 2008 workshops.
I helped out with the 2nd AAAI Video Competition.
In 2007-2008, I served as adult ed chair and web monster of the Congregation for Humanistic Judaism of Morris County. I've also been secretary and vice president.
Journal of Artificial Intelligence Research (advisory board member). Journal of Machine Learning Gossip, American Association for Artificial Intelligence (ex Council member).
Fall 2007, I helped organize the AAAI Fall Symposium on Computational Approaches to Representation Change During Learning and Development.
Fall 2007, I served as local arrangements chair of the 7th International Conference on Epigenetic Robotics, held here at Rutgers.
I served as guest editor, with Amy Greenwald, of the Machine Learning Journal Special Issue on Learning and Computational Game Theory. The issue is now available.
I was on the program committee for the first AI Video Competition in 2007. I won the first "Shakey" for Short Video for Aibo Ingenuity.
I ran The First RL Benchmarking Event as part of a NIPS workshop in 2005.
I co-organized the Reinforcement Learning Benchmarks and Bake-offs Workshop at NIPS 2004.
I stay in touch with Charles Isbell and the Threads project he helped create that will transform computer-science education.
I co-organized the Probabilistic Track of the 2004 International Planning Competition (with Haakan Younes) and co-organized a AAAI 2004 Fall Symposium on Real-Life Reinforcement Learning (with Satinder Singh).
With my CS205 class, 2/8/2005, we thought we found a bug in the US Constitution!
Spring 2004, I became a member of the Rutgers Cognitive Science Center.
I gave two game-theory and learning tutorials summer of 2003, Multiagent Learning: A Game Theoretic Perspective at IJCAI-03 (with Michael Bowling), and Learning Topics in Game -Theoretic Decision Making at COLT-03. I was co-organizer of the Multi-Agent Learning: Theory and Practice workshop at NIPS 2002 (with Gerry Tesauro) and the Personalized Agents 2002 Fall Symposium at AAAI (with Charles Isbell).
I'm a founding member of the MetroBots RoboCup soccer team, which competed in the First American Open and RoboCup in 2003. I have looked into methods for dimension reduction for text applications as part of a NASA-funded research project in 2002.
My family and his family sang Justin Boyan a song for his 40th birthday. Similarly, I put up a file with some of my favorite quotes from Charles Isbell when he turned 40.
My explanation of the Monty Hall Problem (Jan. 2006) can be found online.
There are editorial comments I make very frequently.
I prefer some names more than others.
I compiled a list of Communication Technology Firsts.
Michael L. Littman is a big fan of google but not googlism. Some real-life googlewhacks. I've started collecting slogans to help me understand the world around me. I can't bear to delete the links to stuff I started creating at Duke.