Abstract:

Autonomous agents are artificial intelligence programs that repeatedly sense their environment, decide what actions to take so as to satisfy their goals, and execute those actions. When situated in dynamic, multi-agent environments, machine- learning techniques can be useful for adjusting agents' policies so that they interact favorably with those of the other agents. In this talk, I present two examples of agents learning in such challenging environments.

First, "ATTac" is an autonomous bidding agent that learns to predict closing prices in simultaneous auctions for interacting goods. In auctions such as these, a fundamental problem is price prediction, and more broadly, the modeling of uncertainty regarding these prices. ATTac uses a novel and general boosting-based algorithm for conditional density estimation problems of this kind. ATTac was the top-scoring agent in a recent trading agent competition. In addition to the competition results, I present controlled experiments isolating the effectiveness of our price-prediction algorithm.

Second, I describe a successful scaling up of reinforcement learning methods to a complex subtask of simulated robotic soccer. In particular, our agents simultaneously learn cooperative high-level decisions via episodic SMDP Sarsa (lambda) with linear tile-coding function approximation and variable lambda. Robotic soccer presents many challenges to reinforcement learning methods, including a large state space, hidden and uncertain state, multiple agents, and long and variable delays in the effects of actions. Nonetheless, our agents learn policies that significantly out-perform a range of benchmark policies. I demonstrate the generality of our approach by applying it to a number of task variations.

Speaker is being hosted by Haym Hirsh