Principles of artificial intelligence, Fall 2005

5–6:20pm Tuesdays and Thursdays in Hill 120

Professor: Chung-chieh Shan (Ken)
Office: CoRE 306, 732-445-2003
Office hours: Thursdays 10–11am, or hang around after class, or by appointment
Email: ccshan at cs

Teaching assistant: Pai-Hsi Huang
Office: Hill 402, 732-445-4869
Office hours: Mondays 3–4pm, or by appointment
Email: paihuang at paul

We both prefer email over phone.

Announcements

The definition of HISP in homework 3 is wrong! It should be "some number greater than 1 (Spanish/Hispanic/Latino) or 1 (not)" rather than "1 (Spanish/Hispanic/Latino) or 0 (not)". Sorry. Just fixed.

To repeat the in-class announcement about the midterm on 11/3: It will be closed-book (but I hope it won't matter). It will cover material up to and including the class on 10/27, but not the class on 11/1. For review, I suggest that you go over the textbook sections assigned for reading, and make sure you know about hidden Markov models (either from the assigned paper or from the textbook) and logic (for example, proof search). Please bring a photo ID.

Oops on homework 1: In part 2, I forgot that Stone's algorithm considers the same state over and over again each time a transition enters it! This means that the problem is intractable without dynamic programming. So, please solve part 2 using dynamic programming rather than Stone's algorithm. It'd help you solve the later parts anyway. Thanks to Adrian.

More on homework 1: Please feel free to use any of the equivalent formal definitions of a Markov decision process. For example, you can separate decision nodes from chance nodes or merge them together. You can also attach a reward to an action from a state rather than a state. Finally, note that you can sell pancakes while making them; you just can't make two batches of pancakes at the same time.

In homework 1, if you only have 1 pancake, then it sells in the next 10 minutes with probability 80%. In general, if you have fewer pancakes to sell than the demand, then you sell all the pancakes you have. Also, you can make a new batch of pancakes even as you sell an old batch, but if you want to make a new batch when another batch is already and still being made, then you need to throw out the other batch.

A copy of the textbook is now available on reserve at the math library. The first edition of our textbook is also on reserve at the math library, in case someone else is using the second edition. We're trying to get more copies of the second edition on reserve as well, but in the meantime, a chapter map between the two editions is available online.

Please refer to the department's academic integrity policy.

Syllabus

This course is about

observing the world,
learning from data, and
making decisions

under uncertainty. We will learn some representations and algorithms that have been successfully applied in practice, the principles that motivate their design, and how to evaluate the performance of a system.

Prerequisites

You should be able to write non-trivial programs that others can run and read. You should know basic ways to represent the state of the world and search a space of states. These two prerequisites together entail that you should be able to write a (principled, if possibly slow) program to solve the Rubik's cube. You should also be familiar with basic calculus and probability theory, though not necessarily from a formal course.

Textbook

Stuart Russell and Peter Norvig. 2003. Artificial Intelligence: A Modern Approach, 2nd ed. Prentice Hall. ISBN: 0-13-790395-2. We will also distribute some articles for you to read.

Assignments

There will be 6 homework assignments. Homeworks will generally be due two weeks after being handed out. Some homework problems will ask you to respond in prose or equations. Other problems will ask you to write a program, submit it, and describe how it works and how well it works. (If you are not a CS graduate student and need an account to do the programming assignments, please let us know.)

There will be a midterm in early November. There will not be a final exam. Instead, there will be a final project, in which you choose a problem domain, apply the methods we learn, and evaluate your implementations. You must hand in a written proposal for the final project by November 23, and schedule a meeting to discuss it in early December. The final project paper is due on December 16. We will give more detailed instructions about the final project later.

Grading

The first homework is for us to gauge your background, so it will not count for grading. Each subsequent homework and the midterm will count for 10% of the final grade, for a total of 60%. The final project proposal and the final project paper will each count for 20% of the final grade.

Collaboration policy

Because the first homework is for us to gauge your background, please don't discuss it with other people. Except for the first homework, we encourage you to discuss the assignments with others, but you must write the solutions (including programs) yourself, and acknowledge your collaborators in your solutions.

Late policy

All assignments should be submitted electronically by 5pm on the due date. Each day (or partial day) late discounts the grade for that assignment by 25%, so a submission more than 72 hours late is worthless.

Schedule and handouts

It's tentative! Especially in December.

	Decisions
9/1	Welcome; making a rational decision under uncertainty Sign-up questionnaire Homework 0 Reading: Matthew Stone. 2003. Agents in the real world: Computational models in artificial intelligence and cognitive science. Revised version to appear in Lepore and Pylyshyn, eds., Cognitive Science, vol. 2, Blackwell.
9/6	Making a sequence of decisions (Markov decision processes) Reading: AIMA §§13.1–4, 16.1–3, 17.1–2.
9/8	(cont'd: value iteration, policy iteration) Reading: AIMA §§17.3–4.
9/13	Uncertain perception: partially observable Markov decision processes Homework 1 Reading: AIMA §§21.1–3.
9/15	Uncertain model: reinforcement learning Homework 0 due. Reading: L. R. Rabiner and B. H. Juang. 1986. An introduction to hidden Markov models. IEEE Acoustic, Speech, and Signal Processing Magazine 3(1):4–16. Please don't worry about the solution to problem 3 in the paper for now. If you can't make out the figures in this scan, another version may help.
	Graphical models
9/20	Inference from a sequence of observations (hidden Markov models); model estimation (maximum likelihood, maximum a posterior, Bayesian) Optional readings to refresh your knowledge of probability theory: "A tutorial on probability theory"; common probability distributions. See eCompanion Web site for these materials.
9/22	(cont'd: variable elimination) Optional reading, for those who would like more details on hidden Markov models and speech recognition (not to mention a more readable scan): Lawrence R. Rabiner. 1989. An introduction to hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2):257–286. Thanks to Ali.
9/27	Guest lecture by Prof Michael Littman begins at 4:40pm
9/29	Guest lecture by Prof Vladimir Pavlovic
10/4	Graphical models for probability distributions (Bayes nets) Homework 1 due. Reading: AIMA §§13.5–6, 13.8, 14.1–4.
10/6	Inference using graphical models; causality Homework 2 Optional reading, for those who would like more details on variable elimination: Rina Dechter. 1998. Bucket elimination: a unifying framework for probabilistic inference. In Michael I. Jordan, ed., Learning and Inference in Graphical Models, Kluwer. If you'd like to learn more about causality and Bayesian reasoning, check out Judea Pearl's work, such as: Judea Pearl. 1996. Causation, action, and counterfactuals. In Alex Gammerman, ed., Computational learning and probabilistic reasoning, chap. 15, 235–255, New York: Wiley. Also as Technical Report R-223-U, UCLA Cognitive Systems Laboratory.
10/11	The complexity of exact inference; approximate Bayesian inference: sampling Reading: AIMA §§14.5, 18.1–2.
10/13	Approximate Bayesian inference (factoring); modeling exercise
10/18	First-order probabilistic models Homework 3
10/20	(cont'd) Homework 2 due.
	Learning from data
10/25	Learning decision trees
10/27	Learning to generalize: overfitting and pruning Reading: AIMA §§18.1–3. Reading: Sections 1 and 2 of: David Heckerman. 1995 (revised 1996). A tutorial on learning with Bayesian networks. Microsoft Research Technical Report MSR-TR-95-06. Homework 4
11/1	Learning probability distributions (ML, Bayesian, MAP) Homework 3 due.
11/3	Midterm Minor correction: "12 examples" should be "13 examples".
11/8	Learning probability distributions (cont'd) Final project instructions Reading: Sections (1 and 2 and) 3 and 4 of: David Heckerman. 1995 (revised 1996). A tutorial on learning with Bayesian networks. Microsoft Research Technical Report MSR-TR-95-06.
11/10	Learning probability distributions (cont'd) Reading: Section (1–4 and) 5 of: David Heckerman. 1995 (revised 1996). A tutorial on learning with Bayesian networks. Microsoft Research Technical Report MSR-TR-95-06. Reading: J. R. Quinlan. 1990. Learning logical definitions from relations. Machine Learning 5(3):239-266. Optional reading on writing an abstract for your papers: Kenneth K. Landes. 1966. A scrutiny of the abstract, II. Bulletin of the American Association of Petroleum Geologists 50(9):1992. Optional reading on writing the rest of your papers: Pat Langley. 1990. Advice to Machine Learning authors. Machine Learning 5(3):233–237.
11/15	Inductive logic programming
	Neural networks
11/17	Perceptrons Homework 4 due.
11/22	(no class) Final project proposal due 11/23
11/24	(no class)
11/29	Multi-layer neural networks; discuss final project proposals Reading: AIMA §20.4.
12/1	Structuring neural networks and Bayes nets Homework 5
	More ways to learn
12/6	Non-parametric/instance-based methods (histograms, kernels, nearest neighbors)
12/8	Natural language
12/13	Boosting Homework 5 due. Optional reading on boosting: Robert E. Schapire. 2003. The boosting approach to machine learning: An overview. In Nonlinear estimation and classification, ed. David D. Denison, Mark H. Hansen, Christopher C. Holmes, Bani Mallick, and Bin Yu. Lecture Notes in Statistics 171, Berlin: Springer-Verlag.
12/16	(no final exam) Final project paper due