- 16: In class, we conjectured that certainty equivalence gives the
same answer as either first visit or last visit TD. I don't think
this is true. Draw a two-state example with a cycle and a list of
sample trajectories. Show that first-visit TD, last-visit TD, and
certainty equivalence all give different answers for the value
function.
- 17: Given the figure in this lecture in the section labeled
``homework'', (a) what's the probability that the missing node is Rh-?
(b) What if there are an infinite number of Rh+ kids instead of just
3?
- 18: How can you express matrix multiply using ``mjoin'' and
``margin'' operators we discussed?
- 19: Compute the probability that the state is ``up'' in time step
4 in the hidden Markov model example.
Up: Hidden Markov Models (19)
Previous: More Stuff