Homework 3

Due March 25th.

• 12: What's the probability of a ``natural'' (21 on the first hand) in Blackjack (see Lecture 14 for definitions)?

• 13: In Blackjack, given the dealer's strategy of always sticking on 17 or above and hitting below, what's the probability that the dealer will get a 21 given that she starts with a ten? (Hint: I made this easier by picking a ten to start--there is no choice about what to do with the aces.) Write out the equations solve them.

• 14: Programming assignment, due April 1st. First program: Given the six color probabilities, output a value function (for each location-wedgeset combination, expected turns to goal, see out for an example.) Second program: Given the six color probabilities and a value function, interactively query the user for a location-wedgeset combination. Given a location-wedgeset combination, output, for each die roll, the set of locations you can choose from along with their expected values. Mark the optimal choice. Play with your program a little to find interesting things to report!
• 15: Consider an MDP with a step cost of 0.02 (reward of -0.02 on each step) and a goal state (reachable with positive probability) with value +1. (a) Why is this not a positive bounded model? (b) Why is this not a negative model? Even so, the optimal value function exists. (c) Argue that this is so, by explaining how to convert the MDP into one of the other two models without changing the optimal policy.