### What is a POMDP?

POMDP is an acronym for a partially observable Markov decision process. This is a mathematical model that can capture the domain dynamics that include uncertainty in action effects and uncertainty in perceptual stimuli. Once a problem is captured as POMDP, it them becomes more ammendable for solution using optimization techniques.

### Does this have anything to do with Markov Chains or HMMs?

Sure does. These are all stochastic, discrete state, discrete time models. Borrowing from Michael Littman's nifty explanatory grid:

Markov Models |
Do we have control over the state transitons? |
||
---|---|---|---|

NO | YES | ||

Are the states completely observable? |
YES | ## Markov Chain |
## MDPMarkov Decision Process |

NO | ## HMMHidden Markov Model |
## POMDPPartially ObservableMarkov Decision Process |