POMDP FAQ

What is a POMDP?

POMDP is an acronym for a partially observable Markov decision process. This is a mathematical model that can capture the domain dynamics that include uncertainty in action effects and uncertainty in perceptual stimuli. Once a problem is captured as POMDP, it them becomes more ammendable for solution using optimization techniques.

Does this have anything to do with Markov Chains or HMMs?

Sure does. These are all stochastic, discrete state, discrete time models. Borrowing from Michael Littman's nifty explanatory grid:

Markov
Models		 Do we have control
over the state transitons?
NO YES
Are the states
completely
observable?		 YES

Markov Chain

MDP

Markov Decision Process
NO

HMM

Hidden Markov Model

POMDP

Partially Observable
Markov Decision Process