This article is cited in 31 papers

Markov controlled models with countable state space and continuous time

A. A. Yuškevič

Moscow

Abstract: Following general lines of [10], we expand the Blackwell–Strauch dynamic programming theory, which takes into account policies depending on the whole past, to continuous time Markov decision processes with countable state and Borel action spaces. Nonhomogeneous processes with finite and infinite horizon and non-randomized policies are treated. An optimality equation is obtained for negative models and models with value not equal to $-\infty$. The existence of Markovian $\varepsilon$-optimal policies is proved for models with bounded value and small positive share of far future. The semicontinuous case is also considered.

Received: 10.09.1975

 English version: Theory of Probability and its Applications, 1978, 22:2, 215–235

Bibliographic databases:

• 
•