Research articles

Algorithms for determining the state-time probabilities and the limit matrix in Markov chains

Dmitrii Lozovanu^a, Stefan Pickl<sup>b

a</sup> Institute of Mathematics and Computer Science, Academy of Sciences of Moldova, Chişnău, Moldova
^b Institut fur Angewandte Systemwissenschaften und Wirtschaftsinformatik, Fakultät fur Informatik, Universität der Bundeswehr, München

Abstract: New calculation procedures for finding the probabilities of state transitions of the system in Markov chains based on dynamic programming are developed and polynomial time algorithms for determining the limit state matrix in such processes are proposed. Computational complexity aspects and possible applications of the proposed algorithms for the stochastic optimization problems are characterized.

Keywords and phrases: discrete Markov process, probability of state transition, limit state matrix, dynamic programming, polynomial time algorithm.

MSC: 93E20, 49L20

Received: 06.10.2010

Language: English

Bibliographic databases:

• 
•