Generators of Markov Chains

The theory of Markov chains, whether those time−discrete or those time−continuous, is one of the integral parts of the theory of stochastic processes. These lectures, however, are not devoted to the popular part of this rich theory, so that the students will not learn about recurrent and transient states, ergodic theorems, or convergence to equilibrium. Instead, we will focus on the equally intriguing question of how a continuous−time Markov chain may be described by means of its Kolmogorov (intensity) matrix or its generator, and study interplay between the notions just named. We will argue in particular that, despite their popularity, Kolmogorov (intensity) matrices are less suitable for such description than generators. Whereas, in their relative simplicity, they allow an intuitive formulation of processes, in general they fail to describe more delicate phenomena.
In particular, we will compare these two notions in the light of two illuminating examples due to Kolmogorov, Kendall and Reuter. These examples show that whereas intensity matrix determines in a sense the way the generator acts, it may not determine the generator’s domain, and without information on the shape of the domain a Markov chain is not completely specified. Furthermore, in the part devoted to boundary theory, we will show that an explosive intensity matrix characterizes the chain only locallyL up to a time of explosionN put otherwiseL the matrix characterizes merely the minimal chain, which after explosion is undefined. There are, however, infinitely many post−explosion processes, which dominate the minimal chain. Their generators may differ from the generator of the minimal chain by extra terms and may have di;erent domains; both the domain and the terms contain crucial information on the post−explosion process; by nature, this information cannot be found in the intensity matrix. It is true that, in the eye of a mathematician, Markov chains merely walk in their regular state−space, but on the cliffs of their boundaries they dance.
The way information on boundary behavior of a Markov chain is reflected in its generator will be a recurring subject in the lectures. Roughly: it transpires that if we work in the space of absolutely summable sequences, exit boundary introduces additional terms in the generator, whereas entrance boundary affects its domain. Interestingly, in the dual space of bounded sequences everything is turned upside down.
There are two basic ideas that permeate the course. First of all, instead of describing extraordinary Markov chains by rigorous, but involved, stochastic constructions, we develop intuitions about them by approximating the chains’s semigroups of transition probabilities by sequences of semigroups of transition probabilities related to some finite−space chains. The advantage the latter chains have is that they possess considerably simpler, and well−understood structure.
The second idea is the expression of the lecturer’s belief that intuitive should go before the abstract, and that discovering is more fun than learning. Hence, we will allow ourselves the comfort of discovering new results gradually, step by step, not trying to reach the mountain peak immediately via the most efficient route. The proof of an introductory result may thus be more complicated than that of a general theorem, discussed later; in proving the former we are simply yet not so clear about the general view. Neither will we be afraid to spend some time looking at an illustrative example, which perhaps involves more calculations than one could wish to go through, before the idea of a general theorem, and its simple proof, will dawn on us.