Markov chains

Lecturer: G. Giacomin
Term 1
Hourly Volume: 3 hours per week

This course is an introduction to the general theory of time homogeneous Markov chains.  It will start with a detailed review of the theory in discrete time and discrete state space. Then continuous time and/or more general state spaces will then be considered. Some specific examples (random walks, birth and death chains, renewal processes, branching processes) will be developed in detail. 


1. Discrete-time chains on countable state spaces: construction, transition matrix, communication classes, periodicity.
Recurrence and transience, invariant measures, ergodic theorems. 

2. Potential theory: Martingales, potentials associated with a chain.   
 Reversible case : electrical networks analogy.   

3. Discrete-time chains on general state spaces: Harris chains. Recurrence, transience, invariant measures, asymptotic behavior. 

4. Continuous-time chains on countable state spaces: Construction, generator, asymptotic behavior. 

5. Convergence to diffusion processes: diffusions as Markov processes, Donsker’s Theorem (convergence to Brownian mot!ion), discussion of the general case. 


- F. COMETS, T. MEYRE, Calcul Stochastique et Modèles de Diffusions, Dunod, 3ème edition 2020

- J. F. LE GALL, Intégration Probabilités et Processus Aléatoires,

-  R. DURRETT, Probability : Theory and Examples, Duxbury Press (1996)

- J. R. NORRIS, Markov Chains, Cambridge 1997