Markov decision processes: dynamic programming and applications

ENSTA Course SOD312 & M2 Optimization (IP Paris and Paris-Saclay University)

Marianne Akian



Starting in 2024, the course holds in "Bâtiment d'Enseignement Mutualisé (BEM)" (Av. Fresnel), see the building here.



The aim of this course is to introduce different stochastic control models and to present dynamic programming as a tool for solving them.

In 2023, Lectures 1-5 are common to ENSTA course and M2 course, and necessary to validate the ENSTA course and the M2 course.

Lectures 6-9 are necessary to validate the M2 course. ENSTA students may attend these lectures.


Lecture Notes (including the material of the ENSTA and Master 2 lectures and some problems)

Exams of the last 4 years



Akian Marianne 2022-09-26