Markov decision processes: dynamic programming and applications

ENSTA Course 5OD2A/B & M2 Optimization (IP Paris and Paris-Saclay University)

Marianne Akian



The aim of this course is to introduce different stochastic control models and to present dynamic programming as a tool for solving them.

In 2025, 6 lectures of 3 hours (on Tuesday Mornings from September 9 to October 14) are common to ENSTA course and M2 course, and necessary to validate the ENSTA course and the M2 course.

4 additional lectures of 3 hours (on Wednesday afternoon from October 8 to 22 and on Tuesday morning 21) are necessary to validate the M2 course. ENSTA students may attend these lectures.


Lecture Notes (including the material of the ENSTA and Master 2 lectures and some problems)

Exams of the last 4 years

For solutions of exams and problems and for 2025 projects ask to me.



Akian Marianne 2025-09-04