Skip to content

Reinforcement Learning

Syllabus

Reinforcement Learning. Reinforcement Learning Algorithms. Implementation of autonomous agents using reinforcement learning.

Objectives

By the end of the course, students will be able to:

  1. Build a reinforcement learning-based system for sequential decision-making.
  2. Understand how to formalize a task as a reinforcement learning problem, how to implement a solution, and how to evaluate it.
  3. Understand the types of reinforcement learning algorithms: value-based, policy gradient, and actor-critic.
  4. Understand the relationship between reinforcement learning and supervised and unsupervised learning.

Course Content

  1. Introduction to Reinforcement Learning.
  2. Implementation of autonomous agents using reinforcement learning.
  3. Taxonomy of reinforcement learning algorithms.
  4. Q-Learning Algorithm.
  5. Sarsa Algorithm.
  6. Deep Reinforcement Learning.
  7. Deep Q-Learning algorithms.
  8. Reinforce: a Policy Gradient algorithm.
  9. Actor-Critic algorithms.
  10. Implementation of autonomous agents using projects such as Farama's Gymnasium and Kaggle's reinforcement learning library.
  11. Examples of solutions using reinforcement learning.

Required Bibliography

  1. SUTTON, R.; BARTO, A. Reinforcement Learning: An Introduction. Second Edition. The MIT Press, 2018.
  2. GÉRON, A. Hands-on Machine Learning with Scikit-learn, Keras, and TensorFlow, 2nd ed., O'Reilly, 2021.
  3. Van Hasselt, H., Guez, A. and Silver, D., 2016, March. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30, No. 1).
  4. Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
  5. Brockman, G. et al., 2016. Openai gym. arXiv preprint arXiv:1606.01540.

Supplementary Bibliography

  1. Laura Graesser and Wah Loon Keng. 2019. Foundations of Deep Reinforcement Learning: Theory and Practice in Python (1st. ed.). Addison-Wesley Professional.
  2. NORVIG, P.; RUSSELL, S., Artificial Intelligence: A Modern Approach, 3rd ed., Prentice Hall, 2009.
  3. SILVER, D.; SINGH S.; PRECUP D.; SUTTON R. Reward is enough. Artificial Intelligence. Vol 299, 2021.
  4. MuZero: Mastering Go, chess, shogi and Atari without rules. Published in December, 2020.
  5. SILVER, D.; HUBERT T.; SCHRITTWIESER, J.; ANTONOGLOU, I.; LAI, M.; GUEZ, A. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140-1144 (2018).
  6. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M., 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.
  7. Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. 2018. Deep reinforcement learning that matters. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI'18/IAAI'18/EAAI'18). AAAI Press, Article 392, 3207–3214.
  8. Dohare, S., Hernandez-Garcia, J.F., Lan, Q. et al. Loss of plasticity in deep continual learning. Nature 632, 768–774 (2024). https://doi.org/10.1038/s41586-024-07711-7