Reinforcement Learning for Football

Motivation

This project explores how well multi-agent reinforcement learning works in simulated football games and how scalable it is against to the single-agent case.

Research Question

What insights do multi-agent reinforcement learning techniques offer regarding performance and scalability within simulated football environments?

Tasks and Timeline

Stage 1:

• Define the reinforcement learning problem in a video games setting with a focus on football;
• Delimitate the solution concept to particular scenarios;
• Review the state-of-the-art and proposed solutions in the literature;
• Brainstorm potential solutions for the problem.

Stage 2:

• Plan the experiments and test for the expected solution;
• Set up a suitable football environment for experimentation;
• Develop a pre-processing data methodology;
• Select meaningful features from the dataset;
• Design a proof-of-concept for potential solutions via minimum viable product concept;
• Analyze the proposed proof-of-concept.

Stage 3:

• Evaluate the proof-of-concept with the proposed experiments and tests;
• Benchmark the proof-of-concept against other solutions in the literature.

Stage 4:

• Document the problem description and solution-design methodology;
• Show the results to reviewers and non-technical audiences.

Expected Outcomes

Mandatory Products

• An rl agent capable of playing football in video games;
• Benchmark with other rl algorithms in the literature;
• Mathematical analysis of the proposed algorithm;
• Software with documentation associated with the project;
• Technical report with problem description, proposed solutions, experimental results, and project conclusions by following the University guidelines;
• A public dissertation following the University guidelines.

Optional Products

• Summary paper from the technical report suitable for conferences or journals;
• 3-minute elevator pitch video of the project;
• Blog post or video explaining the problem and proposed solution for a general audience.

Bibliography

• Addison Howard, Anton Raichuk, Bertrand Rondepierre, City Football Group, Greg Swimer, Marcin Michalski, Piotr Stanczyk, Sam Harris, tsmall. (2020). Google Research Football with Manchester City F.C.. Kaggle. https://kaggle.com/competitions/google-football
• Kurach, K., Raichuk, A., Stańczyk, P., Zając, M., Bachem, O., Espeholt, L., … & Gelly, S. (2020, April). Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 04, pp. 4501-4510).