Abstract
We propose ORAA, a novel incentive-driven algorithm that guides agents in a property-based Multi-Agent Reinforcement Learning domain to act sustainably considering a common pool of resources in an online manner. ORAA implements our proposed P-MADDPG model to learn and make decisions over the decentralised agents. We test our solutions in our novel domain, the “Pollinators’ Game”, which simulates a property-based scenario and the incentivisation dynamics. We show significant improvement in the incentives’ cost-efficiency, reducing the budget spent while increasing the collection of rewards by individual agents. Besides that, our application shows better results when using learned (approximated) models instead of using and simulating the true models of each agent for planning, saving up to 50% of the available budget for incentivisation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
GitHub webpage: https://github.com/lsmcolab/oraa.
References
Hughes, E., et al.: Inequity aversion improves cooperation in intertemporal social dilemmas. In: NIPS (2018)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments (2017)
Perolat, J., Leibo, J.Z., Zambaldi, V., Beattie, C., Tuyls, K., Graepel, T.: A multi-agent reinforcement learning model of common-pool resource appropriation. In: NIPS, pp. 3643–3652 (2017)
Peysakhovich, A., Lerer, A.: Prosocial learning agents solve generalized stag hunts better than selfish ones. In: Proceedings of the 17th AAMAS, pp. 2043–2044 (2018)
Rankin, D.J., Bargum, K., Kokko, H.: The tragedy of the commons in evolutionary biology. Trends Ecol. Evol. 22(12), 643–651 (2007)
Wang, J.X., Hughes, E., Fernando, C., Czarnecki, W.M., Duéñez-Guzmán, E.A., Leibo, J.Z.: Evolving intrinsic motivations for altruistic behavior. In: Proceedings of AAMAS 2019, pp. 683–692 (2019)
Yang, J., Li, A., Farajtabar, M., Sunehag, P., Hughes, E., Zha, H.: Learning to incentivize other learning agents, p. 20 (2020)
Acknowledgements
We want to acknowledge the São Paulo Research Foundation (FAPESP), 2018/15472-9, and Agência Nacional do Petróleo, Gás Natural e Biocombustíveis (ANP)/Petrobras for partially funding this research and resulting paper. We also thank the staff working on the Lancaster University’s High End Computing (HEC) Cluster for providing the necessary computational resource and support to this project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pelcner, L., do Carmo Alves, M.A., Marcolino, L.S., Harrison, P., Atkinson, P. (2025). Incentive-Driven Multi-agent Reinforcement Learning Approach for Commons Dilemmas in Land-Use. In: Arisaka, R., Sanchez-Anguix, V., Stein, S., Aydoğan, R., van der Torre, L., Ito, T. (eds) PRIMA 2024: Principles and Practice of Multi-Agent Systems. PRIMA 2024. Lecture Notes in Computer Science(), vol 15395. Springer, Cham. https://doi.org/10.1007/978-3-031-77367-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-77367-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-77366-2
Online ISBN: 978-3-031-77367-9
eBook Packages: Computer ScienceComputer Science (R0)