Skip to main content

Incentive-Driven Multi-agent Reinforcement Learning Approach for Commons Dilemmas in Land-Use

  • Conference paper
  • First Online:
PRIMA 2024: Principles and Practice of Multi-Agent Systems (PRIMA 2024)

Abstract

We propose ORAA, a novel incentive-driven algorithm that guides agents in a property-based Multi-Agent Reinforcement Learning domain to act sustainably considering a common pool of resources in an online manner. ORAA implements our proposed P-MADDPG model to learn and make decisions over the decentralised agents. We test our solutions in our novel domain, the “Pollinators’ Game”, which simulates a property-based scenario and the incentivisation dynamics. We show significant improvement in the incentives’ cost-efficiency, reducing the budget spent while increasing the collection of rewards by individual agents. Besides that, our application shows better results when using learned (approximated) models instead of using and simulating the true models of each agent for planning, saving up to 50% of the available budget for incentivisation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    GitHub webpage: https://github.com/lsmcolab/oraa.

References

  1. Hughes, E., et al.: Inequity aversion improves cooperation in intertemporal social dilemmas. In: NIPS (2018)

    Google Scholar 

  2. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments (2017)

    Google Scholar 

  3. Perolat, J., Leibo, J.Z., Zambaldi, V., Beattie, C., Tuyls, K., Graepel, T.: A multi-agent reinforcement learning model of common-pool resource appropriation. In: NIPS, pp. 3643–3652 (2017)

    Google Scholar 

  4. Peysakhovich, A., Lerer, A.: Prosocial learning agents solve generalized stag hunts better than selfish ones. In: Proceedings of the 17th AAMAS, pp. 2043–2044 (2018)

    Google Scholar 

  5. Rankin, D.J., Bargum, K., Kokko, H.: The tragedy of the commons in evolutionary biology. Trends Ecol. Evol. 22(12), 643–651 (2007)

    Article  Google Scholar 

  6. Wang, J.X., Hughes, E., Fernando, C., Czarnecki, W.M., Duéñez-Guzmán, E.A., Leibo, J.Z.: Evolving intrinsic motivations for altruistic behavior. In: Proceedings of AAMAS 2019, pp. 683–692 (2019)

    Google Scholar 

  7. Yang, J., Li, A., Farajtabar, M., Sunehag, P., Hughes, E., Zha, H.: Learning to incentivize other learning agents, p. 20 (2020)

    Google Scholar 

Download references

Acknowledgements

We want to acknowledge the São Paulo Research Foundation (FAPESP), 2018/15472-9, and Agência Nacional do Petróleo, Gás Natural e Biocombustíveis (ANP)/Petrobras for partially funding this research and resulting paper. We also thank the staff working on the Lancaster University’s High End Computing (HEC) Cluster for providing the necessary computational resource and support to this project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Leandro Soriano Marcolino .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pelcner, L., do Carmo Alves, M.A., Marcolino, L.S., Harrison, P., Atkinson, P. (2025). Incentive-Driven Multi-agent Reinforcement Learning Approach for Commons Dilemmas in Land-Use. In: Arisaka, R., Sanchez-Anguix, V., Stein, S., Aydoğan, R., van der Torre, L., Ito, T. (eds) PRIMA 2024: Principles and Practice of Multi-Agent Systems. PRIMA 2024. Lecture Notes in Computer Science(), vol 15395. Springer, Cham. https://doi.org/10.1007/978-3-031-77367-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-77367-9_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-77366-2

  • Online ISBN: 978-3-031-77367-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics