We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Data Science & Analytics/On Gradient-Based Learning in Continuous Games
PeriodicalsData Science & Analytics

On Gradient-Based Learning in Continuous Games

Checking your access to this record.

Abstract & Overview

We introduce a general framework for competitive gradient-based learning that encompasses a wide breadth of multiagent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory. For both general-sum and potential games, we characterize a nonnegligible subset of the local Nash equilibria that will be avoided if each agent employs a gradient-based learning algorithm. We also shed light on the issue of convergence to non-Nash strategies in general- and zero-sum games, which may have no relevance to the underlying game, and arise solely due to the choice of algorithm. The existence and frequency of such strategies may explain some of the difficulties encountered when using gradient descent in zero-sum games as, e.g., in the training of generative adversarial networks. To reinforce the theoretical contributions, we provide empirical results that highlight the frequency of linear quadratic dynamic games (a benchmark for multiagent reinforcement learning) that admit global Nash equilibria that are almost surely avoided by policy gradient.

Cite this record

Eric Mazumdar, Lillian J. Ratliff, S. Shankar Sastry (2020). On Gradient-Based Learning in Continuous Games. SIAM Journal on Mathematics of Data Science, 2(1), 103–131. https://doi.org/10.1137/18m1231298

Copy in another style

Mazumdar, E., Ratliff, L. J., & Sastry, S. S. (2020). On Gradient-Based Learning in Continuous Games. SIAM Journal on Mathematics of Data Science, 2(1), 103–131. https://doi.org/10.1137/18m1231298
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.