A unified switching system perspective and convergence analysis of Q-learning algorithms

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 55
  • Download : 0
This paper develops a novel and unified framework to analyze the convergence of a large family of Q-learning algorithms from the switching system perspective. We show that the nonlinear ODE models associated with Q-learning and many of its variants can be naturally formulated as affine switching systems. Building on their asymptotic stability, we obtain a number of interesting results: (i) we provide a simple ODE analysis for the convergence of asynchronous Q-learning under relatively weak assumptions; (ii) we establish the first convergence analysis of the averaging Q-learning algorithm, and (iii) we derive a new sufficient condition for the convergence of Q-learning with linear function approximation.
Publisher
Conference on Neural Information Processing Systems
Issue Date
2020-12-07
Language
English
Citation

34th Conference on Neural Information Processing Systems, NeurIPS 2020

URI
http://hdl.handle.net/10203/278702
Appears in Collection
EE-Conference Papers(학술회의논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0