DC Field | Value | Language |
---|---|---|
dc.contributor.author | Cho, Hanseul | ko |
dc.contributor.author | Yun, Chulhee | ko |
dc.date.accessioned | 2023-12-07T23:00:31Z | - |
dc.date.available | 2023-12-07T23:00:31Z | - |
dc.date.created | 2023-12-07 | - |
dc.date.issued | 2023-05-03 | - |
dc.identifier.citation | 11th International Conference on Learning Representations, ICLR 2023 | - |
dc.identifier.uri | http://hdl.handle.net/10203/316024 | - |
dc.description.abstract | Stochastic gradient descent-ascent (SGDA) is one of the main workhorses for solving finite-sum minimax optimization problems. Most practical implementations of SGDA randomly reshuffle components and sequentially use them (i.e., without-replacement sampling); however, there are few theoretical results on this approach for minimax algorithms, especially outside the easier-to-analyze (strongly-)monotone setups. To narrow this gap, we study the convergence bounds of SGDA with random reshuffling (SGDA-RR) for smooth nonconvex-nonconcave objectives with Polyak-{\L}ojasiewicz (P{\L}) geometry. We analyze both simultaneous and alternating SGDA-RR for nonconvex-P{\L} and primal-P{\L}-P{\L} objectives, and obtain convergence rates faster than with-replacement SGDA. Our rates extend to mini-batch SGDA-RR, recovering known rates for full-batch gradient descent-ascent (GDA). Lastly, we present a comprehensive lower bound for GDA with an arbitrary step-size ratio, which matches the full-batch upper bound for the primal-P{\L}-P{\L} case. | - |
dc.language | English | - |
dc.publisher | International Conference on Learning Representations (ICLR) | - |
dc.title | SGDA with shuffling: faster convergence for nonconvex-PŁ minimax optimization | - |
dc.type | Conference | - |
dc.type.rims | CONF | - |
dc.citation.publicationname | 11th International Conference on Learning Representations, ICLR 2023 | - |
dc.identifier.conferencecountry | RW | - |
dc.identifier.conferencelocation | Kigali | - |
dc.contributor.localauthor | Yun, Chulhee | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.