Reinforcement with Fading Memories

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 125
  • Download : 0
We study the effect of imperfect memory on decision making in the context of a stochastic sequential action-reward problem. An agent chooses a sequence of actions, which generate discrete rewards at different rates. She is allowed to make new choices at rate β, whereas past rewards disappear from her memory at rate μ. We focus on a family of decision rules where the agent makes a new choice by randomly selecting an action with a probability approximately proportional to the amount of past rewards associated with each action in her memory. We provide closed form formulas for the agent’s steady-state choice distribution in the regime where the memory span is large (μ→0) and show that the agent’s success critically depends on how quickly she updates her choices relative to the speed of memory decay. If β≫μ, the agent almost always chooses the best action (that is, the one with the highest reward rate). Conversely, if β≪μ, the agent chooses an action with a probability roughly proportional to its reward rate.
Publisher
INFORMS
Issue Date
2020-11
Language
English
Article Type
Article
Citation

MATHEMATICS OF OPERATIONS RESEARCH, v.45, no.4, pp.1258 - 1288

ISSN
0364-765X
DOI
10.1287/moor.2019.1031
URI
http://hdl.handle.net/10203/277436
Appears in Collection
RIMS Journal Papers
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0