Here we review recent developments in the application of reinforcement-learning theory as a means of understanding how the brain learns to select actions to maximize future reward, with a focus on human neuroimaging studies. We evaluate evidence for the distinction between model-based and model-free reinforcement-learning and their arbitration, and consider hierarchical reinforcement-learning schemes and structure learning. Finally we discuss the possibility of integrating across these different domains as a means of gaining a more complete understanding of how it is the brain learns from reinforcement.