Repository logo
 

Scaling prediction errors to reward variability benefits error-driven learning in humans.


Change log

Authors

Diederen, Kelly MJ 

Abstract

Effective error-driven learning requires individuals to adapt learning to environmental reward variability. The adaptive mechanism may involve decays in learning rate across subsequent trials, as shown previously, and rescaling of reward prediction errors. The present study investigated the influence of prediction error scaling and, in particular, the consequences for learning performance. Participants explicitly predicted reward magnitudes that were drawn from different probability distributions with specific standard deviations. By fitting the data with reinforcement learning models, we found scaling of prediction errors, in addition to the learning rate decay shown previously. Importantly, the prediction error scaling was closely related to learning performance, defined as accuracy in predicting the mean of reward distributions, across individual participants. In addition, participants who scaled prediction errors relative to standard deviation also presented with more similar performance for different standard deviations, indicating that increases in standard deviation did not substantially decrease "adapters'" accuracy in predicting the means of reward distributions. However, exaggerated scaling beyond the standard deviation resulted in impaired performance. Thus efficient adaptation makes learning more robust to changing variability.

Description

Keywords

adaptation, probability distribution, reinforcement learning, risk, standard deviation, Adolescent, Adult, Anticipation, Psychological, Female, Humans, Knowledge of Results, Psychological, Male, Models, Neurological, Models, Statistical, Reward

Journal Title

J Neurophysiol

Conference Name

Journal ISSN

0022-3077
1522-1598

Volume Title

114

Publisher

American Physiological Society
Sponsorship
Wellcome Trust (093270/Z/10/Z)
Wellcome Trust (093875/Z/10/Z)
Wellcome Trust (095495/Z/11/Z)
This work was supported by the Wellcome Trust and the Niels Stensen Foundation.