Associative Learning and Reinforcement¶
Scope: Learning of stimulus-stimulus, stimulus-response, and action-outcome contingencies under reinforcement; Pavlovian and instrumental conditioning; extinction; reversal; model-based and model-free reinforcement learning; reward prediction error.
Out of scope: Structure learning without scalar reinforcement (that is Implicit and Statistical Learning).
This category contains 13 processes.
Associative learning¶
Process ID: hed_associative_learning
Learning of co-occurrence relations between stimuli or between stimuli and responses.
Tasks that engage this process: Causal Learning Task, Contextual Cueing Task, Digit Symbol Substitution Task, Implicit Association Task, Paired Associates Learning Task, Pavlovian Fear Conditioning Task, Probabilistic Classification Learning Task
Further references
Shanks (2010) Annual Review of Psychology 61:273–301 (DOI)
Extinction¶
Process ID: hed_extinction
Also known as: extinction learning
Decrease in a previously reinforced response when reinforcement is withheld; a form of new inhibitory learning rather than erasure.
Tasks that engage this process: Pavlovian Fear Conditioning Task
Fundamental references
Pavlov (1927) Conditioned reflexes Oxford University Press
Further references
Goal-directed behavior¶
Process ID: hed_goal_directed_behavior
Also known as: goal-directed action
Behavior that is sensitive to current outcome value, characteristic of action–outcome learning.
Tasks that engage this process: Instrumental Conditioning Task
Fundamental references
Dickinson & Balleine (1994) Animal Learning & Behavior 22:1–18 (DOI)
Further references
Habit¶
Process ID: hed_habit
Also known as: habit learning
Behavior that is insensitive to the current value of its outcome, characteristic of stimulus–response learning.
Tasks that engage this process: Instrumental Conditioning Task
Further references
Instrumental conditioning¶
Process ID: hed_instrumental_conditioning
Also known as: Operant conditioning - Skinnerian terminology; emphasizes the operant response and reinforcement schedules.
Learning that an action produces an outcome; also called operant conditioning. Encompasses both goal-directed (action–outcome) and habitual (stimulus–response) control, studied via reinforcement schedules and outcome-devaluation procedures.
Tasks that engage this process: Instrumental Conditioning Task
Further references
Staddon & Cerutti (2003) Annual Review of Psychology 54:115–144 (DOI)
Model-based learning¶
Process ID: hed_model_based_learning
Also known as: model-based reinforcement learning
Reinforcement learning that uses an internal model of the environment’s transition and reward structure to plan.
Tasks that engage this process: Two-Stage Decision Task
Fundamental references
Model-free learning¶
Process ID: hed_model_free_learning
Also known as: model-free reinforcement learning
Reinforcement learning from cached value estimates updated by prediction errors, without an explicit model of the environment.
Tasks that engage this process: Two-Stage Decision Task
Fundamental references
Pavlovian conditioning¶
Process ID: hed_pavlovian_conditioning
Also known as: classical conditioning; Pavlovian
Learning that a neutral stimulus predicts a biologically significant outcome, leading to conditioned responding.
Tasks that engage this process: Pavlovian Fear Conditioning Task
Fundamental references
Pavlov (1927) Conditioned reflexes Oxford University Press
Further references
LeDoux (2014) PNAS 111:2871–2878 (DOI)
Policy learning¶
Process ID: hed_policy_learning
Direct learning of a mapping from states to actions without necessarily estimating values.
Tasks that engage this process: none in the current Catalog.
Reinforcement learning¶
Process ID: hed_reinforcement_learning
Learning to select actions that maximize cumulative reward through experience with reward prediction errors.
Tasks that engage this process: Instrumental Conditioning Task, Iowa Gambling Task, Multi-Armed Bandit Task, Probabilistic Classification Learning Task, Probabilistic Selection Task, Reversal Learning Task, Two-Stage Decision Task
Fundamental references
Schultz, Dayan & Montague (1997) Science 275:1593–1599 (DOI)
Further references
Niv (2009) Journal of Mathematical Psychology 53:139–154 (DOI)
Reversal learning¶
Process ID: hed_reversal_learning
Relearning after contingencies between stimuli (or responses) and outcomes are switched.
Tasks that engage this process: Reversal Learning Task
Fundamental references
Reward prediction error¶
Process ID: hed_reward_prediction_error
Signed difference between received and expected reward, instantiated by phasic midbrain dopamine firing.
Tasks that engage this process: Causal Learning Task, Instrumental Conditioning Task, Iowa Gambling Task, Multi-Armed Bandit Task, Probabilistic Selection Task, Reversal Learning Task, Two-Stage Decision Task
Fundamental references
Schultz, Dayan & Montague (1997) Science 275:1593–1599 (DOI)
Further references
Glimcher (2011) PNAS 108(Suppl 3):15647–15654 (DOI)
Value learning¶
Process ID: hed_value_learning
Acquisition of the expected value of stimuli, actions, or states from experience with outcomes.
Tasks that engage this process: Multi-Armed Bandit Task, Probabilistic Selection Task
Fundamental references