Exploration biased by former stimulus-response associations due to plasticity in the subthalamic nucleus – external globus pallidus loop in the basal ganglia
Oliver Maith 
Javier Baladron
Fred Hamker
Imagine you want to turn on a computer. As usual you press the on button, but unexpectedly the computer doesn’t start. You may look immediately at the power strip instead of opening the computer to see if all cables are properly seated. Testing well known alternative solutions before others is often effectively and time-saving. In a recent computational study, it was shown that the subthalamic nucleus (STN) – external globus pallidus (GPe) loop in the basal ganglia could play an important role for this cognitive ability [1]. Depending on how the connections in the STN-GPe loop were defined, the loop could bias exploratory decisions, which occur after the learned action isn’t rewarded anymore. The authors hypothesized that these actions could be promising alternatives, such as former rewarded actions. Here we introduce a possible mechanism by which such connectivity patterns in the STN-GPe loop can be learned from former experiences. We took the computational spiking basal ganglia model from the above-mentioned study and implemented a plastic connection from the STN neurons to the GPe neurons. The implemented learning rule consists of a LTP part and a LTD part. The LTP part increases the weights when the postsynaptic neuron activity is below average, and the LTD part decreases the weights when the presynaptic activity is above average. This simple plasticity mechanism leads to a stronger facilitation of previously rewarded actions in future exploration periods. We let the model perform several stimulus-response tasks, in which the rewarded action changed for multiple times. The model learned repeatedly to choose the new rewarded action. In the different tasks we varied how often certain actions become the rewarded action and how long they remain the rewarded action. The results show a significant change in the exploration behavior over time for the model with the plastic STN-GPe connection compared to the model with a fixed STN-GPe connection. The more often actions have been rewarded in the past, the more likely they are to be explored after a rule change and a faster behavioral switch is achieved. Also, the response times of action selections during the exploration periods change due to the plastic STN-GPe connection. Response times decrease for exploration of former rewarded actions and increase for actions that rarely or never have been rewarded.
Back to topMaith, O., Baladron, J., & Hamker, F. (2019). Exploration biased by former stimulus-response associations due to plasticity in the subthalamic nucleus – external globus pallidus loop in the basal ganglia. Bernstein Conference 2019. doi: 10.12751/nncn.bc2019.0052