We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Timing, or the decision of when to act, is essential to mammalian behaviors from escaping predators to driving a car. It requires cognitive functions such as working memory for time-based rules and attention to the passing of time. Thus, it can be used as a proxy for higher order executive functions that are difficult to measure but are impaired in many neurological disorders. Therefore, insights from studies of interval timing, tasks which require estimating time intervals of several seconds, have great value for our understanding of human disease. Crucial to timing is the basal ganglia, which integrates cortical activity with midbrain dopamine signals and sends out signals to the spinal cord that regulate movement, motivation, and other behaviors. We have previously found that within the basal ganglia, medium spiny neurons of the striatum exhibit ramping activity in time-related tasks. In other words, they gradually increase or decrease firing frequency across a timed interval, and this is thought to encode time. Yet it is still unknown how the encoding of time is translated into time-based motor responses. To answer this question, we turned to the external globus pallidus (GPe) because it is a regulatory hub within the basal ganglia and is thus well positioned to regulate timing behavior. We sought to examine how the GPe functions in response to time-based demands.
Participants and Methods:
We recorded from neuronal ensembles using 16 channel electrode arrays implanted in the GPe of five mice while they performed an interval timing task called the switch interval timing task. Spike sorting was then used to identify signal from individual neurons.
Results:
Data were compiled from 43 neurons over several trials. Principal component analysis of neural firing activity was then conducted and revealed a downward ramping pattern in GPe neurons during interval timing trials. Data were then separated based on trials in which mice made correct decisions and those in which mice made a mistake. We found that when mice make correct timing decisions, there is downward ramping activity in the GPe, yet when mice make timing mistakes, this ramping pattern is lost.
Conclusions:
Our findings suggest that the GPe processes timing signals through ramping activity, before projecting to the output nuclei of the basal ganglia. This is a novel finding and contributes to a growing understanding of the temporal code of the basal ganglia. The full extent of this code is still unknown, but this insight contributes to a better understanding of how the globus pallidus represents cognition. If we can better explain the neural correlates of timing, we can use this knowledge to inform therapeutic interventions for basal ganglia dysfunction, which could have profound implications for diseases like Parkinson’s disease, which affects millions around the world.
The focus of this chapter is on neurobiologically informed and constrained models of working memory as defined by Miller, Galanter, and Pribram (1960): the holding of goals and subgoals in mind in service of planning and executing complex behaviors. In particular, the chapter focuses on models specifically addressing critical challenges and mechanisms following from the need for rapid and selective gating of working memory contents. To start, the important computational challenges posed by the tradeoff between maintaining vs. updating are discussed, providing motivation for the rest of the chapter.After that, several seminal models that have contributed to current thinking are reviewed, including the authors’ own PBWM framework that has proven influential. Finally, several recent developments from the deep learning and neurophysiology literatures are addressed and critical questions and some directions for future progress are discussed.
Reinforcement learning (RL) is a computational framework for an active agent to learn behaviors on the basis of a scalar reward feedback. The theory of reinforcement learning was developed in the artificial intelligence community with intuitions from psychology and animal learning theory and mathematical basis in control theory. It has been successfully applied to tasks like game playing and robot control. Reinforcement learning gives a theoretical account of behavioral learning in humans and animals and underlying brain mechanisms, such as dopamine signaling and the basal ganglia circuit. Reinforcement learning serves as the “common language” for engineers, biologists, and cognitive scientists to exchange their problems and findings in goal-directed behaviors. This chapter introduces the basic theoretical framework of reinforcement learning and reviews its impacts in artificial intelligence, neuroscience, and cognitive science.
This chapter first reviews advanced methods in reinforcement learning (RL), namely, hierarchical RL, distributional RL, meta-RL, RL as inference, inverse RL, and multi-agent RL. Computational and cognitive models based on reinforcement learning are then presented, including detailed models of the basal ganglia, variety of dopamine neuron responses, roles of serotonin and other neuromodulators, intrinsic reward and motivation, neuroeconomics, and computational psychiatry.
Quantitative susceptibility mapping (QSM) demonstrates elevated iron content in Parkinson’s disease (PD) patients within the basal ganglia, though it has infrequently been studied in relation to gait difficulties including freezing of gait (FOG). Our purpose was to relate QSM of basal ganglia and extra-basal ganglia structures with qualitative and quantitative gait measures in PD.
Methods:
This case–control study included PD and cognitively unimpaired (CU) participants from the Comprehensive Assessment of Neurodegeneration and Dementia study. Whole brain QSM was acquired at 3T. Region of interests (ROIs) were drawn blinded manually in the caudate nucleus, putamen, globus pallidus, pulvinar nucleus of the thalamus, red nucleus, substantia nigra, and dentate nucleus. Susceptibilities of ROIs were compared between PD and CU. Items from the FOG questionnaire and quantitative gait measures from PD participants were compared to susceptibilities.
Results:
Twenty-nine participants with PD and 27 CU participants were included. There was no difference in susceptibility values in any ROI when comparing CU versus PD (p > 0.05 for all). PD participants with gait impairment (n = 23) had significantly higher susceptibility in the putamen (p = 0.008), red nucleus (p = 0.01), and caudate nucleus (p = 0.03) compared to those without gait impairment (n = 6). PD participants with FOG (n = 12) had significantly higher susceptibility in the globus pallidus (p = 0.03) compared to those without FOG (n = 17). Among quantitative gait measures, only stride time variability was significantly different between those with and without FOG (p = 0.04).
Conclusion:
Susceptibilities in basal ganglia and extra-basal ganglia structures are related to qualitative measures of gait impairment and FOG in PD.
The motor thalamus is interconnected with the brainstem, cortex, and basal ganglia and plays major roles in planning, sequencing, and executing action. In this chapter, I highlight roles of input-defined thalamic circuits in motor sequence production and learning. Brainstem–motor thalamic pathways carry efference copy signals important for the production of both innate and learned motor sequences, for example, during saccades, grooming, and birdsong. Basal ganglia thalamocortical loops implement aspects of reinforcement learning, including the generation of motor exploration during vocal babbling. Classic "gating" models of basal ganglia–thalamic transmission fail to explain thalamic discharge during behavior, which instead appears strongly driven by cortical inputs. A challenge going forward is to determine if there are conserved principles of thalamic function across diverse motor thalamic subregions.
The higher-order thalamus (e.g., the pulvinar) is widely thought to play a critical role in its interactions with the neocortex, but identifying precisely what that role is has been somewhat challenging.Here, we describe how a computational approach to understanding the nature of learning and memory in the neocortex suggests three distinct, well-defined contributions of the thalamus: (1) attention, which is perhaps the most widely discussed function of the pulvinar, is supported by a pooled inhibition dynamic involving the thalamic reticular nucleus; (2) predictive learning, where the pulvinar serves as a kind of screen on which predictions are projected, and a temporal difference between predictions and subsequent outcomes can drive error-driven learning throughout the thalamocortical system; and (3) executive function in the circuits involving the frontal cortex, where the mediodorsal (MD) thalamus is largely similar anatomically to the pulvinar and could thus support similar attentional and predictive learning functions, whereas ventral thalamic nuclei receive inhibitory modulation from the basal ganglia, supporting a gating function to regulate action based on a strong competition of Go versus No Go informed by reinforcement learning.Taken together, these important modulatory and learning contributions of the thalamus suggest that a full computational understanding of the neocortex is significantly incomplete without an integration of the thalamic circuitry.
Anxiety can interfere with attention and working memory, which are components that affect learning. Statistical models have been designed to study learning, such as the Bayesian Learning Model, which takes into account prior possibilities and behaviours to determine how much of a new behaviour is determined by learning instead of chance. However, the neurobiological basis underlying how anxiety interferes with learning is not yet known. Accordingly, we aimed to use neuroimaging techniques and apply a Bayesian Learning Model to study learning in individuals with generalised anxiety disorder (GAD).
Methods.
Participants were 25 controls and 14 individuals with GAD and comorbid disorders. During fMRI, participants completed a shape-button association learning and reversal task. Using a flexible factorial analysis in SPM, activation in the dorsolateral prefrontal cortex, basal ganglia, and hippocampus was compared between groups during first reversal. Beta values from the peak of these regions were extracted for all learning conditions and submitted to repeated measures analyses in SPSS.
Results.
Individuals with GAD showed less activation in the basal ganglia and the hippocampus only in the first reversal compared with controls. This difference was not present in the initial learning and second reversal.
Conclusion.
Given that the basal ganglia is associated with initial learning, and the hippocampus with transfer of knowledge from short- to long-term memory, our results suggest that GAD may engage these regions to a lesser extent during early accommodation or consolidation of learning, but have no longer term effects in brain activation patterns during subsequent learning.
Iron plays a key role in a broad set of metabolic processes. Iron deficiency is the most common nutritional deficiency in the world, but its neuropsychiatric implications in adolescents have not been examined.
Methods
Twelve- to 17-year-old unmedicated females with major depressive or anxiety disorders or with no psychopathology underwent a comprehensive psychiatric assessment for this pilot study. A T1-weighted magnetic resonance imaging scan was obtained, segmented using Freesurfer. Serum ferritin concentration (sF) was measured. Correlational analyses examined the association between body iron stores, psychiatric symptom severity, and basal ganglia volumes, accounting for confounding variables.
Results
Forty females were enrolled, 73% having a major depressive and/or anxiety disorder, 35% with sF < 15 ng/mL, and 50% with sF < 20 ng/mL. Serum ferritin was inversely correlated with both anxiety and depressive symptom severity (r = −0.34, p < 0.04 and r = −0.30, p < 0.06, respectively). Participants with sF < 15 ng/mL exhibited more severe depressive and anxiety symptoms as did those with sF < 20 ng/mL. Moreover, after adjusting for age and total intracranial volume, sF was inversely associated with left caudate (Spearman's r = −0.46, p < 0.04), left putamen (r = −0.58, p < 0.005), and right putamen (r = −0.53, p < 0.01) volume.
Conclusions
Brain iron may become depleted at a sF concentration higher than the established threshold to diagnose iron deficiency (i.e. 15 ng/mL), potentially disrupting brain maturation and contributing to the emergence of internalizing disorders in adolescents.
Progressive brain structural MRI changes are described in schizophrenia and have been ascribed to both illness progression and antipsychotic treatment. We investigated treatment effects, in terms of total cumulative antipsychotic dose, efficacy and tolerability, on brain structural changes over the first 24 months of treatment in schizophrenia.
Methods
A prospective, 24-month, single-site cohort study in 99 minimally treated patients with first-episode schizophrenia, schizophreniform and schizoaffective disorder, and 98 matched healthy controls. We treated the patients according to a fixed protocol with flupenthixol decanoate, a long-acting injectable antipsychotic. We assessed psychopathology, cognition, extrapyramidal symptoms and BMI, and acquired MRI scans at months 0, 12 and 24. We selected global cortical thickness, white matter volume and basal ganglia volume as the regions of interest.
Results
The only significant group × time interaction was for basal ganglia volumes. However, patients, but not controls, displayed cortical thickness reductions and increases in white matter and basal ganglia volumes. Cortical thickness reductions were unrelated to treatment. White matter volume increases were associated with lower cumulative antipsychotic dose, greater improvements in psychopathology and cognition, and more extrapyramidal symptoms. Basal ganglia volume increases were associated with greater improvements in psychopathology, greater increases in BMI and more extrapyramidal symptoms.
Conclusions
We provide evidence for plasticity in white matter and basal ganglia associated with antipsychotic treatment in schizophrenia, most likely linked to the dopamine blocking actions of these agents. Cortical changes may be more closely related to the neurodevelopmental, non-dopaminergic aspects of the illness.
Neurodegeneration with brain iron accumulation (NBIA) is a term used for a group of hereditary neurological disorders with abnormal accumulation of iron in basal ganglia. It is clinically and genetically heterogeneous with symptoms such as dystonia, dysarthria, Parkinsonism, intellectual disability, and spasticity. The age at onset and rate of progression are variable among individuals. Current therapies are exclusively symptomatic and unable to hinder the disease progression. Approximately 16 genes have been identified and affiliated to such condition with different functions such as iron metabolism (only two genes: Ferritin Light Chain (FTL) Ceruloplasmin (CP)), lipid metabolism, lysosomal functions, and autophagy process, but some functions have remained unknown so far. Subgroups of NBIA are categorized based on the mutant genes. Although in the last 10 years, the development of whole-exome sequencing (WES) technology has promoted the identification of disease-causing genes, there seem to be some unknown genes and our knowledge about the molecular aspects and pathogenesis of NBIA is not complete yet. There is currently no comprehensive study about the NBIA in Iran; however, one of the latest discovered NBIA genes, GTP-binding protein 2 (GTPBP2), has been identified in an Iranian family, and there are some patients who have genetically remained unknown.
The symptoms of obsessive–compulsive disorder (OCD) are suggestive of cognitive rigidity, and previous work identified impaired flexible responding on set-shifting tasks in such patients. The basal ganglia are central to habit learning and are thought to be abnormal in OCD, contributing to inflexible, rigid habitual patterns of behaviour. Here, we demonstrate that increased cognitive inflexibility, indexed by poor performance on the set-shifting task, correlated with putamen morphology, and that patients and their asymptomatic relatives had common curvature abnormalities within this same structure. The association between the structure of the putamen and the extradimensional errors was found to be significantly familial in OCD proband–relative pairs. The data implicate changes in basal ganglia structure linked to cognitive inflexibility as a familial marker of OCD. This may reflect a predisposing heightened propensity toward habitual response patterns and deficits in goal-directed planning.
The context-based selection of semantic representations is presented as an essential issue in understanding the interpretation of speech meaning. Clinical findings serve to illustrate the role of the thalamus, the basal ganglia, and the cerebellum in processing subtle context-related differences in meaning. The paucity of findings relating to spoken language is emphasized along with the need to specify neuropragmatic principles of context-based activations of semantic and episodic representations.With a view on developing such principles, several relevant findings are reviewed relating to cortico-thalamic interactions and the pivotal role of the motor thalamus in integrating multisensory information from basal-ganglia circuits and the cerebellum. The on-line selection of semantic and episodic representations is also discussed in terms of experiments on the role of the hippocampus and frontal circuits suggesting some parallels with navigation, but the on-line processing of speech requires a chunking of action-related sequences which appears to involve the basal ganglia and critical cortico-thalamic loops.
Why do we run toward people we love, but only walk toward others? Why do people in New York seem to walk faster than other cities? Why do our eyes linger longer on things we value more? There is a link between how the brain assigns value to things, and how it controls our movements. This link is an ancient one, developed through shared neural circuits that on one hand teach us how to value things, and on the other hand control the vigor with which we move. As a result, when there is damage to systems that signal reward, like dopamine and serotonin, that damage not only affects our mood and patterns of decision-making, but how we move. In this book, we first ask why, in principle, evolution should have developed a shared system of control between valuation and vigor. We then focus on the neural basis of vigor, synthesizing results from experiments that have measured activity in various brain structures and neuromodulators, during tasks in which animals decide how patiently they should wait for reward, and how vigorously they should move to acquire it. Thus, the way we move unmasks one of our well-guarded secrets: how much we value the thing we are moving toward.
To investigate the frequency of bradykinesia in patients with obsessive-compulsive disorder (OCD) and to see whether patients with OCD who also have bradykinesia display distinctive neuropsychological and neuropsychiatric features.
Methods
We studied 23 antipsychotic-free patients with OCD and 13 healthy controls. Bradykinesia was assessed with section III of the Unified Parkinson Disease Rating Scale. The Wechsler Adult Intelligent Scales-Revised (WAIS-R) was used to assess the Full Scale IQ and to measure visuospatial, visuoconstructional ability and psychomotor speed/mental slowness.
Results
Of the 23 patients with OCD studied, 8 (34%) had mild symptoms of bradykinesia. No relationship was found between bradykinesia and the sociodemographic variables assessed but this motor symptom was significantly associated with the severity of compulsions. Patients with bradykinesia differed from those without: they had a higher frequency of repeating compulsions, and lower IQ scores, performance scores, and WAIS-R subtest scores for similarities and picture completion. No significant differences were found between patients without bradykinesia and healthy controls in any test.
Conclusions
Clinical assessment of motor symptoms in adult patients with OCD often discloses mild bradykinesia sometimes associated with repeating compulsions and poor WAIS-R performance scores.
It has been shown these last years that optogenetic tool, that uses a combination of optics and genetics technics to control neuronal activity with light on behaving animals, allows to establish causal relationship between brain activity and normal or pathological behaviors [3]. In combination with animal model of neuropsychiatric disorder, optogenetic could help to identify deficient circuitry in numerous pathologies by exploring functional connectivity, with a specificity never reached before, while observing behavioral and/or physiological correlates. To illustrate the promising potential of these tools for the understanding of psychiatric diseases, we will present our recent study where we used optogenetic to block abnormal repetitive behavior in a mutant mouse model of obsessive-compulsive disorder [1]. Using a delay-conditioning task we showed that these mutant mouse model had a deficit in response inhibition that lead to repetitive behaviour. With optogenetic, we could stimulate a specific circuitry in the brain that connect the orbitofrontal cortex with the basal ganglia; a circuitry that has been shown to be dysfunctional in compulsive behaviors. We observed that these optogenetic stimulations, through their effect on inhibitory neurons of the basal ganglia, could restore the behavioral response inhibition and alleviate the compulsive behavior. These findings raise promising potential for the design of targeted deep brain stimulation therapy for disorders involving excessive repetitive behavior and/or for the optimization of already existing stimulation protocol [2].