Comprehensive Computational Model of ACC: Expected Value of Control

Figure 1: Example of cognitive control failure

A new comprehensive computational model of dorsal anterior cingulate cortex function (dACC) was published in last week's issue of Neuron, sending shockwaves throughout the computational modeling community and sending computational modelers running to neuroscience magazinestands in droves. (That's right, I used the word droves - and you know I reserve that word only for special cases.)

The new model, published by Shenhav, Botvinick, and Cohen, attempts to unify existing models and empirical data of dACC function by modifying the traditional monitoring role usually ascribed to the dACC. In previous models of dACC function, such as error detection and conflict monitoring, the primary role of the dACC was that of a monitor involved in detecting errors, or monitoring for mutually exclusive responses and signaling the need to override prepotent but potentially wrong responses. The current model, on the other hand, suggests that the dACC monitors the expected value associated with certain responses, and weighs the potential cost of recruiting more cognitive control against the potential value (e.g., reward or other positive outcome) for implementing cognitive control.

This kind of tradeoff is best illustrated with a basic task like the Stroop task, where a color word - such as "green" - is presented in an incongruent ink, such as red. The instructions in this task are to respond to the color, and not the word; however, this is difficult since reading a word is an automatic process. Overriding this automatic tendency to respond to the word itself requires cognitive control, or strengthening task-relevant associations - in this case, focusing more on the color and not the word itself.

However, there is a drawback: using cognitive control requires effort, and effort isn't always pleasant. Therefore, it stands to reason that the positives for expending this mental effort should outweigh the negatives of using cognitive control. The following figure shows this as a series of meters with greater cognitive control going from left to right:

Figure 1B from Shenhav et al, 2013

As the meters for control signal intensity increase, so does the probability of choosing the correct option that will lead to positive feedback, as shown by the increasing thickness of the arrows from left to right. The role of the dACC, according to the model, is to make sure that the amount of cognitive control implemented is optimal: if someone always goes balls-to-the-wall with the amount of cognitive control they bring to the table, they will probably expend far more energy then would be necessary, even though they would have a much higher probability of being correct every time. (Study question: Do you know anybody like this?) Thus, the dACC attempts to reach a balance between the cognitive control needed and the value of the outcome, as shown in the middle column of the above figure.

This balance is referred to as the expected value of control (EVC): the difference between control costs and outcome values you can expect for a range of control signal intensities. The expected value can be plotted as a curve integrating both the costs and benefits of increased control, with a clear peak at the level of intensity that maximizes the difference between the expected payoff and control cost (Figure 2):

EVC curves (in blue) integrating costs and payoffs for control intensity. (Reproduced from Figure 4 from Shenhav et al, 2013)

That, in very broad strokes, is the essence of the EVC model. There are, of course, other aspects to it, including a role for the dACC in choosing the control identity which orients toward the appropriate behavior and response-outcome associations (for example, actually paying attention to the color of the stroop stimulus in the first place), which can be read about in further detail in the paper. Overall, the model seems to strike a good balance between complexity and conciseness, and the equations are relatively straightforward and should be easy to implement for anyone looking to run their own simulations.

So, the next time you see a supermodel in a bathtub full of Nutella inviting you to join her, be aware that there are several different, conflicting impulses being processed in your dorsal anterior cingulate. To wit, 1) How did this chick get in my bathtub? 2) How did she fill it up with Nutella? Do they sell that stuff wholesale at CostCo or something? and 3) What is the tradeoff between exerting enough control to just say no, given that eating that much chocolate hazelnut spread will cause me to be unable to move for the next three days, and giving in to temptation? It is a question that speaks directly to the human condition; between abjuring gluttony and the million ailments that follow on vice, and simply giving in, dragging that broad out of your bathtub and toweling the chocolate off her so you don't waste any of it, showing her the door, and then returning to the tub and plunging your insatiable maw into that chocolatey reservoir of bliss, that muddy fountain of pleasure, and inhaling pure ecstasy.

Video Games Can Increase Cognitive Ability, Make You Dangerous

As an adult, when I look back on my childhood and consider the ungodly number of hours I put into video games - Command & Conquer, Counter-Strike, Diablo II, Starcraft, Halo, Legend of Zelda, Tetris, Space Quest, Myst, Civilization, just to name a few - I shake my head in disbelief at how much time I frittered away. If I had spent half that amount of time in the gym, for example, I would have been one buff mamma jamma. Instead, all I have to show for it are several deeply ingrained but practically useless motor reflexes - such as the ability to buy an AWP and two flashbang grenades along with kevlar (but no helmet) in record time - and perfect recall of lines of dialogue from Metal Gear Solid and the finer plot points in Final Fantasy VII. In addition, I also developed a thick skin in response to the flood of insults, invectives, put-downs, and unending verbal abuse from other online players with names like AznMaGiC and Legali$e_iT. At the time, all of this seemed incredibly important; now, not so much.

However, a recent study suggests that this may not have all been an entire waste; and that video games relevant to specific cognitive processes, such as attention, spatial working memory, and decision-making, can actually improve these functions and show crossover to different cognitive domains.

A research group from Singapore recruited a sample of non-gamers - people I once would have looked down upon with contempt - and had them play a variety of different games, such as Bejeweled, Hidden Expedition, and The Sims (remember that?). The participants were tested on a battery of tasks tapping into abilities such as working memory and filtering out distracting stimuli. After completing these measures, subjects were randomly assigned to play one of five games (for a total of five groups), and to play that game for an hour a day over the course of four weeks. The subjects were then retested on the same tasks as before.

Participants improved markedly on those tasks most related to the game that they played - for example, those who spent time playing a memory matrix game showed significant improvement in the working-memory task. However, there was some cross-over between tasks as well, such as action-game playing associated with both improved filtering of task-irrelevant stimuli and increased ability to track multiple objects. Overall, the findings corroborated other studies focused on habitual gamers who primarily played first-person shooters, but showed that these improvements could be extended to non-gamers and to non-violent games. Taken together, this suggests that daily video game practice can both improve your cognitive abilities, and also make you a more efficient hunter of the most dangerous prey of all - man.

So, should we all start shelling out more money for video games and less money for books, poetry, and exercise equipment, such as whiffle balls? Not quite. First, there was no mention of how long the effects lasted - whether it was just for the post-test, or whether the effects could last for weeks or months. Second, although there was a considerable interval between the pre-test and post-test (one month), since this was a repeated-measures design, there is the possibility that there may have been some carryover effect; i.e., some effect of practice from the pre-test. Although there were significant differences between groups, it is unclear how this was affected by the testing before the training period. Lastly, there was no mention about playing an hour of video games a day affected other aspects of life, such as proportionally less time devoted to exercising, social interaction, and your girlfriend getting pissed off that you forgot to pick her up for your two-year anniversary date because you got so caught up in one of those three-hour Metal Gear Solid cutscenes that time seemed to stand still. Chicks, they'll never understand. (Guys, amiright?).

The paper can be found here. And, in case you were wondering, deals for the latest Starcraft II expansion can be found here.

Lesion Studies: Recent Trends

A couple weeks ago I blogged hard about a problem presented by lesion studies of the anterior cingulate cortex (ACC), a broad swath of cortex shown to be involved in aspects of cognitive control such as conflict monitoring (Botvinick et al, 2001) and predicting error likelihood (Brown & Braver, 2005). Put simply: The annihilation of this region, either through strokes, infarctions, bullet wounds, or other cerebral insults, does not result in a deficit of cognitive control, as measured through reaction time (RT) in response to a variety of tasks, such as Stroop tasks - a task that requires overriding prepotent responses to a word presented on a screen, as opposed to the color of the ink that the word is written in - and paradigms which involve task-switching.

In particular, a lesion study by Fellows & Farah (2005) did not find a significant RT interaction of group (either controls or lesion patients) by condition (either low or high conflict in a Stroop task; i.e., either the word and ink color matched or did not match), suggesting that the performance of the lesion patients was essentially the same as the performance of controls. This in turn prompted the question of whether the ACC was really necessary for cognitive control, since those without it seemed to do just fine, and were about a pound lighter to boot. (Rimshot)

However, a recent study by Sheth et al (2012) in Nature examined six lesion patients undergoing cingulotomy, a surgical procedure which removes a localized portion of the dorsal anterior cingulate (dACC) in order to alleviate severe obsessive-compulsive symptoms, such as the desire to compulsively check the amount of hits your blog gets every hour. Before the cingulotomy, the patients performed a multisource interference task designed to elicit cognitive control mechanisms associated with dACC activation. The resulting cingulotomy overlapped with the peak dACC activation observed in response to high-conflict as contrasted with low-conflict trials (Figure 1).

Figure 1 reproduced from Sheth et al (2012). d) dACC activation in response to conflict. e) arrow pointing to lesion site

Furthermore, the pattern of RTs before surgery followed a typical response pattern replicated over several studies using this task: RTs were faster for trials immediately following trials of a similar type - such as congruent trials following congruent trials, or incongruent trials following incongruent trials - and RTs were slower for trials which immediately followed trials of a different type, a pattern known as the Gratton effect.

The authors found that global error rates and RTs were similar before and after the surgery, dovetailing with the results reported by Fellows & Farah (2005); however, the modulation of RT based on previous trial congruency or incongruency was abolished. These results suggest that the ACC functions as a continuous updating mechanism modulating responses based on the weighted past and on trial-by-trial cognitive demands, which fits into the framework posited by Dosenbach (2007, 2008) that outlines the ACC as part of a rapid-updating cingulo-opercular network necessary for quick and flexible changes in performance based on task demands and performance history.

a) Pre-surgical RTs in response to trials of increasing conflict. b, c) Post-surgical RTs showing no difference between low-conflict trials preceded by either similar or different trial types (b), and no RT difference between high-conflict trials preceded by either similar or different trial types (c).

Above all, this experiment illustrates how lesion studies ought to be conducted. First, the authors identified a small population of subjects about to undergo a localized surgical procedure to lesion a specific area of the brain known to be involved in cognitive control; the same subjects were tested before the surgery using fMRI and during surgery using single-cell recordings; and interactions were tested which had been overlooked by previous lesion studies. It is an elegant and simple design; although I imagine that testing subjects while they had their skulls split open and electrodes jammed into their brains was disgusting. The things that these sickos will do for high-profile papers.

(This study may be profitably read in conjunction with a recent meta-analysis of lesion subjects (Gläscher et al, 2012; PNAS) dissociating cortical structures involved in cognitive control as opposed to decision-making and evaluation tasks. I recommend giving both of these studies a read.)

Lesion Studies: Thoughts

(Note: I recently completed my candidacy exam, which involved writing a trio of papers focusing on different aspects of my research. Most of this post is cannibalized from a section I wrote on lesion studies of the anterior cingulate cortex, which produce counterintuitive results when contrasted to lesions of other areas, such as the DLPFC and OFC, which do indeed seem to disrupt the processes that those regions are implicated in from the neuroimaging literature.

My work primarily involves healthy people with intact brains, and observing indirect measures of neural firing through tracking slow blood flow changes in the brain. However, "activation" as defined by fMRI is not the same as the underlying neural dynamics, and, barring invasive single-cell recordings, we have few options for directly measuring neural firing in response to different tasks and psychological contexts. This caveat inherent in fMRI research becomes particularly important when interpreting the results of lesion studies.) 

Although the majority of the neuroimaging literature has implicated the dACC as playing a critical role in the signaling for cognitive control when necessary, the most direct test of a brain structure’s necessity in a cognitive process is through examining subjects presenting with lesions in that part of the brain. For example, if it can be demonstrated that a subject without an ACC still performs equivalent to controls on tasks involving cognitive control, then that would argue against the necessity of that area’s involvement in the hypothesized cognitive process. Studies involving human subjects with lesions are relatively rare and suffer from low power, but can still reveal important aspects of neural functioning.

The ACC, in particular, has been the subject of several lesion studies that have shown conflicting and counterintuitive results. For example, a single-subject lesion study of a patient with left ACC damage exhibited both smaller ERNs and increased RT in response to incongruent stimuli in a spatial Stroop paradigm. This study showed that conflict monitoring and error detection, at least in this patient, do not both come from the same area of ACC, suggesting that these processes occur in different areas. However, while the ERN was shown to be attenuated in the patient, the conflict response (a waveform called the N450) was actually enhanced (Swick & Turken, 2002). This suggests that conflict monitoring occurs in a nearby prefrontal area, such as the DLPFC, before information about the conflict is sent to the ACC.

Figure of the lesion for the single subject analyzed by Turken & Swick (2002). Overlaid are coordinates of peak activation for conflict-related tasks from other studies.

On the other hand, a lesion study conducted by Fellows & Farah (2005) compared the performance of individuals with dACC lesions to that of controls across a battery of tasks hypothesized to involve cognitive control. These tasks included a Stroop task and a go-nogo task which are known to elicit significantly greater increases in RT after errors, and to induce significantly greater amounts of errors during incongruent trials. The results showed no significant interactions between group and task, suggesting that the dACC is not necessary for the implementation of cognitive control. Furthermore, the authors pointed out that tasks involving cognitive control may be confounded with emotional responding, which in turn could simply be associated with the ACC's involvement in regulating muscle tone. In any case, it is apparent that although this structure is somehow associated with cognitive control, it is not strictly necessary for it. 

Figure showing group overlap of lesions in the Fellows & Farah (2005) study.  Circles and squares represent an overlay of a meta-analysis by Bush et al (2000), with circles representing peak activations for cognitive tasks, and squares representing peak activations for emotional tasks.

Comparison of Stroop effect (measured in percent signal change from mean congruent trial RT) and error rate between lesion patients and controls. No significant difference was found on either measure between the two groups.

In sum, these lesion studies suggest that the dACC may not be indispensable for signaling the DLPFC to implement cognitive control. However an alternative explanation is that patients with ACC lesions are usually ipsilateral, and that furthermore they may be compensating for required cognitive control by recruiting nearby cortical areas. However, two lines of evidence argue against this interpretation. First, one of the lesion patients examined in the Fellows & Farah (2005) had extensive medial ACC damage encompassing dACC bilaterally, but showed a similar pattern of error rates and RT difference between congruent and incongruent conditions as did the other lesion patients and the control group. Secondly, lesion studies of other areas of the brain – such as the orbitofrontal cortex – have shown that those regions appear to be specific to the cognitive processes they are hypothesized to be involved in. For example, patients with OFC lesions exhibit significantly impaired performance in decision-making tasks such as the Iowa Gambling Task and Wisconsin Card Sorting Task, as well as decreased autonomic activity in response to highly risky gambles (Bechara et al, 1994). Even though the patients in this study had suffered from their lesions for a comparable amount of time as the lesion subjects in the Fellows & Farah (2005) study, there was no evidence of recruitment of other cortical areas in order to support their deficits in decision-making.

However, although these lesion studies have shown no significant differences in error rates between the lesion patients and controls, other experiments have revealed that patients with ACC damage are less likely to correct for their mistakes on trials immediately following an error. In addition, patients with ACC lesions are less likely to be aware that an error has occurred (Swick & Turken, 2002). These results suggest that there may be a necessary role for of the ACC for the actual detection of errors, which would be consistent with the hypothesis that this area is involved in the comparison of actions against their predicted outcomes. How lesions affect the transfer of information from the ACC to the DLPFC and other cortical regions supposedly involved in the implementation of cognitive control, however, is less well understood.

Bottom line: If the inferences from neuroimaging studies are to believed, then the ACC is necessary somehow for cognitive control or executive function; however, lesion studies belie this claim, suggesting perhaps that the necessary processes for these cognitive functions take place elsewhere and merely light up the ACC as some sort of epiphenomenon. Admittedly, I am unsure of what to make of all this. The most useful experiments to carry out, in my opinion, would be to apply transcranial magnetic stimulation (TMS) to temporarily knock out this area in healthy controls, and then observe what happens; however, as TMS is only able to disrupt neural firing on surface areas of the cortex, stimulation of deeper areas remains impractical. With continuing advances in the ability of TMS to stimulate deeper cortical (and, possibly, subcortical?) structures, we may get a better grasp of what is going on.