AFNI Part 1: Introduction

As promised, we now begin our series of AFNI tutorials. These walkthroughs will be more in-depth than the FSL series, as I am more familiar with AFNI and use it for a greater number of tasks; accordingly, more advanced tools and concepts will be covered.

Using AFNI requires a solid understanding of Unix; the user should know how to write and read conditional statements and for loops, as well as know how to interpret scripts written by others. Furthermore, when confronted with a new or unfamiliar script or command, the user should be able to make an educated guess about what it does. AFNI also demands a sophisticated knowledge of fMRI preprocessing steps and statistical analysis, as AFNI allows the user more opportunity to customize his script.

A few other points about AFNI:

1) There is no release schedule. This means that there is no fixed date for the release of new versions or patches; rather, AFNI responds to user demands on an ad hoc basis. In a sense, all users are beta testers for life. The advantage is that requests are addressed quickly; I once made a feature request at an AFNI bootcamp, and the developers updated the software before I returned home the following week.

2) AFNI is almost entirely run from the command line. In order to make the process less painful, the developers have created "uber" scripts which allow the user to input experiment information through a graphical user interface and generate a preprocessing script. However, these should be treated as templates subject to further alteration.

3) AFNI has a quirky, strange, and, at times, shocking sense of humor. Through clicking on a random hotspot on the AFNI interface, one can choose their favorite Shakespeare sonnet; read through the Declaration of Independence; generate an inspirational quote or receive kind and thoughtful parting words. Do not let this deter you. As you become more proficient with AFNI, and as you gain greater life experience and maturity, the style of the software will become more comprehensible, even enjoyable. It is said that one knows he going insane when what used to be nonsensical gibberish starts to take on profound meaning. So too with AFNI.

The next video will cover the to3d command and the conversion of raw volumetric data into AFNI's BRIK/HEAD format; study this alongside data conversion through mricron, as both produce a similar result and can be used to complement each other. As we progress, we will methodically work through the preprocessing stream and how to visualize the output with AFNI, with an emphasis on detecting artifacts and understanding what is being done at each step. Along the way different AFNI tools and scripts will be broken down and discussed.

At long last my children, we shall take that which is rightfully ours. We shall become as gods among fMRI researchers - wise as serpents, harmless as doves. Seek to understand AFNI with an open heart, and I will gather you unto my terrible and innumerable flesh and hasten your annihilation.

Unix for Neuroimagers: For Loops and Shell Scripting

With this tutorial we begin to enter into AFNI territory, as the more advanced Unix commands and procedures are necessary for running an AFNI analysis. First, we cover the basics of for loops. For loops in fMRI analysis are used to loop over subjects and execute preprocessing and postprocessing scripts for each subject individually. (By the way, I've heard of preprocessing and I've heard of postprocessing; but where does plain, regular processing take place? Has anyone ever thought of that? Does it ever happen at all? I feel like I'm taking crazy pills here.)

Within the t-shell (and I use the t-shell here because it is the default shell used by AFNI), a for loop has the following syntax:


foreach [variableName] ( [var1] [var2]...[varN])
     
     [Executable steps go here]

end


So, for example, if I wanted to loop over a list of values and have them assigned to variable "x" and then return those values, I could do something like the following:

foreach x (1 2 3 4 5)
    
     echo $x

end

Which would return the following:

1
2
3
4
5

Notice in this example that the for loop executes the line of code "echo $x", and assigns a new value to x for each step of the for loop. You can imagine how useful this can be when extended to processing individual subjects for an analysis, e.g.:

foreach subj ( subj1 subj2 subj3...subjN )

    tcsh afni_proc.py $subj

end


That is all you need to process a list of subjects automatically, and, after a little practice, for loops are easy and efficient to use. The same logic of for loops applies to FSL analyses as well, if you were to apply a feat or gfeat command to a list of feat directories. In the next batch of tutorials, we will begin covering the basics of AFNI, and eventually see how a typical AFNI script incorporates the use of for loops in its analysis.

The second part of the tutorial covers shell scripting. A shell script executes a series of commands within a text file (or, more formally, shell script), allowing the user more flexibility in applying a complex series of commands that would be unwieldy to type out in the command line. Just think of it as dumping everything you would have typed out explicitly into the command line, into a tidy little text file that can be altered as needed and executed repeatedly.

Let us say that we wish to execute the code in the previous example, and update it as more subjects are acquired. All that would be added would be a shebang, in order to specify the shell syntax used in the script:

#!/bin/tcsh

foreach subj (subj1 subj2 subj3)

     tcsh afni_proc.py $subj

end


After this file has been saved, make sure that it has executable permissions. Let's assume that the name of our shell script is AFNI_procScript.sh; in order to give it executable permissions, simply type

chmod +x AFNI_procScript.sh

And it should be able to be run from the command line. In order to run it, type the "./" before the name of the script, in order to signify that it should be executed from the shell, e.g.:

./AFNI_procScript.sh

Alternatively, if you wish to override the shebang, you can type the name of the shell you wish to execute the script. So, for example, if I wanted to use bash syntax, I could type:

bash AFNI_procScript.sh

However, since the for loop is written in t-shell syntax, this would return an error.


Due to popular demand, an optseq tutorial will be uploaded tomorrow. Comparisons between the output of optseq and AFNI's 1d_tool.py is encouraged, as there is no "right" sequence of stimulus presentations you should use; only reasonable ones.

Unix for Neuroimagers: Shells and Variables

First, a few updates:

1) We just finished our first week of the semester here, and although things haven't been too busy, it may be a couple of weeks before I get back on a steady updating schedule. I'll do what I can to keep dropping that fatty knowledge on the regular, and educating your pale, soy-latte-white, Famous Dave's BBQ-stained faces on how to stay trill on that data and stack that cheddah to the ceiling like it's your job. And if you got one of those blogs dedicated to how you and your virgin-ass Rockband-playing frat brothers with names like Brady and Troy and Jason eating those cucumber salad sandwiches or whatever and you drop a link to this site, I'll know it. You show me that love, and I show it right the hell back.

2) While you're here, how about you donate a piece of that stack to the American Cancer Society. I mean, damn; I'm out there seven days a week on those roads, sweating and suffering, but you - you're at work procrastinating again, wringing your snow-bunny white hands over whether you should drop out of graduate school or just toughen it out and graduate in eight years, and while you're at it possibly take a swipe at that new Italian breezey who just entered the neuroscience program. Donate first, worry about those problems later.

3) We got another performance for you all this November, including Schumann's Adagio and Allegro for cello and piano, Resphigi's Adagio con Variazioni, and the Debussy cello sonata. Time and location TBA. Also, more music videos will be uploaded soon, but while you're waiting, you can listen to the latest Mozart Fantasie in D Minor, which has proved one of my most popular videos to date; last I checked, it had 57 views, which I think qualifies for viral status. We goin' worldwide, baby! World-WIDE!!

4) AFNI tutorials are next on the docket, after wrapping up the intro Unix tutorials for neuroimagers, and possibly doing a couple more FSL tutorials on featquery, FSL's ROI analysis tool. Beyond that, there isn't much else I have to say about it; now that you've mastered the basics, you should be able to get the program to jump through whatever hoops you set up for it and to do whatever else you need. There are more complex and sophisticated tools in FSL, to be sure, but that isn't my focus; I will, on the other hand, be going into quite a lot of details with AFNI, including how to run functional connectivity and MVPA analyses. It will take time, but we will get there; as with the FSL tutorials, I'll start from the bottom up.


Anyway, the latest Unix tutorial covers the basics on shells and variables. Shells are just ways of interfacing with the Unix OS; different shells, such as the t-shell (tcsh) and bash shell, do the same thing, but have different syntax and different nomenclature for how they execute commands. So, for example, an if/else statement in the t-shell looks different from a similar statement in the bash shell.

Overall, there's no need to worry too much about which shell you use, although AFNI's default is tcsh, so you may want to get yourself used to that before doing too much with AFNI. I myself use tcsh virtually all of the time, except for a few instances where bash is the only tool that works for the job (running processes on IU's supercomputer, Quarry, comes to mind). There are lots of tcsh haters out there for reasons that are beyond me, but for everything that I do, it works just fine.

As for variables, this is one of the first things you get taught in any intro computer science class, and those of you who have used other software packages, such as R or Matlab, already know what a variable is. In a nutshell, a variable is a thing that has a value. The value can be a string, or a letter, or a number, or pretty much anything. So, for example, when I type in the command

set x=10

in the t-shell, the variable is x, and the value is now 10. If I wish to extract the value from x at any time, I prepend a dollar sign ('$') to it, in order to tell Unix that what follows is a variable. You can also use the 'echo' command to dump the value of the variable to the standard output (i.e., your terminal). So, typing

echo $x

returns the following:

10

which is the value that I assigned to x.

From there, you can build up more complicated scripts and, by having the variable as a placeholder in various locations in your script, only have to change the value assigned to it in order to change the value in each of those locations. It makes your programming more flexible and easier to read and understand, and is critical to know if you wish to make sense of the example scripts generated by AFNI's "uber" scripts.

With all of the tutorials so far, you have essentially all of the fundamentals you need to operate FSL. Really, you only need to understand how to open up a terminal and make sure your path is pointing to the FSL binaries, but after that, all you need to do is understand the interface, and you can get by with pointing and clicking. However, a more sophisticated understanding is needed for AFNI, which will be covered soon. Very soon. Patience, my pretties.

Unix for Neuroimagers: Part 2 (More commands, and the PATH variable)

After you have gotten comfortable and (emotionally) intimate with the basic Unix commands for navigating your environment and listing contents of directories, the other essential tools you will need, if you are using a package like FSL, is a working understanding of the PATH variable. It is not critical that you know how to set the path on your own, but it helps to know conceptually what it does, and why the FSL and AFNI people require you to amend it in certain ways when installing their software.

The PATH variable (Most Unix variables are in all upper case; by the way, now might be the time to define what a variable is; it is an object storing a value, such as a number or a word) is a pointer to specific directories in your directory tree, and allows you to execute the programs and binaries within that directory. For example, almost all of the Unix commands you use are in the /bin folder; instead of having to navigate to the /bin folder to execute a certain command, or explicitly type out the full path to a directory where a command is located, the PATH variable will allow you to execute that command from any directory in your environment. For example, typing the command "echo $PATH" in the terminal might return the following list of directories in my PATH (echo is a command to return the output of a command to the command line, and the dollar sign in $PATH signals that PATH is a variable):

/usr/local/bin:/bin:/usr/local/fsl/bin:/Users/andrewjahn/abin

Note that each directory is separated by a colon. This is a Unix convention, and may take some time to adjust your pitiful human eyes to this.

In this example, all of my FSL binaries (or commands, or executables; I use the words interchangeably, although there are differences in meaning) are located in the directory /usr/local/fsl/bin, while all of my AFNI binaries are located in /Users/andrewjahn/abin. Having these directories listed in my path saves me from having to type the entire path to the command. For example, instead of typing:

/Users/andrewjahn/abin/afni

I can just type:

afni

From anywhere in my Unix environment, and the afni command will execute.

A couple of other important topics covered in this tutorial are startup scripts and sourcing. Startup scripts refer to a series of commands that is executed every time you open up a new terminal or a new shell. In this example, I modify the a startup script called .cshrc (CSH stands for the c-shell; RC stands for run command), and add the lines given by the FSL installation instructions. You can add anything else in there as well, including aliases (i.e., alternate and easier to remember names for more complex commands). The point is that everything within this file will be run each time you begin a new Unix session.

The source command is not strictly necessary, but saves you the time from having to close a session and begin a new one in order to execute everything inside the startup file. Just typing in "source .cshrc" from your home directory is enough to execute the file.


Also uploaded recently is another tutorial covering the basic remove and remove directory commands, as well as how to interact with text files from the Unix command line. I mention it in the video, but I will also restate it here: The rm command is extremely powerful, and removes things forever. That's "forever" with an "f". It does not send them to some Recycling Bin or Trash Bin or anything fruity like that; those files are straight up gone, which means that you better know what you are doing before you use it. This is mostly a word of warning to beginning Unix users, although I have torched innumerable datasets in my day through carelessness, maliciousness, or that fitful combination of both that sometimes seizes me in sudden bursts of anti-fMRI fury.

Unix for Neuroimagers (Part 1): Fundamentals

As promised, we have begun a new series of tutorials aimed at the beginning neuroimager, who may think that he can get by without learning any computer science whatsoever. This is, unfortunately, not true, and those who wish to pursue a career in cognitive neuroscience need to at least learn the fundamentals; if not for constructing your own scripts, then at least for being able to understand and interpret what others have done. Those who refuse to go beyond their comfort zone and test the waters of what must seem to them a foreign and unfriendly language, will forever be relegated to pushing the Big Button that executes commands or scripts without really knowing what is going on. To enter this field without a working knowledge of programming is to be partially blind.

To that end, these first few screencasts will guide the beginner over the initial hurdles of navigating their computer environment solely through the command line interface. At the basic level of navigation and file manipulation, some simple analogies can be drawn between typing in commands such as cd and ls, and pointing and clicking within the standard Windows or Macintosh operating systems that most people are used to. However, at the more abstract levels involving if/else statements and for loops, there are no ready analogies I can think of, and at this point the Unix shell must be treated as any other foreign language; the only way to learn it is by doing, and by consistent repetition.

Usually I try to supplement the videos with some written commentary, but in this case, I believe that there is a very good introductory resource already out there, which can be found here. Walking through all of these tutorials will probably take less than an hour, but it is worth coming back to them from time to time to reinforce what you have learned.

For those of you who are new to neuroimaging or programming and may doubt whether it will ever really stick, let me assure you that it will, in time. Five years ago I was engaged in a research project the summer of my senior year; however, I knew nothing about computer programming. Nothing. I had taken a class of Java in high school, but barely managed to scrape by with a B-, and that was with copying all of the homeworks from one of the Asian kids in the class under threat of death. I was so useless, I didn't even know how to use Excel (no lie), and needed my advisor to hold my hand through doing the most basic analyses in SPSS. It was humiliating at first, because I knew that many of my colleagues were well versed in all of these tools which appeared as second nature to them, and I felt as though they looked down upon me as some hateful and nasty piece of filth clinging to the soles of their shoes.

However, over time I came to teach myself, more through necessity than anything else. My first job out of college required the use of Unix, the experience with which I greatly exaggerated to a ridiculous degree in my application (I had skimmed through a 1980 Unix textbook borrowed from the public library, and said that I had "experience" with Unix); and I set about to learn whatever I could and whatever I saw was most needed. What I found was that, out of the vast Pacific of commands and options present in Unix, only a minuscule fraction of them were required to successfully carry out the work that I needed to do. More important was molding my mind to accommodate the language and understand how to think through operations before translating my will into a series of typed commands. If I, who had virtually no programming background, a terrible memory, and the attention span of a bag full of puppies, could learn the basics within a year, then so can you. Persist.

This could be you!

There is no need to go it entirely alone. As with any skill, much of the fruits of your labors and lightning flashes of insight will only come through long and solitary hours of study and practice; however, it is as important to learn how to identify external resources, whether they float within the ether of the Internet or are made incarnate in the large, stinking, fleshy pile of humanity which sits next to you in your laboratory. One particularly useful resource I have found is unix.com, a message board related to all things Unix. More than once have I posted problems which I had been breaking my head against for many hours or days, only to have a complete solution posted within a matter of seconds. As long as you are willing to tolerate a little condescension and can put up with some invisible nerdling berating you for sloppy indentation, you should be able to solve most of your problems quite easily.

Good luck, comrade.