The History of Brain Imaging Data Structure (BIDS)

One development in neuroimaging research that everyone should be aware of is the advent of Brain Imaging Data Structure (BIDS). This was intended to correct the old ad hoc way of organizing data, which back in the day meant doing whatever the hell you wanted. No organizing scheme was too hare-brained; no folder labels were too outlandish. Here is a sample of what such a scheme looked like during those benighted times; read, and despair:

• BrainData

• OnsetTimesRevised

• MyFirstAnal

• GroupAnal

These names, which I definitely did not use during my post-graduate years, are not only ambiguous, but can mislead. For example, MyFirstAnal (which, for the record, is shorthand for “Analysis” (I hope)) clearly contains some kind of analysis; but which one, and why is it the first? How many did the researcher go through before hitting upon the one that worked, and which ones were used for the GroupAnal(ysis)?

With such harum-scarum labels, slip-shod and slap-dash, as well as many other denigrating hyphenated words whose very structure seems destined to denote confusion and general sloppiness, it is no wonder that back then so many people were stressed out, perplexed, and angry; unlike today, in which everything is neatly organized, clean, and well-run, and researchers are consequently pleasant and amiable. It is a wonder our forebears were able to do anything at all; and it may be safely assumed that everything written, thought about, or said up until about five years ago is rubbish. Under the new dispensation, we have finally found truth.

What brought about this change? About a decade ago during a meeting at the Organization for Human Brain Mapping conference, a group of researchers met to discuss whether data organization could be standardized; the welter of idiosyncratic names would be unknotted, the babel of languages would be reduced to one, and everybody could communicate more clearly. Reading another’s code would be much easier, and applying someone else’s script would require only minor edits to the file names; the rest of the folders would more or less be the same. No longer would the message board moderators at AFNI suffer apoplectic shock when trying to decipher a block of code rendered unintelligible by a naming scheme both deeply personal to the creator and utterly baffling to the outsider. Everything would be streamlined, simplified, and easy to understand.

Years later, their efforts have been fruitful, with BIDS widely adopted by a variety of data repositories and software analysis packages. For example, any data organized in BIDS format can be read and imported by the CONN toolbox, and this process takes considerably less time than doing it manually, as well as reducing the potential for human error; a website like OpenNeuro.org requires any uploads to be in BIDS format, and instructors who use it for courses or workshops can rest assured that their scripts should work for nearly anyone; preprocessing pipelines such as fMRIPREP require BIDS format, and it is likely that any new tool or software package intended for a wide audience will have to adopt it as well.

Although the use of BIDS may seem irresistible, there will always be some disaffected researchers who kick against it. Who came up with this convention, they say, and why should everybody have to use it? Virtually all labs have had their own way of doing things since their inception, and some may see no reason to change. Moreover, some people may simply not like the way that BIDS is laid out, and they may have legitimate grievances about the naming conventions. One size does not fit all, they say, and by having so many websites and software packages not only encourage but require BIDS, this in effect forces a kind of monopoly on the marketplace of neuroimaging research - one that they neither wanted nor asked for.

These complaints are valid, but at the end of the day, BIDS was not created to cause difficulties or confusion for anyone; it was a response to a long-standing, general level of discontent with how data was organized. At last a group of people decided to address it, and the rapid and widespread adoption of their standard indicates that many people agree with the severity of the difficulty and the usefulness of the proposed solution. Whether you agree with it or not, it looks like it’s here to stay, and the modern researcher needs to know how to convert their own data into BIDS, and how to use BIDS data from other groups. Such is the evolution of any professional field, and the modern researcher must adapt.

---

More details about the history of BIDS can be found in the preprint, which has been accepted for publication and should be available soon.

AFNI Command of the Week: 3dNotes

Those of you who know me, know that I like to stay organized. The pencils on my desk are arranged in ascending order, neatly as organ pipes; the shoes in the foyer of my apartment are placed according to when they were last used, so that I never run in the same pair on consecutive days; the music scores on my bookshelf are stacked so that the first one I take off the top is the one I love best - which, incidentally, always happens to be Liszt's Hungarian Rhapsodies.

However, the game is different when attempting to organize and stay on top of your neuroimaging analyses, as each experiment usually requires dozens, many of them unforeseen but nevertheless pursued, as if by sheer compulsion, until either the waste of your body or until their final endarkenment. At the times that names are given we think them apt; but return weeks, months later, and find to your horror that you have little idea what you did. This collective misery is shared, I think, by many.

AFNI tries to mitigate this by providing a history of each dataset, which traces, step by step, each command that led to the birth of the current dataset. This is useful for orienting yourself; but if you wish to go a step further and append your own notes to the dataset, you can do so readily with the command 3dNotes.

This is a simple program, but a useful one. The options are -a, to add a note; -h, to add a line to the history; -HH, to replace the entire history; and -d, to delete a note. This can also be done through the Plugins of the AFNI interface, all of which is shown in the video below.

Using SPM.mat to Stay on Track

Life, I have observed, is a constant struggle between our civilized taste for the clean, the neat, and the orderly, on the one hand, and the untrammeled powers of disorganization, disorder, and chaos, on the other. We feel compelled to organize our household and our domestic sphere, including the arrangement of books and DVDs in alphabetical order, placing large items such as vacuum cleaners and plungers in sensible locations when we are done with them, and cleaning and putting away the dishes at least once a week. However, this all takes time and effort, which is anathema to our modern tendency to demand everything immediately.

The same is true - especially, painfully true - in analyzing neuroimaging data. Due to the sheer bulk of data collected during the course of a typical study, and the continual and irresponsible reproduction and multiplication of files, numbers, and images for each analysis, dealing with such a formidable and ever-increasing mountain of information can be paralyzing. The other day, for example, I was requested to run an analysis similar to another analysis I had done many months before; but with little idea of how I had done the first analysis in the first place, I was at a complete loss as to where to start. Foreseeing scenarios such as this, I had taken the precaution to place a trail of text files in each directory where I had performed a step or changed a file, in the hopes that it would enslicken my brain and guide me back into the mental grooves of where I had been previously. However, a quick scan of the document made my heart sink like an overkneaded loaf of whole wheat bread, as I realized deciphering my original intentions would baffle the most intrepid cryptologist. Take, for example, the following:


README.txt
---------------
20 July 2011
Input data into cell matrix of dimensions 30x142x73; covariates entered every other row, in order to account for working memory span, self-report measure of average anxiety levels after 7pm, and onset of latest menstrual cycle. Transposed matrix to factor out singular eigenvariates and determinants, then convolved with triple-gamma hemodynamic response function to filter out Nyquist frequency, followed by reverse deconvolution and arrangement of contrast images into pseudo-Pascal's Triangle. I need scissors! 61!


Deepening my confusion was a list of cross-references to handwritten notes I had scribbled and scrawled in the margins of notebooks and journals over the course of months and years, quite valuable back then, quite forgotten now, as leafing through the pages yielded no clue about when it was written (I am terrible at remembering to mark down dates), or what experiment the notes were about. But just as the flame of hope is about to be snuffed out forever, I usually espy a reference to a document located once again on my computer in a Dropbox folder, and I am filled with not so much pride or hope, as gladness at some end descried; which invariably sets me again on a wild goose chase through the Byzantine bowels of our server, which, if not precisely yielding any concrete result, at least makes me feel stressed and harried, and therefore productive.

Imagine my consternation then, during the latest round of reference-chasing, when I came to the point where I could go no further; where there was not even a chemical trace of where to go next, or what, exactly, I was looking for in the first place. My mind reeled; my spine turned to wax; my soul sank faster than the discharge of a fiberless diet. At wit's end, I cast about for a solution to my predicament, as I mentally listed my options. Ask for help? Out of the question; as an eminently and internationally respected neuroscience blogger, to admit ignorance or incompetence in anything would be a public relations disaster. Give up? As fond a subscriber as I am to the notion that discretion is the better part of valor, and as true a believer as any that there is nothing shameful, base, or humiliating about retreating, surrendering, or rearguard actions, this situation hardly seemed to merit my abject capitulation; and deep down I knew that overcoming this obstacle and chronicling my struggle would inspire my children and grandchildren to similar feats of bravery.

And so it was precisely at this moment, at the nadir of my existence, in the slough of despond, that, through either the random firing of two truculent interneurons in my hippocampus or through intervention by the divine hand of Providence, I had a sudden epiphany. The circumstances of my present situation echoed parallels to the gruesome detective stories I used to read as a child straight before bedtime, and I imagined myself standing in the shoes of a fleabitten detective attempting to piece together the origin and denouement of a puzzling murder, as in Gore by the Gallon or Severed Throats; and I therefore reasoned that, as every strangulation, bludgeoning, shooting, stabbing, poisoning, drowning, and asphyxiation leave traces of their author, so too must each analysis bear the fingerprint of its researcher. Straightaway I navigated to the directory of the analysis I was attempting to replicate, loaded the SPM.mat file into memory, displayed its contents, and quickly realized that I had no idea what any of it meant.

Thus, although the output of the SPM.mat file appears to me as hieroglyphs, I have faith that a more experienced user will know what they mean; and it still stands to reason that these files do contain everything that was done to create them, much as the strands of genetic information coursing through our bodies are virtual volumes of the history of the gonads and gametes from whence they came. I encourage the beginning neuroimager to be aware of this, as the designers of these software packages have proved far more prescient than we, and have installed safeguards to prevent us from the ill effects of our own miserable disorganization.