OCTOBER 28, 2013
PRESERVING AND MAPPING THE BRAIN'S CONNECTOME

Dr. Ken Hayworth

 

Founding President of the Brain Preservation Foundation and Senior Scientist at the Howard Hughes Medical Institute's Janelia Farm Research Campus

 

AbstractEvery memory, skill, and personality trait that makes you a unique individual is structurally encoded in your brain’s neural connectivity, i.e. your “connectome”. This is the overwhelming consensus view of those working in the cognitive and neural sciences. I will review technologies available today to preserve and map the connectome of small animals and how these technologies are quickly progressing to the level where they could be applied to humans. I will present the latest results in my foundation’s Brain Preservation Prize contest including preliminary evidence that it is now possible to chemically preserve an entire mouse brain in a block of inert plastic such that today’s electron imaging technology can image any region of it at the nanometer scale – i.e. sufficient resolution to perfectly extract its preserved connectome. From a technical perspective, it is quite possible that such a perfected, reliable, and relatively inexpensive chemical brain preservation procedure could be made widely available to human patients in hospitals within the next 5 years. Extrapolating today’s progress in electron imaging technologies, it is quite possible that the ability to map the entire connectome of such a preserved human brain could be available within a few decades.


I will also consider key implications. We can now sketch out a scientifically sound, technological roadmap that would make human mind uploading possible by midcentury. More amazingly, anyone reading this abstract today could personally experience such mind uploading if they are able to secure a quality chemical brain preservation prior to their natural death. It is imperative that over the next few years we muster the scientific, medical, social, and political resources necessary to transition today’s successful laboratory brain preservation techniques into an elective surgical procedure which can be performed on human patients in a hospital setting.


The biggest obstacle to this is not technological. It is the persistent refusal by many to accept what cognitive science tells us about the nature of the human ‘soul’. Daniel Dennett, the great American philosopher who has done more than anyone to popularly explain how the results of science demystify consciousness, put it this way: “Yes, we have a soul, but it’s made of lots of tiny robots. It’s made of neurons. And we can actually explain the structure and operation of that kind of soul, whereas an eternal, immortal, immaterial soul is just a metaphysical rug under which you sweep your embarrassment for not having any explanation.” We need to start accepting what science has told us about the nature of the human soul, that it is analogous to software running on the computing hardware of our brain, and that this software ‘soul’ is encoded in our brain’s structural connectome. Once we accept this scientific viewpoint we can begin to see how truly liberating it is, for it means that the future is not just something that our grandchildren will experience, it is something we could personally experience with greater physical and mental health than is even imaginable today.


Transcript

 

In this talk I want to get right to the main point. I want to lay out a plan for uploading a particular human mind into a computer by midcentury. Now obviously this is quite difficult to address such a topic in such a short talk, so I will mostly rely on providing pointers to the relevant literature.

 

I want to first give a pointer to a paper which I published last year, which lays out this complete plan in much greater detail. This paper is entitled "Electron Imaging Technology for Whole Brain Neural Circuit Mapping," and it's available on my website at brainpreservation.org.

 

Now, let me lay out what I'll be talking about. We cannot talk about mind uploading without first understanding how the brain gives rise to mind, including consciousness and identity. So I'm going to take you on a whirlwind tour of neuro and cognitive science ideas that support my central thesis that our identity is encoded in the structural connections among our brain's neurons - what is commonly called our connectome. This lays out the problem to be solved. What remains is the technology to accomplish it, the key technology being mapping the complete connectome of a particular human brain. I will discuss the new technologies invented within the last few years, which are indeed mapping the connectomes of small pieces of animal brains. I'll make the case for one of these technologies in particular seems to have all the ingredients necessary to eventually scale up to human level. I will briefly discuss the current state-of-the-art in brain connectome preservation and I will finally conclude with a tentative plan of action.

 

Okay, so first of all, how do neural circuits in the brain give rise to our particular mind? Well first, let's talk about how specific ways neurons connect determine the functioning of neuronal circuits. Well here's a classic example which should be familiar to everyone who has read an introductory neuroscience book: the visual simple and complex cells found by Hubel and Wiesel in the cat visual cortex. Photoreceptor cells in the retina make specific connections with horizontal cells and bipolar cells, which in turn make specific connections with two other types of retinal ganglion cells, on-center and off-center. And these are the main output cells of the retina, which signal a particular image contrast of a particular point in the visual field. These ganglion cells send their axons to the thalamus, which then relays these signals to the primary visual cortex in the occipital lobe. There, these axons synapse in very particular patterns on cells in layer 4 of this cortical area, causing V1 simple cells to respond only when a line is present at a particular position and orientation of the visual field. This is very similar to what Ray was talking about with the crossbar in the "A". These simple cells send their axons in turn to V1 complex cells, which pool connections from the simple cells having the same orientation preference bu different spatial positions causing them to respond to a line at a particular orientation anywhere over a much wider range of positions.

 

Now, the key point here is that all the experiments done to date suggest that the particular functioning of these visual cortical cells, these simple and complex cells, could be completely predictable based on which connections they make with other cells originating in the retina and the thalamus. Now it's not as well known, but it is thought that the same process of pooling synaptic connections from early visual areas to later visual areas explains how we recognize objects in a translation invariant manner. This is the standard model of the primate visual system. Here is depicted a feature hierarchy of cells laid out in our occipital cortex in a diagram of how this models our perception of objects, faces, places, etc. This theory has a tremendous amount of electrophysiological support as well as computer modeling results, and it predicts that the functions of neurons, even at these highest levels of cortical areas. should be absolutely predictable from their connections with lower areas.

 

Now, here's something that is absolutely crucial to what I'm trying to get across: the upper level of these visual feature hierarchies consists of a set of distinct cortical buffer areas, each one designed to represent one aspect of the world: color, motion, shape, facial identity, etc. And each of these areas has their own vocabulary of symbols to represent the world in neural firing. The shape representing area might have 1 million neurons, and a particular pattern of firing over this set of neurons might represent the shape "circle." A different pattern of firing would represent the shape "square" to the rest of the brain. The key point is that these brain symbols - these ARE the brain's symbols, and those symbols are the vocabulary of the mind. It's the mind's internal language. And all of these symbols are grounded in the real world via feature hierarchies like the one depicted here. Thus all the brain's symbols can in principle be read off by looking at the brain's connectome.

 

Now, what I've said so far is true not only for the human brain, but for most animal brains. The brains of animals are built with feature hierarchies able to turn sensory input into symbolic representations of some particular important aspect of the external world. And each symbol is encoded as a particular pattern of firing over a particular set of neurons. So how does this translate into intelligent behavior? Well, in mammals there are circuits in brain areas, like the basal ganglia and cortex, which are trained by reinforcement learning to recognize particular symbols in the cortical buffers, and then to initiate firing in motor areas leading to appropriate action. I don't have time to go into the details, obviously, but let me assure you again that these basal ganglia circuits are thought to be simple pattern recognition circuits, which are again directly encoded in the connections between neurons. If you have the connectome of an animal, you should be able to tell what symbols would trigger what particular actions.

 

Okay, now neural symbols driving learned pattern recognition circuits may explain simple animal behavior, but what about human intelligence? Well cognitive science gives us a simple and elegant answer to this question. The primate brain has developed ways in which motor actions learned and initiated by pattern recognition circuits in the basal ganglia can initiate, instead of motor actions, internal routing operations, to shift symbols between cortical buffers. This is the theory at the heart of the ACT-R cognitive architecture. ACT-R is the most advanced model of the human mind available today. Dozens of labs are working with it, probably hundreds of labs are working with it, to model everything from simple stimulus response tasks, to natural language understanding, solving math problems, everything. The ACT-R model explains how stringing together a simple basal ganglia routing operations, called productions, into longer sequences can give rise to all the fantastically complex intelligent behaviors we display as human beings. And what is great about this theory is it allows the high level intelligent processing to be tied directly to the neural circuits. In fact, I've just recently published a paper on how such ACT-R routing operations might be performed in the at the neural circuit level in the brain. I recommend it for anybody doubting that such a symbolic architecture like ACT-R can actually be implemented in biological brain circuits.

 

Okay, so what of consciousness, qualia, feeling, and a sense of self? Well, cognitive science is not silent on these points. From a cognitive science viewpoint consciousness arises from the brain's ability to create a self model, which represents a person's perceptions, goals and decisions. It is actually quite straightforward to see how such a self model can map onto the ACT-R cognitive architecture, and thus on to the connectome. I described how in more detail in my paper. Such a phenomenal self model naturally creates an "I" which is the experiencer of perceptions and feelings in the unitary decision-maker. This is the standard cognitive science view of consciousness, and I believe it is 100% correct. If we accept this view of consciousness in the self as most likely true, then we can move forward with a plan for mind uploading. If we reject this scientific view of consciousness, then we may as well go home because in my opinion, there really is no other game in town. Here are three excellent books which describe how consciousness can be explained through such a self model in the brain.

 

Now, the technical feasibility of mind uploading also hinges on our notion of identity - for example, the "just a copy" question above. Again science provides an answer, but not one that is easily reconciled with our naive preconceptions itself. I recommend these two books as providing very straightforward, scientifically sound answers to that very difficult question.

 

In summary, I believe that there is an absolutely sound scientific case for how an individual's connectome directly encodes the person's memories, skills and identity.


Okay, I hope I've provided a bit of support to the idea that our identities are encoded in our connectome. What is really exciting about living in 2013 is that just within the last few years, the first techniques for mapping connectomes have appeared. This is poised to revolutionize our understanding of the brain, since all the theories of neural circuit function I described previously will be being rigorously tested in the upcoming years using direct connectome mapping.


The modern connectomics revolution can be traced back to 2004 paper by Winfried Denk in which he showed that a scanning electron microscope could be used to obtain images of the freshly cut surface of a piece of plastic-embedded brain tissue. Since then there has been an explosion of different techniques to obtain 3-D volume images of brain tissue. This paper came out last year and is a complete review of four of these technologies written by two of the pioneers in the field: Davi Bock and Kevin Briggman. This is a figure from Briggman and Bock's paper depicting the four main connectomics technologies: serial section TEM, serial block EM, automatic tape collection for SEM and focused ion beam SEM. Let's look at each in turn.

 

First, there is the cutting and collecting of ultrathin sections for transmission electron microscopy. This is a technique that literally goes back decades, but new inroads are being made to automate the section collection and especially to automate the TEM imaging steps. The key stumbling block to large-scale automation of this is the necessity to collect sections on a gossamer thin plastic film shown here. Here is a 3-D volume image of brain tissue using this technique, both in-plane at the top and in cross-section over many, many images at the bottom. And here is a fantastic reconstruction of neural circuits in the visual cortex produced by this technique.

 

As I mentioned, one of the main limitations of that serial section TEM technique is the requirement to collect thin sections on a gossamer thin film. The ATUM SEM technique, which I invented with colleagues at Harvard University, instead collects thin sections on a solid tape. These sections are thus made very robust to later processing and imaging steps which are done using a scanning electron microscope instead of a transmission electron microscope. And here is a characteristic 3D volume and reconstructions produced by this technique.


The next connectomics technique is a serial block face EM technique invented by Denk, the one that kicked off the connectomics revolution to begin with. In that technique, a diamond knife ultra microtome is mounted inside the vacuum chamber of a scanning electron microscope. The knife scrapes away thin layer of tissue off the surface of the block and then the electron beam images this freshly revealed block face. This is repeated over and over again producing a series of electron micrograph images from which a 3D reconstruction can be made. Here is a characteristic volume images and a reconstruction of part of the retinal connectome performed using this technique.


Finally, there is the focused ion beam SEM technique. It works very similarly to the serial block face SEM technique, but instead of removing each section with the diamond knife, each thin layer of the block is instead vaporized away using a high-energy beam of metal ions. Here is a 3-D volume image and tracings using this technique. Something to notice is that the cross-section view at the bottom here, the cross-section view from this technique looks better than the previous techniques. This is because all the other techniques were limited in their Z resolution due to the need to physically cut a section with a diamond knife. Because of this the previous techniques struggled to obtain Z resolutions below about 30 nm. The FIBSEM technique, in contrast, can easily obtain 5 nm Z resolution.

 

Now, let's get right to the main question. Can we even imagine any of these being scaled to the level of human mind uploading? Well, it's important to note that none of these have mapped a connectome of even a cubic millimeter of brain tissue yet. It is therefore a stretch to imagine how any of them could be scaled up by a factor of 1 million, however, we're not talking about doing this tomorrow. We're talking about starting a targeted research program which will over the next several decades lead to that result. With this caveat clearly in mind, I believe one of the technique stands out as having clear long-term advantages, and that technology is FIBSEM - the focused ion beam SEM technique. FIBSEM has already demonstrated the highest isotropic resolution of all the technologies given that it can remove an image layer as thin as 5 nm. This is crucially important because at these resolutions, there is no ambiguity in extracting the connectome. All of the other technologies use a diamond knife to cut or scrape away very thin sections. In contrast, FIBSEM uses a beam of ions, which can be sharpened simply by adjusting the voltage.


To repeatedly remove 10 nm thick sections with a physical knife is almost impossible. The knife position must be kept accurate to the nanometer during the physical sectioning event. In contrast, even a dully focused ion beam a thousand of nanometers wide can accurately ablate a 5 nm thick layer of the block, so let's see why that's the case. This is a simulation of the surface of a block of tissue being slowly ablated away by a focused ion beam. On the left is the profile of the current density of the ion beam. In this case, the beam is very wide, it's about 1000 nm wide, but you can see as the simulation progresses that the block surface is gently eroded away a few nanometers at a time. This is the true advantage of the FIBSEM technique. The ion beam acts as a soft knife whose size and position does not need to be constrained anywhere near as tightly as is necessary for sectioning with a diamond blade. At Janelia, we recently implemented a closed loop control of the ion beam position relative to the block. One simply monitors the current generated by ions sputtering off the block surface and feed this signal back into a control loop controlling the height of the ion beam. This makes for an extremely robust and simple system capable of imaging and ablating tens of thousands of 5 nm thick sections over months of unattended imaging. Here is a very, very short stack of images taken using this closed loop FIB technique.

 

Now I need to address the Achilles' heel of the FIBSEM technique. The ion beam can only finely mill blocks which are less than about 100 microns wide in the direction of the ion beam. Also imaging takes far too long for large volumes to be practical on a single machine. The obvious solution to these limitations is to come up with a technique which can reliably cut a large piece of brain tissue and eventually an entire brain into smaller chunks, large enough to be cut reliably, but small enough to be efficiently imaged in the FIBSEM. This subdivision of the brain into chunks must however be absolutely lossless and it must allow the individual imaged volumes to be stitched back together again.

 

I've been working hard to perfect such a lossless subdivision technique for the FIBSEM and I've made considerable progress, I think, which I'll present now. Using a heated ultrasonic diamond knife, which is lubricated with oil, I can now robustly section plastic embedded brain tissue into 20 micron thick slabs like the one from the fly, shown here. Here are two matched thick sections cut from a larval fly at 20 microns thick. Why 20 microns thick? Well, 20 microns is optimally sized for high-resolution ion beam milling, while at the same time being so thick that much much fewer diamond knife cuts are needed for a given volume. And these cuts and sections are much more robust to damage. Zooming in a bit, we can see the smooth surfaces of this hot knife cut, and we can identify large matching features across the cut.

 

I've obtained volume FIBSEM images from matched regions, zooming in again here, and here is an outline of this volume stitching technique, and here is a movie of the stitched volumes in cross-section. You can see that the neuronal processes are easily traced across this thick section boundary, which appears as a whitish horizontal line in the center of this movie, and here is the volume split open along the hot knife cut surfaces, showing neuronal fibers that have been successfully traced across this thick section gap. Here's another example of a matched pair of thick cut sections image separately and then stitched back together again. The key point here is that this proves that the focused ion beam SEM technique is not limited to small volumes. Any size brain volume should be easily thick sectioned by such a hot-knife technique and then the pieces can be imaged in parallel across many FIBSEM machines.

 

Now, I've been talking a lot about large volumes of brain tissue, but until recently, the chemical fixation and plastic embedding process used by neuroscientists was only capable of preparing volumes smaller than about a cubic millimeter. Recently, however, a researcher named Shawn Mikula has stunningly overcome these limitations, and has shown very good results in preparing an entire mouse brain for the serial block face SEM approach. Here is a paper that he published last year on this technique. You can see him holding a complete mouse brain preserved in plastic at the bottom. He has pushed this technique either even further in the meantime, and I'm pleased to present a stack of SEM images he provided me from one of his recently prepared mouse brains showing what appears to be near perfect EM quality preservation across the entire mouse brain. The absolutely revolutionary thing about this stack of EM images is that if you look anywhere in that mouse brain, you get the same level of quality. It seems that he has developed a technique to preserve the connectome of an entire mouse brain and prepare it for mapping using the connectomics technologies that I've just described.

 

OK, I think we're ready for the plan here. So, the plan: well if you asked Werner von Braun in the 1950s what his plan was for getting men to the moon, he might have tersely said "build a really big rocket." Of course he would have more to say regarding the details and complications, but in essence the problem of getting a man to the moon is one of building a rocket large enough to reach orbital velocity with a significant payload.

 

When asked what a plan might be for uploading a person by midcentury, I would suggest:

 

1. Figure out how to chemically fix, heavy-metal stain, and plastic embed a whole human brain.

 

2. Build a hot-knife thick sectioning machine which can losslessly cut this brain into 20-micron-thick slabs suitable for FIBSEM imaging.

 

And 3. Develop a mass-manufacturable FIBSEM machine which can be mass-produced by the thousands, so that the 20-micron-thick slabs can be FIBSEM imaged in parallel in a reasonable amount of time.

 

Like the Apollo project, such a project would require enormous resources, but I think it is a straightforward plan to achieving what should be considered an even more important goal for humanity - putting the first human mind in cyberspace and returning her safely to consciousness.

 

Now, let me close this talk with a note of realism. In my opinion, it is likely to take a multibillion-dollar program several decades to succeed in uploading the first human and it is likely to be a while before the cost and difficulty to do so reduces sufficiently for common citizens to enjoy the advantages of mind uploading. I look forward to the day when almost everyone can afford to be uploaded into a brand-new robotic body and enjoy health and indefinite lifespans. However, I'm under no illusion that that they will come anytime soon. Certainly not in my natural lifespan. But the beginning part of this envisioned mind uploading procedure is to first chemically fix and plastic-embed the person's brain; and after that step is occurred, the brain could sit there on a shelf for the decades necessary for the uploading technology to mature. Although such a procedure not been demonstrated in practice, there is every reason to believe that one could be perfected in just a few years if qualified research labs put moderate resources toward that goal. The Mikula mouse brain results that I showed earlier should clearly demonstrate how within grasp that goal is.

 

The message is clear: Every single person in this audience could experience mind uploading firsthand, IF we make sure that a quality brain preservation procedure is quickly developed and implemented in hospitals worldwide. That is why I formed the Brain Preservation Foundation a few years ago, and that's why we've issued a challenge prize to all scientists and medical researchers to develop such a brain preservation technique. Once this prize is one and quality brain preservation start being offered in regulated hospitals worldwide, the real mind uploading revolution will have begun.

 

Acknowledgements

 

Our thanks go to our volunteers Giulio Prisco, Kim Solez, Chris Smedley, Philip Wilson, Xing Chen, including anonymous volunteers for their help with the transcription of Congress presentations.

Congress announcements
01.10.2013
In June 2013, the Initiative hosted GF2045, the Global Future 2045 International Congress: Towards a New Strategy for Human Evolution, in New…
19.06.2013
At the Global Future 2045 conference (GF2045) in New York City on June 15-16, 2013, emcee Philippe van Nedervelde said, “It used to be that the…
15.06.2013
'How's this for a weekend conference: Some of the smartest people in the world are gathering in New York to try to figure out how to build…
14.06.2013
To accomidate popular demand, we are now able to offer a limited number of tickets for one day attendance for Saturday, June 15th or Sunday,…
04.06.2013
Topics for the Two-Day Conference at New York’s Alice Tully Hall June 15 – 16, 2013 Include: Human-Like Androids, Brain-Computer Interfaces,…