The neocortex rises as an amazing achievement of biological evolution. All mammals have this strip of tissue covering their brains, and the six layers of densely packed neurons within drive the sophisticated calculations and associations that produce cognitive feats. Since no animal other than mammals has a neocortex, scientists wondered how such a complex brain region evolved.
The reptile brains seemed to offer a clue. Not only are reptiles the closest living relatives of mammals, but their brains have a three-layered structure called the dorsal ventricular ridge, or DVR, with functional similarities to the neocortex. For more than 50 years, some evolutionary neuroscientists have argued that both the neocortex and the DVR were derived from a more primitive feature in a shared mammalian and reptile ancestor.
Now, however, by analyzing molecular details invisible to the human eye, scientists have refuted that view. By examining gene expression patterns in individual brain cells, researchers at Columbia University have shown that despite anatomical similarities, the neocortex in mammals and the DVR in reptiles are unrelated. Instead, mammals seem to have evolved the neocortex into an entirely new brain region, built without a trace of what came before. The neocortex is made up of new types of neurons that seem unprecedented in ancestral animals.
The paper describing this work, which was led by evolutionary and developmental biologist Maria Antonietta Toscheswas published last September in Science.
This process of evolutionary innovation in the brain is not limited to the creation of new parts. Other work by Tosches and his colleagues in the same issue of Science showed that even seemingly old brain regions continue to evolve by reconnecting with new cell types. The discovery that gene expression can reveal these kinds of important distinctions between neurons is also prompting researchers to rethink how they define certain brain regions and re-evaluate whether some animals might have more complex brains than they thought. .
Active genes in single neurons
In the 1960s, influential neuroscientist Paul MacLean proposed an idea about brain evolution that was flawed but still had a lasting impact on the field. He suggested that the basal ganglia, a grouping of structures near the base of the brain, were a remnant of a “lizard brain” that evolved in reptiles and was responsible for survival instincts and behaviors. When the first mammals evolved, they added a limbic system for emotion regulation above the basal ganglia. And when humans and other advanced mammals came along, according to MacLean, they added a neocortex. Like a “thinking plug”, it sat at the top of the pile and conferred higher cognition.
This “trinity brain” model captured the public imagination after Carl Sagan wrote about it in his Pulitzer Prize-winning book in 1977. The Dragons of Eden. Evolutionary neuroscientists were less impressed. Studies quickly debunked the model by conclusively showing that brain regions do not scale perfectly on top of each other. Instead, the brain evolves as a whole, with older parts undergoing changes to accommodate the addition of new parts, explained Paul Cisek, cognitive neuroscientist at the University of Montreal. “It’s not like upgrading your iPhone, where you load a new app,” he said.
The best-supported explanation for the origin of the new brain regions was that they evolved primarily by duplicating and modifying pre-existing neural structures and circuits. For many evolutionary biologists, such as Harvey Karten from the University of California, San Diego, the similarities between the mammalian neocortex and the reptilian DVR suggest that they are, in terms of evolution, homologous – that they both evolved from a structure transmitted by an ancestor shared by mammals and reptiles.
But other researchers, including Luis Puelles from the University of Murcia in Spain, disagreed. In the development of mammals and reptiles, they saw signs that the neocortex and the DVR took shape through completely different processes. This hinted that the neocortex and the DVR evolved independently. If so, their similarities had nothing to do with homology: they were probably coincidences dictated by the functions and constraints of the structures.