Redundancy Reduction and Pattern Recognition: Richard Dawkins's Answer to the Annual Edge Question
By RICHARD DAWKINS - EDGE
Updated: Mon, 16 Jan 2012 19:23:02 UTC
Polish translation below
This is Richard's answer to John Brockman's annual Edge question for 2012. This year, the question was "What is your favorite deep, elegant, or beautiful explanation?
Deep, elegant, beautiful? Part of what makes a theory elegant is its power to explain much while assuming little. Here, Darwin's natural selection wins hands down. The ratio of the huge amount that it explains (everything about life: its complexity, diversity and illusion of crafted design) divided by the little that it needs to postulate (non-random survival of randomly varying genes through geological time) is gigantic. Never in the field of human comprehension were so many facts explained by assuming so few. Elegant then, and deep—its depths hidden from everybody until as late as the nineteenth century. On the other hand, for some tastes natural selection is too destructive, too wasteful, too cruel to count as beautiful. In any case, coming late to the party as ever, I can count on somebody else choosing Darwin. I'll take his great grandson instead, and come back to Darwin at the end.
Horace Barlow FRS is the youngest grandchild of Sir Horace Darwin, Charles Darwin's youngest child. Now a very active ninety, Barlow is a member of a distinguished lineage of Cambridge neurobiologists. I want to talk about an idea that he published in two papers in 1961, on redundancy reduction and pattern recognition. It is an idea whose ramifications and significance have inspired me throughout my career.
The folklore of neurobiology includes a mythical 'grandmother neurone', which fires only when a very particular image, the face of Jerry Lettvin's grandmother, falls on the retina (Lettvin was a distinguished American neurobiologist who, like Barlow, worked on the frog retina). The point is that Lettvin's grandmother is only one of countless images that a brain is capable of recognising. If there were a specific neurone for everything we can recognise—not just Lettvin's grandmother but lots of other faces, objects, letters of the alphabet, flowers, each one seen from many angles and distances, we would have a combinatorial explosion. If sensory recognition worked on the 'grandmother principle', the number of specific recognition neurones for all possible combinations of nerve impulses would exceed the number of atoms in the universe. Independently, the American psychologist Fred Attneave had calculated that the volume of the brain would have to be measured in cubic light years. Barlow and Attneave independently proposed redundancy reduction as the answer.
Claude Shannon, inventor of Information Theory, coined 'redundancy' as a kind of inverse of information. In English, 'q' is always followed by 'u', so the 'u' can be omitted without loss of information. It is redundant. Wherever redundancy occurs in a message (which is wherever there is nonrandomness), the message can be more economically recoded without loss of information (although with some loss in capacity to correct errors). Barlow suggested that, at every stage in sensory pathways, there are mechanisms tuned to eliminate massive redundancy.
The world at time t is not greatly different from the world at time t-1. Therefore it is not necessary for sensory systems continuously to report the state of the world. They need only signal changes, leaving the brain to assume that everything not reported remains the same. Sensory adaptation is a well-known feature of sensory systems, which does precisely as Barlow prescribed. If a neurone is signalling temperature, for example, the rate of firing is not, as one might naively suppose, proportional to the temperature. Instead, firing rate increases only when there is a change in temperature. It then dies away to a low resting frequency. The same is true of neurones signalling brightness, loudness, pressure and so on. Sensory adaptation achieves huge economies by exploiting the non-randomness in temporal sequence of states of the world.
What sensory adaptation achieves in the temporal domain, the well-established phenomenon of lateral inhibition does in the spatial domain. If a scene in the world falls on a pixellated screen, such as the back of a digital camera or the retina of an eye, most pixels see the same as their immediate neighbours. The exceptions are those pixels which lie on edges, boundaries. If every retinal cell faithfully reported its light value to the brain, the brain would be bombarded with a massively redundant message. Huge economies can be achieved if most of the impulses reaching the brain come from pixel cells lying along edges in the scene. The brain then assumes uniformity in the spaces between edges.
As Barlow pointed out, this is exactly what lateral inhibition achieves. In the frog retina, for example, every ganglion cell sends signals to the brain, reporting on the light intensity in its particular location on the surface of the retina. But it simultaneously sends inhibitory signals to its immediate neighbours. This means that the only ganglion cells to send strong signals to the brain are those that lie on an edge. Ganglion cells lying in uniform fields of colour (the majority) send few if any impulses to the brain because they, unlike cells on edges, are inhibited by all their neighbours. The spatial redundancy in the signal is eliminated.
The Barlow analysis can be extended to most of what is now known about sensory neurobiology, including Hubel and Wiesel's famous horizontal and vertical line detector neurones in cats (straight lines are redundant, reconstructable from their ends), and in the movement ('bug') detectors in the frog retina, discovered by the same Jerry Lettvin and his colleagues. Movement represents a non-redundant change in the frog's world. But even movement is redundant if it persists in the same direction at the same speed. Sure enough, Lettvin and colleagues discovered a 'strangeness' neurone in their frogs, which fires only when a moving object does something unexpected, such as speeding up, slowing down, or changing direction. The strangeness neurone is tuned to filter out redundancy of a very high order.
Barlow pointed out that a survey of the sensory filters of a given animal could, in theory, give us a read-out of the redundancies present in the animal's world. They would constitute a kind of description of the statistical properties of that world. Which reminds me, I said I'd return to Darwin. In Unweaving the Rainbow, I suggested that the gene pool of a species is a 'Genetic Book of the Dead', a coded description of the ancestral worlds in which the genes of the species have survived through geological time. Natural selection is an averaging computer, detecting redundancies—repeat patterns—in successive worlds (successive through millions of generations) in which the species has survived (averaged over all members of the sexually reproducing species). Could we take what Barlow did for neurones in sensory systems, and do a parallel analysis for genes in naturally selected gene pools? Now that would be deep, elegant and beautiful.
Redukcja redundancji i rozpoznawanie wzorów
Autor tekstu: Richard Dawkins
Tłumaczenie: Małgorzata Koraszewsk
To jest odpowiedź Richarda Dawkinsa na doroczne pytanie Edge, postawione przez Johna Brockmana na rok 2012. W tym roku pytanie brzmi: „Jakie jest twoje ulubione, głębokie, eleganckie lub piękne wyjaśnienie?"
Głębokie, eleganckie, piękne? Elegancja teorii polega na tym, że potrafi wyjaśnić bardzo dużo, zakładając bardzo mało. Tutaj bezapelacyjnie wygrywa teoria doboru naturalnego Darwina. Stosunek olbrzymiej ilości tego, co wyjaśnia (wszystko o życiu: jego złożoności, różnorodności i złudzenia, że zostało zaprojektowane), do tej niewielkiej ilości, które musi założyć (nielosowe przetrwanie losowo zmieniających się genów w czasie geologicznym), jest gigantyczny. Nigdy w dziedzinie pojmowania ludzkiego tak wiele faktów nie zostało wyjaśnionych przy tak skromnych założeniach. Eleganckie więc i głębokie — jego głębia ukryta przed wszystkimi aż do
drugiej połowy XIX wieku. Z drugiej strony, zbyt destruktywne, zbyt marnotrawne, zbyt okrutne, by niektórzy uważali je za piękne. W każdym razie, jak zwykle spóźniony na przyjęcie, mogę liczyć się z tym, że ktoś inny wybierze Darwina. Zamiast tego wezmę jego prawnuka i powrócę do Darwina na końcu.
Dave Mosher - National Geographic Comments
The sun is the roundest natural object ever precisely measured, astronomers say.
Geraint Jones - The Guardian Comments
Scientists who encoded the book say it could soon be cheaper to store information in DNA than in conventional digital devices
Ed Yong - Nature News Comments
Under the supervision of guards and graduate students, a small group of prisoners is breeding the beautiful orange-and-white insects in a greenhouse outside the prison. They have even carried out research to show what plants the butterfly prefers to lay its eggs on.
- - Scientific American Comments
Teachers, scientists and policymakers have drafted ambitious new education standards. All 50 states should adopt them
John Roach - NBC News Comments
An artificial “brain” built by a 17-year-old whiz kid from Florida is able to accurately assess tissue samples for signs of breast cancer, providing more confidence to a minimally invasive procedure.
MORE BY RICHARD DAWKINS
Richard Dawkins - RichardDawkins.net Comments
Rats Manipulated to be Attracted to Cats
Richard Dawkins - RichardDawkins.net Comments
No Precedent? Then Set One?
Richard Dawkins - for the Press and... Comments
I was invited several months ago to speak at the Faclan Book Festival in Stornoway and I was delighted to accept, as I have a great affection for the Highlands and Islands but have never visited Lewis and have heard such good things about its beauty and about the friendliness of the islanders.
Richard Dawkins - RichardDawkins.net Comments
It's What Moral Philosophers Do
Richard Dawkins - - Comments
Freethinkers, atheists, agnostics, secular humanists – whatever name non-believers go under, they are not America’s most popular minority.
Richard Dawkins - Prospect 188 Comments
Richard Dawkins's review of The Social Conquest of Earth, by Edward O Wilson (WW Norton, £18.99, May) - (with Polish translation)