Exploring the Landscape of the Mind: Understanding Human Thought and Behaviour

By Stephen S. Clark

This book is based on the premise that humankind is, first and foremost, the outcome of the process of biological evolution. Recognition of this is fundamental to our understanding of who we are and how we behave. All living things have evolved the physical and mental attributes that promote their prospects for survival; they are good at doing the things that enable them to pass on their genes to succeeding generations, and we are no exception. Of course, through the development of culture, we have gained some freedom from our biological origins. Nevertheless, evolution has constructed the foundation upon which culture is built.

The first part of the book, Ourselves Interacting with the World, presents an overview of the main capabilities that evolution has endowed us with and that enable us to interact with the environment in advantageous ways. This includes our senses, which act as windows on the world and also, of great importance, our emotions and ability to remember. Our ability to think is perhaps the crowning achievement of our evolutionary journey, and, of course, we must be able to act in a timely and effective manner.

The second part of the book, Living Together, traces the history of how we became social creatures. To be truly human, we had to be capable of sharing and cooperation. We also needed to be able to control our aggressiveness and talent for deception. We settled down, making the transition from hunter-gatherers to urban dwellers, and agreed upon values and norms of behavior that enhanced our ability to get along. Ultimately, we came to see good and bad as a morality of right and wrong, further augmenting group cohesiveness.

In the final part of the book, Challenges and Opportunities, attention turns to a consideration of the constraints and possibilities that must be considered in looking to the future. These realities can be seen to play out in four social arenas: the pursuit of fairness, the seeking of justice, the interplay of political beliefs and good government, and ultimately, a united society that is, at the same time, a true community. Our quest for these things will be greatly aided by a deep knowledge and appreciation of our evolutionary past and the indelible imprint it has left upon us. It may even lead us to that most elusive of all things, happiness.

• Language: English
• Publisher: Xlibris AU
• Release date: Apr 19, 2017
• ISBN: 9781524519162

Stephen S. Clark

The author, born and raised in Wisconsin, has a BA degree from Harvard University and a PhD from the University of Michigan (Ann Arbor). Australia has been his home since 1969. His professional career has included working as a curator in environmental studies at the Australian Museum in Sydney and lecturing in ecology and environmental management at Macquarie University. More recently, he has worked with the New South Wales National Parks and Wildlife Service on the conservation of threatened species and biological communities. He is presently living in Canberra, where he plays the saxophone and, together with his partner, enjoys kayaking and a variety of other outdoor activities as well as traveling extensively in Australia and overseas.

Book preview

Copyright © 2017 by Stephen S. Clark.

Library of Congress Control Number:      2016918496

ISBN:                  Hardcover                        978-1-5245-1899-8

                            Softcover                          978-1-5245-1898-1

                            eBook                               978-1-5245-1916-2

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the copyright owner.

Any people depicted in stock imagery provided by Thinkstock are models, and such images are being used for illustrative purposes only.

Certain stock imagery © Thinkstock.

Rev. date: 02/02/2018

Xlibris

1-800-455-039

www.Xlibris.com.au

740474

CONTENTS

Literary Perspectives On Human Nature

Preface

Introduction

Acknowledgements

Part I — Ourselves Interacting With the World

Chapter 1:     Windows on the World

Chapter 2:     Getting Emotional

Chapter 3:     Lest We Forget

Chapter 4:     Thinking About It

Chapter 5:     Just Do It

Chapter 6:     Who Am I?

Part II — Living Together

Chapter 7:     What a Tangled Web We Weave

Chapter 8:     Becoming Human

Chapter 9:     Anger and Aggression

Chapter 10:   From Wanderer to Urban Dweller

Chapter 11:   From Caring to Morality

Chapter 12:   The Nature of Culture

Part III — Challenges and Opportunities

Chapter 13:   Political Beliefs

Chapter 14:   A Fair Society

Chapter 15:   Justice for All

Chapter 16:   Good Government

Chapter 17:   A United Society

Chapter 18:   Finding Happiness

References

Glossary

Biographical Note

3-D BRAIN APP - An excellent visual representation of all the major brain areas is available online. It is called the 3-D Brain App and can be found at sciencenetlinks.com/tools/3d-brain (follow the links ‘launch tool’ and ‘launch online 3Dbrain’). It is possible at this site to select a particular brain area and then rotate it around vertical and horizontal axes in order to see its size and exact position in relation to the rest of the brain.

To my parents,

Margaret and Raymond Clark

In gratitude for the greatest gift of all,

A love of learning

LITERARY PERSPECTIVES ON HUMAN NATURE

Chapter 1. Windows on the World

Marcel Proust (1871–1922) Remembrance of Things Past. The Guermantes Way

Chapter 2. Getting Emotional

Marcel Proust Remembrance of Things Past. Swan’s Way

Chapter 3. Lest We Forget

Marcel Proust Remembrance of Things Past. Swan’s Way

Marcel Proust Remembrance of Things Past. Within a Budding Grove

Chapter 4. Thinking About It

Blaise Pascal (1623–1662) Pensées

Giorgio Vasari (1511–1574) The Lives of the Artists

Chapter 5. Just Do It

Thucydides (460–395 BC) History of the Peloponnesian War

Chapter 6. Who Am I?

Arnold M. Ludwig How Do We Know Who We Are?

Herman Melville (1819–1891) Moby Dick

Chapter 7. What a Tangled Web We Weave

Niccolo Machiavelli (1469–1527) Discourses on the First Decade of Titus Livius

Chapter 8. Becoming Human

William Shakespeare (1564–1616) The Merchant of Venice

Chapter 9. Anger and Aggression

Herman Melville (1819–1891) Moby Dick

William Shakespeare (1564–1616) Henry V

Chapter 10. From Wanderer to Urban Dweller

Friedrich Nietzsche (1844–1900) ‘The Genealogy of Morals’

Chapter 11. From Caring to Morality

David Hume (1711–1776) ‘An Enquiry Concerning Human Understanding’

Friedrich Nietzsche (1844–1900) ‘The Genealogy of Morals’

David Hume (1711–1776) ‘An Enquiry Concerning Human Understanding’

Chapter 12. The Nature of Culture

Michel de Montaigne (1533–1592) The Complete Essays

Chapter 13. Political Beliefs

Giuseppe Tomasi di Lampedusa (1896–1957) The Leopard

Chapter 14. A Fair Society

Marcel Proust (1871–1922) Remembrance of Things Past. Within a Budding Grove

Chapter 15. Justice for All

John Locke (1632–1704) ‘An Essay Concerning the True Original Extent and End of Civil Government’

Chapter 16. Good Government

Alexis de Tocqueville (1805–1859) Democracy in America

Marcus Tullius Cicero (106–43 BC) ‘Epilogue: The Fallen State’

Chapter 17. A United Society

John Stuart Mill (1806–1873) ‘On Liberty’

Chapter 18. Finding Happiness

Aristotle (384–322) Nicomachean Ethics

PREFACE

‘Men ought to know that from the brain, and from the brain only, arise our pleasures, joys, Laughter and jests, as well as our sorrows, pains, griefs and tears. Through it, in particular, we think, see, hear and distinguish the ugly from the beautiful, The bad from the good, the pleasant from the unpleasant ….. It is the same thing which makes us mad or delirious, inspires us with dread, fear, Whether by night or by day, brings sleeplessness, inopportune mistakes, Aimless anxieties, absent-mindedness, and acts that are contrary to habit.

These things that we suffer all come from the brain.’

Hippocrates (5th Century BC)

As a physical and biological scientist, I find it helpful to think of the mind as a landscape. Our reading of a natural landscape can be informed at many levels. If we understand something about geology and the history of the earth or about the evolution of living organisms and functioning of the ecosystems to which they belong, then our perception of any landscape will be immeasurably enriched. So too with our minds. If we are aware of how our minds evolved to work as they do, of why we think and feel and act as we do, then our very sense of being alive will be raised to a higher level.

Our minds have been like a foreign land. Some early thinkers such as Hippocrates had a surprisingly clear glimpse of what might be there but since then (until relatively recently), we seem not to have been very curious about finding out more. Or perhaps it is just that the terrain was so inaccessible to us.

It is often said that we are living in very exciting times because of the rapid advances being made in knowledge and understanding in many areas of scientific enquiry. This is especially the case for research converging on an understanding of the workings of our minds. The answers that are emerging from molecular foundations right up to social organisation and morality are fascinating, but not only in themselves. They also have profound implications right across the whole spectrum of human existence. If we are to succeed in improving society and making a better world, then we must be honest about human nature and the constraints within which we have to work.

Just as the human mind has evolved, so too is our understanding of this remarkable part of ourselves evolving. There is no ideal time to attempt a synthesis in any field of knowledge, particularly one in which advances are being made at such a rapid rate. Nevertheless, it can be a worthwhile exercise to take stock of what has been learned and what remains to be investigated. What has already been revealed is indeed magnificent and prompts a true sense of wonder, even awe, when contemplating where our species has come from and what we are.

Attempts to understand the human brain today draw upon scientific research in a wide variety of areas of enquiry including neurophysiology, genetics, and evolutionary theory as well as cognitive experimental psychology and even reaching into fields such as economics, philosophy, and systems theory. New fields are being carved out in territories at the crossroads of more traditional subjects. Some things, indeed some of the most important things, cannot be thought or written about within the comfortable confines of traditional academic disciplines.

Several important developments have opened the way for such fruitful collaboration and cross-fertilisation. Various kinds of brain imaging make it possible to see which parts of the brain are active during different experimental tasks. This capability has revolutionised our understanding of both the specialisation of brain areas and the complex interactions between them. The techniques of neurophysiology have made it possible to begin to understand the working of our brains at the level of individual neurons. Most significantly, the application of evolutionary theory to brain functioning, asking what our brain has evolved to do, has provided profound insights into why it might work the way it does. It has even been suggested that ‘Knowledge of the adaptive problems humans faced . . . can function as a kind of Rosetta Stone’,¹ making it possible for us to decipher the meaning of our behaviour in terms of our adaptations to the survival problems that confronted us during our evolutionary past. Finally, the sequencing of the human genome has made it possible, for the first time, to catch a glimpse of what makes us tick right down at the most fundamental level of our genes.

It may seem highly presumptuous to attempt an overview of the current state of our knowledge about the human brain and how it works to produce our thoughts and actions. I can only argue in my defence that no one who is the product of a conventional academic training can be fully prepared for newly emergent interdisciplinary areas of scientific enquiry. The task I have set myself is to assimilate the ideas of some of the world’s best minds, past and present, and by ordering and presenting these ideas in a coherent fashion, to make them as accessible as possible to a wider audience. Also, to see the world and humankind at some sort of metalevel, not just as a player in the ruck of life but from above, as it were. The goal has been to understand what animates us, the overall grand design of what and who we are. Importantly, such understanding must be approached fearlessly and with an open mind, not wanting things to be a certain way but, so far as possible, to see and tell it as it is.

In some ways, the task was like an enormous jigsaw puzzle, daunting and almost overwhelming at times, not only because there were so many pieces but because their shapes were not fixed but evolving as scientific understanding grows. Nor did I want to force the pieces into place, to distort them in the process of making them fit. There was a sense of excitement and satisfaction in finding a long-searched-for piece of the puzzle or one that had lain on the table for a long time, which didn’t seem to fit anywhere, and seeing for the first time where it belonged.

This book began as a kind of personal exploration. It is first and foremost a quest for understanding. If my explanations to myself can also be of interest to others, then so much the better. I have endeavoured to present what is known as accurately and honestly as I possibly can, indicating where uncertainty exists. Finding the truth is never easy and yet it is important to maintain a belief that it does exist and is discoverable. The tools of scientific enquiry offer one of the most powerful means at our disposal for doing this. What is written here represents an attempt to summarise the current state of understanding of how our minds work. Some of it will change as new research findings are made but mostly this will, I expect, be at the level of detail. The broad outlines and the overall composition will remain as it has been sketched out.

I would not presume to present myself as the only person to write such a book; it could no doubt be done by others willing to make the necessary commitment and effort. My resolve to embark on such a project is the result of a lifetime curiosity and desire to understand the nature of humankind and our relationship with the world. We, each and every one of us, as we go through life, construct a model as a way of understanding how this world we find ourselves in works. Some models are very simple, others complex, some work well and others not so well, but for each of us they are a life’s work and we become deeply attached to them. The mental model that we construct of the world becomes a kind of user’s guide, a much-thumbed manual for living. It may be patchy and woefully inadequate in parts but we seldom, if ever, notice. It is like a favourite book, an old and trusted friend.

While we may not think about it very often at a conscious level, we must all know in our heart of hearts that when we cease to draw breath, our model configured in the neurons of our brain is erased in an instant and gone forever. Don’t we all wish that after we have gone, some record will exist of who we were and what went on in our heads? It would be nice to think that all that effort in making sense of the world could be of some use and interest to someone besides us. One of the remarkable things about writing is that it helps us to find out for ourselves what we think about something. Of all the members of our species who have lived since we first walked the face of this earth those tens of thousands of years ago, only a very few have written down their thoughts about anything. After all, we have only been writing for a few thousand years. And of those who have troubled to write down their thoughts, an even smaller number are still read today. My guess is that the ones that we still do read were able to say something about us and our world that remains true even today.

Despite all that science has contributed to our understanding of how our minds work, it has its limits. Our minds are also capable of insights that do not depend upon the hypotheses, empirical observations, and analysis of data that are the tools of the scientific method. Storytelling has been an important way of understanding ourselves, probably for almost as long as we have been able to speak. The invention of writing was of enormous significance because it freed us from a dependence on an oral tradition; our stories could be recorded for all posterity. Some of these stories we recognise as great literature because they contain lessons about who we are and how we behave that are relevant to any time and place. It therefore seemed appropriate to include some of this treasure house of wisdom along with the insights of science.

The messages couched in the language of science and literature often need repeating because we are a wilful species, entirely capable of rejecting what doesn’t suit us at any particular time or place. Perhaps the greatest gift of writing is that it makes it possible for us to revisit and rediscover the understanding and insights from the past. So this book is offered as a small contribution to this growing body of knowledge that now stocks the libraries of the world. Of course, not all of it will be seen to be the truth by everyone alive today, let alone future generations! Nor, for that matter, is everything filed away in the world’s libraries the truth. The best we can do is to try our honest best to find and express it but in the end to have the good grace to let time be the judge.

When we are young, we want to save the world and to do this we must change it. The world, however, doesn’t always want to change in the way we think it should. Later in life, we may become more realistic in our ambitions. For me now a worthy and more interesting goal is to try to understand the world, to place no conditions on it but to see it as it is rather than as we would wish it to be.

I had a more modest view of my book and it would be incorrect to say even that I was thinking of those who might read it as ‘my readers’. For to my mind, they would not be my readers but the very readers of themselves, my book serving only as a sort of magnifying glass, such as the optician of Combray used to offer to a customer; my book might supply the means by which they could read themselves. So that I would not ask them to praise me or to speak ill of me, but only to tell me that it is as I say, if the words which they read within themselves are, indeed, those which I have written.

Marcel Proust, Remembrance of Things Past. Time Regained. trans. Andreas Mayor. Chatto and Windus. 1927.

Stephen S. Clark

1 December 2016

1.   J. Tooby and L. Cosmides, ‘The Psychological Foundations of Culture’ (chapter 1), in J. H. Barkow, L. Cosmides, and J. Tooby (eds.), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (Oxford University Press, 1992), 19–136.

INTRODUCTION

This book is based on the premise that humankind is first and foremost the outcome of the process of biological evolution. Recognition of this is fundamental to our understanding of who we are and how we behave. All living things have evolved the physical and mental attributes that promote their prospects for survival; they are good at doing the things that enable them to pass on their genes to succeeding generations and we are no exception. It is of great importance at the present time that we understand and accept this. Of course, through the development of culture we have gained some freedom from our biological origins. Nevertheless, evolution has constructed the foundation upon which culture is built. Our continued survival depends upon a full appreciation of this reality and the constraints that it imposes upon us.

The first part of the book, ‘Ourselves Interacting with the World’, presents an overview of the main capabilities that evolution has endowed us with and that enable us to interact with the environment in advantageous ways. This begins with our senses, including seeing, hearing, touching, and tasting, which act as windows on the world. We then go on to consider our emotions which make possible a timely response to unexpected challenges, and our memory, which opens the way to a knowledge of the past and an ability to imagine the future. Thinking is perhaps the crowning achievement of our evolutionary journey. It enables us to construct a mental model of the world and how it works, to solve problems and decide what to do next. Thought, together with memory, sets the stage for action. We learn from the past; rewarding and punishing experiences offer a guide to what to do and what not to do in the future. One of the most remarkable attributes we possess is a sense of self which enables us to see ourselves as actors in the world. Equipped with this ability, we can step into the minds of others and even empathise with them, one of the foundations of social behaviour.

The second part of the book, ‘Living Together’, traces the history of how we became social creatures. Before we could become truly human, it was necessary for us to develop an enhanced ability to care for others, to be willing to share and cooperate. This was a development of major significance because being able to live amicably with others of our species was crucial to our survival. These newly emergent qualities, however, did not mean that we left our evolutionary past behind us. We retained and, if anything, refined our talent for deception, anger, and aggressive behaviour, and these traits remain serious problems for us to this day. Nevertheless, through caring and the need to belong to the group, we were able to become trusting of one another, to agree upon values and norms of behaviour that enhanced our ability to get along. We settled down, making the transition from hunter-gatherers to urban dwellers. Ultimately, we came to see good and bad as a morality of right and wrong, further augmenting group cohesiveness. Through the development of culture, we freed ourselves, to a degree, from the biological imperatives that had gripped us. Nevertheless, we would do well to remember that in all that we think and do, we still dance with the ghosts of the past.

In the final part of the book, ‘Challenges and Opportunities’, attention turns to a consideration of the constraints and possibilities that must be considered in looking to the future. Though we get along remarkably well, it must also be accepted that conflict of interests is a human universal; individually, we want different things and there is also the difficult challenge of balancing the interests of individuals vs. the interests of the social group as a whole. These realities can be seen to play out in four social arenas: the pursuit of fairness, the seeking of justice, the interplay of political beliefs and good government, and ultimately, a united society that is at the same time a true community. Our quest for these things will be greatly aided by a deep knowledge and appreciation of our evolutionary past and the indelible imprint it has left upon us. Such an understanding may also lead us to that most elusive of all things, happiness.

ACKNOWLEDGEMENTS

I am grateful to Katherine Betts of Swinburne University for reading some of my early drafts of chapters and for her kind encouragement at the very beginning of my project.

I particularly wish to thank Cobie Brinkman of the Research School of Psychology at the Australian National University for her careful and thorough reading of the chapters in part 1 and for her many helpful insights, comments and suggestions. Our wide-ranging discussions of topics in biology and evolution have been one of the most rewarding and satisfying aspects of writing this book.

Jen Badham of Critical Connections has also been kind enough to read chapters in parts 2 and 3 and made numerous thoughtful and valuable comments on sociological and anthropological topics.

I also owe a debt of gratitude to the staff of Xlibris. Their continued support and professionalism throughout the process of bringing this book to fruition have made the book much more than it otherwise would have been.

It is also, I believe, appropriate to acknowledge the many researchers and writers, experts in their fields, whose books have inspired and informed me. Without their work, I could not have even contemplated this book. They include

William Calvin, Carlos Castaneda, Patricia Churchland, Antonio Damasio, Richard Dawkins, Daniel Dennett, Robert Edgerton, Loren Eiseley, Niall Ferguson, Francis Fukuyama, Frank Furedi, Willard Gaylin, Michael Gazzaniga, Eric Hoffer, Susan Jacoby, Daniel Kahneman, Peter Kramer, Joseph LeDoux, Arnold Ludwig, William I. Miller, Charles Murray, Steven Pinker, Loyal Rue, Roger Sandall, Roger Scruton, Daniel Wegner, Diana West, Timothy Wilson, and Robert Wright.

The book Principles of Cognitive Neuroscience by Dale Purves, Elizabeth M. Brannon, Roberto Cabeza, Scott A. Huettel, Kevin S. LaBar, Michael L. Platt, and Marty G. Woldorff has been an invaluable source of information as a comprehensive overview of neurobiology.

Grateful acknowledgment is made to the following publishers for permission to reproduce extracts from previously published material:

Princeton University Press: ‘Epilogue: The Fallen State’ in How to Run a Country. Marcus Tullius Cicero. trans. Philip Freeman, 2013

Penguin Random House: The Leopard. Giuseppe Tomasi di Lampedusa, 1988

Oxford University Press: How Do We Know Who We Are? A. M. Ludwig, 1997

Brain image on cover: Brain in Profile (by pearish)

Human Brain Sagittal View Medical Sketchy Illustration by Alena Hovorkova used under license from Shutterstock.com

I have endeavoured in everything that I have written to be accurate and to avoid misrepresenting the research findings of others that I have reported. I alone am responsible for any errors or inaccuracies that remain.

Finally, I want to thank my partner, Deborah Metz, who understood how important writing this book was to me, believed that what I was doing was worthwhile, and provided a place in our lives to make writing it possible.

Cover Photo – Deborah Metz

Cover Design – S. Maroc, graphic designer, Majorlook Design

PART I

Ourselves Interacting With the World

1

Windows on the World

The task of sensory systems is to provide a faithful representation of biologically relevant events in the external environment. The physical signals transduced by receptors are very different from the biologically important information in the environment. The sensitivity and precision of sensory transduction* are remarkable but the true challenge of our sensory systems is to make accurate inferences from the receptor signals. These inferences are richer and simpler than the basic measurements of light intensity, force and chemical composition. They are richer because they contain representations of objects, states and events that are abstracted from the primitive sensory signals; they are simpler because they represent the distillation of the vast quantities of raw measurement information offered to the central nervous system by each sensory surface. The fundamental questions of sensory neuroscience are: What are the computations that extract meaning from the incoming receptor data? What neural mechanisms perform these computations? In what ways do different sensory systems operate differently and in what ways are they the same?¹

*Transduction is the process that takes place in cells which converts incoming energy into neural signals.

Introduction

It is easy to take our sensory systems (sight, sound, smell, taste, touch, balance, and proprioception) for granted. They operate continuously during our waking hours, most of the time without any particular conscious effort on our part. And yet they are absolutely vital to our existence as windows through which we receive glimpses of our surrounding environment. The various forms of energy in the environment, electromagnetic, chemical and mechanical, constantly bombard us with signals that convey information of great potential relevance to our survival and well-being. Such signals offer us clues about what to approach and what to avoid. An advancing understanding of how our brains transform and interpret that information to guide our behaviour is one of the triumphs of cognitive neuroscience.

Our senses do not capture anything like all the information that is on offer in our surroundings. We know, for example, that our vision is confined to a relatively narrow band of the spectrum of electromagnetic radiation. The invention of telescopes and microscopes has also made it clear how limited our powers of sight are, even within that band. As part of our general knowledge, most of us are aware that other animals have sensory abilities greater than ours; hawks are able to see further, dogs have much more discriminating noses than we do, and bats detect sounds that our ears cannot capture. It is clear from such observations that, subject to some constraints, evolution has equipped each species with sensory systems that deliver the information it needs to survive in prevailing environmental circumstances. It is perhaps of equal importance that evolution has ensured that we ignore information that will do little to enhance our prospects.

Even within the relatively narrow range of stimuli available to us, there is far more information than we could possibly manage to process with the brainpower at our disposal. A further likely limit is that our brains consume a considerable amount of energy in producing neural representations of information. While they only amount to 2 per cent of body weight, they consume 20 per cent of our resting oxygen requirement.² It is easy to appreciate how much about our surroundings we don’t take in by, for example, mentally reviewing the route we travel each day to work. Of course we process enough information to find our way by means of visual cues, but if questioned, how much detail could we provide about the houses along the way, their designs, materials, and colours? As the popular expression has it, we tune in or tune out to things. In the language of psychology, we have attentional systems that alert us to non-routine signals from our environment such as the siren or flashing lights of an emergency vehicle. We readily distinguish such sounds or sights from the general sensory ‘white noise’ that we are continuously bathed in and, thankfully, are able to ignore. Our powers of attention also enable us to focus consciously on particular sensory stimuli, for example, to identify an unclear sight or sound or make an effort to listen carefully to a concert hall performance or to study for an exam.

It is of fundamental importance to appreciate at the outset that the mental representations we construct from the information received via our senses bear only an indirect relationship to the real world around us. This must be so not only because of the already mentioned limits to the range of sights and sounds and so on available to us but also because our sensory systems are limited in their ability to resolve all the ambiguities that arise from the complexities of the real world. For example, although the world undeniably is three-dimensional, our vision must initially represent it in the two dimensions of the surface of the retina. As we will see, other parts of our brain receiving this information then construct a three-dimensional image that approximates what exists in the real world. This is not as alarming as it might seem. Our survival depends on our ability to perceive things pretty much as they are, but it would be unwise to assume a 1:1 correspondence between the world out there and our perception of it. Our experiences of the real world need not precisely resemble it; it is sufficient that they provide a guide to behaviour that meets our needs for survival.³

However, before any processing of sensory information even begins, the incoming energy from the environment passes through various kinds of collecting devices that channel and even amplify the signals we receive. For example, the eye acts like the lens of a camera to focus and filter light before it reaches the retina. In a similar manner, the shape of the outer ear and the specialised bones of the middle ear act to gather and amplify sound, the hairs and ridges on the skin surface are receptive to mechanical forces. Our sense of smell and taste are aided by the structure of the nose with its internal mucous membranes designed to maximize interaction with volatile airborne molecules, and the taste buds of our tongue that present taste receptors to the water-soluble molecules that we take into our mouths.⁴

It makes perfect sense, though we scarcely think about it, that our various sensory organs are located at our front end, the part of us that first moves into a new environment.⁵ Olfactory receptors are strategically placed at the front end of the respiratory system. Eyes and ears are also positioned at the front end, as is the vestibular apparatus, by which we keep our balance, located within the inner ear. Quite logically, taste receptors and their sensory nerves are at the front end of the digestive system.

Our Nervous System

In order to understand how our brains are able to convert the signals coming to us through our senses into meaningful information, it will be helpful to know some very basic things about our nervous system. Our brains are thought to contain 100 billion nerve cells or neurons⁶ interconnected by about 100,000 kilometres of axons⁷ (long appendages that extend from the cell body containing the nucleus). Each neuron can potentially connect to a large number of other neurons making up a complex network of as many as 100 trillion connections.⁸ A dissection of the human brain reveals easily distinguished areas of white and grey matter. The grey matter covers the surface of the brain, forming the cortex, and is also found in pockets in the internal part of the brain. It contains a very high density of the nerve cell bodies and also of the synaptic connections between neurons. It is here that information is processed and encoded. In contrast, the white matter beneath the surface contains the neuronal axons covered with insulating myelin sheaths. These axons, ranging in length from millimetres to a metre or more, serve to connect up the various parts of the brain and to transmit information in the form of electrical signals between them.⁹ This information is received by appendages of other neurons called dendrites. The dendrites are generally shorter than the axons and more numerous, making it possible for a neuron to be connected to a number of others neurons. This number varies widely (from 1 to 100,000!) according to the function of the neuron.¹⁰

The broad pattern of connections in these neural circuits is predetermined by a genetic program that unfolds as our brains develop,¹¹ ultimately resulting in the human nervous system in all its complexity. Axons carry information from our extremities to the spinal cord and brain that together form the central nervous system (CNS). They also carry information in the opposite direction, from the CNS to the outer parts of our bodies. The neural circuits are organised into systems that perform different functions for us. The subject of this chapter is the sensory systems that serve the purpose of bringing information from the different sensory receptors (eyes, ears, etc.) to the CNS and transform and interpret that information in a way that is relevant and useful in directing our behaviour.

Though many aspects of our behaviour are built into our nervous systems from the beginning, it is now believed that some aspects of this wiring-up are not completed until we are well into our twenties. There is, therefore, still a great deal of scope for experience and learning to fill in the detail of which synaptic connections will be strengthened (and which might be pruned away). This process of incorporating what we experience and what we have learned as we go through life makes each of us who we are: the way we think, our attitudes and beliefs, and how we behave. It makes each of us unique.

The processing of incoming information begins with specialised receptor cell neurons for each of our senses that generate an electrical signal (a wave of electrical energy called an action potential) in response to a sensory stimulus. The frequency of the electrical signal is related to the strength of the stimulus. A strong stimulus also makes it more likely that the firing of this neuron will induce other neurons to fire. Their activation in turn carries the signal further along the neural network dedicated to that particular sensory input toward the CNS. Along the way, at certain points, the information undergoes various transformations that enhance its usefulness. Ultimately, the information reaches areas of the brain specialised for each of the senses, where final processing culminates in a refined neuronal representation of the initial stimulus information. This is in effect a distillation that we can use as a guide to appropriate action. Remarkably, it is only at the very end of this complex process that we become conscious of what emerges as a perception.

In order to understand more fully what happens we need to look at the area of contact between neurons, where, to use Diane Ackerman’s colourful phrase, they ‘shake hands’.¹² The area between the axon of one neuron and the dendrite or cell body of another neuron is called a synaptic cleft. At this junction, the electrical signal must be transformed into a chemical signal in a complex process involving chemicals called neurotransmitters. When the signal arrives at the specialised ending of the axon called a bouton, it prompts the secretion of neurotransmitter molecules which travel across the synaptic cleft. On arrival at the receiving neuron, these molecules bind to specialised neurotransmitter receptors and when a sufficient number do so, an electrical signal is produced in the receiving cell.¹³ Remarkably, this biochemical mechanism for the transfer of electrical signals between neurons accounts for virtually all the transfer of information in the human nervous system.¹⁴

It is important to note that these signals can either excite or inhibit activity in the receiving cell. The inhibition of certain signals can be a means of filtering out noise, a process that will act to sharpen other signals. Most importantly, at any one time a cell may be receiving inputs from a number of other cells that will collectively determine the cell’s response. This control of the activity at synapses, termed modulation, is accomplished by means of more than 100 different neurotransmitter molecules and a great many different receptor molecules. Some of these neurotransmitters,¹⁵ such as dopamine, serotonin, acetylcholine, epinephrine, and norepinephrine, will be considered in detail in later chapters. Together with receptors, they act in concert to control the response of neurons to input. A neuron is able to ‘integrate the electrical information provided by all the excitatory and inhibitory synapses acting on it at any given moment. Whether the sum of synaptic inputs results in the production of an action potential depends on the balance between excitation and inhibition.’¹⁶

Vision

It has been argued that people vary in the degree to which they rely on their different senses for information (being more or less visual or auditory, for example). Nevertheless, the visual cortex in the brain takes up more space than all the other senses do together.¹⁷ Other lines of evidence argue strongly for the pre-eminence of vision. According to Martin Conway, 80 per cent of our memories come to mind as visual images.¹⁸ It has also been argued convincingly by Andrew Parker that the evolution of eyes and the consequent ability to see led to an explosion in evolution and a proliferation of life forms at the end of the Cambrian era not experienced before or since.¹⁹ Over the course of five million years (a brief period in geological time) the number of major groups in the classification of animals increased from three to thirty-eight. Seeing evidently matters a great deal.

How something as complex as the eye could have evolved is difficult to comprehend and this difficulty has even been used by creationists as an argument for the existence of God. Charles Darwin, viewing the problem from an evolutionary perspective, was able to come up with a perfectly plausible explanation.²⁰ Evolution proceeds by tiny increments, working on chance mutations. All that is required is that each small change to an organism confers some survival value. By this line of reasoning, an eye could begin merely as a tiny light-sensitive patch of skin which subsequent mutations could modify little by little, making a series of small advances leading to such improvements as focusing ability and colour sensitivity that eventually resulted in the eyes we have today.²¹ The chance nature of this process is revealed by the fact that there are many different kinds of eyes in the animal kingdom, each representing a different way of achieving the highly desirable ability to see the world around us. Our vision responds to only a small part of a much larger spectrum of electromagnetic energy which, significantly, corresponds to frequencies that pass readily through water. This coincidence can most plausibly be traced back to our distant ancestors who developed vision before leaving the ocean for terrestrial habitats.²²

Perhaps the most useful property our sense of sight is its ability to give us information at a distance. It can act as a kind of advance warning system, alerting us to the presence of something dangerous or desirable before it is upon us. Even so, things we might wish to see can be easily obscured and our other senses such as hearing or smell often act in a complementary fashion. A moment’s thought should serve to convince us that we often use our senses in this way. Comparing and merging information from seeing and hearing, for example (or touching and seeing), can often resolve uncertainty and provide the identification we seek to make an informed decision.

Of fundamental importance is the ability to control the amount of light entering the eye which is achieved by the dilation or contraction of the pupil and the iris. As already noted, actual seeing begins with the eye focusing light on the surface of the retina. Our eyes are equipped with a lens that does just this by changing its shape through the contraction and relaxation of muscles attached to it.²³ The retina itself, though microscopically thin, is made up of layers of cells. These include receptor cells that, by means of chemical reactions, convert the incoming light energy into electrical signals and other cells that send those signals out of the eye to parts of the brain for processing.²⁴

There are two types of receptor cells, called rods and cones. The eye contains an almost unbelievable number of these cells: approximately 120 million rods and 6 million cones.²⁵ Part of their role is to enable us to respond with sensitivity to both low and high light intensities. The rods come into play at night-time or when we enter a darkened room and the cones are active in broad daylight or artificially illuminated circumstances. Because of their differences in sensitivity to light intensity, they also provide information about the brightness of what we are observing. In addition, and most importantly, the cones provide the basic information that enables us to perceive colours. The rods and cones of the retina can be thought of as initially providing a pixellated image or, if you prefer, something that can be likened to a pointillist painting by Seurat, which will receive further processing and refinement in other parts of the brain.

Another remarkable function performed by our eyes is adjusting the degree of resolution of visual information according to how much detail we need. The part of the retina that does this is called the fovea and it possesses a much higher concentration of cones than the rest of the retina. In this small area, the cones number in the tens of thousands and make possible high-resolution colour vision in the immediate vicinity of our direct line of sight under conditions of high light intensity. In effect, we are equipped with a kind of magnifying glass or spotlight which we can move around a visual scene by shifting our line of sight to areas we need to see in more detail. We pay a price for this capability, however, in the processing of this detailed information, which consumes more than half of the computing resources of the visual cortex.²⁶

The initiation of sensory signals in our eyes is only the beginning of a sequence of neural processing stages that eventually determines what we become aware of seeing.²⁷ It is important to keep in mind that most of the sensory stimulation that we receive will never reach the level of consciousness but may still influence our behaviour. Extensive processing takes place at various levels between the sense receptors and the cortex. In the case of vision, considerable processing, filtering, and sharpening of the information occurs in the ganglion cells of the retina even before it progresses to the thalamus.

The thalamus is the first subcortical way-station for visual information. As we will see, it also receives input from all other sensory systems with the exception of the olfactory system. This highly important cluster of neurons relays information to cortical areas through discrete pathways that are kept separate for each sensory system. Visual information, after the initial processing in the retina, passes to the part of the thalamus called the lateral geniculate nucleus in two separate streams, one for motion and the other for image form and brightness and details such as colour. The thalamus does not simply pass this streamed information on; it is believed to perform a further filtering and sharpening function. It also receives higher-up signals from cortical areas that are thought to exert an influence on the information passed on for further processing in the cortex.²⁸

The areas of the cortex that initially receive signals from the thalamus are termed primary sensory cortex. The primary visual cortex (V1) is located in the parietal lobe. It is here and in surrounding areas of association cortex that higher-order processing of visual information received from our eyes commences. Remarkably, the relative positions of the parts of an image on the retina are maintained and are mapped onto the cortical areas that process the information.²⁹ While the spatial relationships are maintained, there is some distortion in the relative size of the cortical areas devoted to processing the information coming in from the peripheral receptors. This is a consequence of the previously mentioned greater density of peripheral receptors in the fovea. It is important to bear in mind that in all the sensory modalities, what we perceive in the end is not merely a reflection of the information obtained by our peripheral sensory neurons. Rather it is the result of a complex process of interpretation of that information and the creation of a representation of reality that serves our purposes, the ultimate test of which is the enhancement of our survival prospects.

At first, this idea of constructing what we see (and indeed perceive with all our other senses) is difficult to grasp intuitively. It seems obvious that what we see is simply what is out there, and yet compelling evidence suggests otherwise. To begin with, we know that the image projected onto the retina of the eye can only be two-dimensional. At high magnification, the surface of the retina with its rods and cones appears like the kind of pixellated picture we have all come to know through enlarging digital camera images. Beginning from this and employing various rules, we build our representation of reality which is so vivid and convincing that we are compelled to believe in it. All of this happens without our having to give it a thought. The most plausible explanation for this ability is that evolution has to some degree built-in interpretive ‘rules of thumb’ or heuristics for making advantageous interpretations of the information available to us. There is most probably also an element of learning involved; with practice, we are able to resolve ambiguities in sensory signals we receive, on the basis of what such signals meant when we previously encountered them.

Brightness

Although the processing of different visual qualities (image brightness, colour, form, dimensionality, and movement) proceeds in parallel, brightness, or the perceived intensity of light, is a logical place to begin to look at what happens. In complete darkness we, of course, see nothing. Yet the threshold at which receptor cells react to light is astonishingly low. The high sensitivity of the cones clustered in the fovea of the eye is demonstrated by the fact that if a single cone captures just ten photons (a photon is a unit of electromagnetic radiation) we will see light.³⁰ The rod photoreceptor cells must be sensitive to even lower light levels as they play a vital role in enabling us to see in near-dark conditions. In such circumstances, however, we perceive colour only poorly because the rod cells contain only a single photopigment called rhodopsin. The different responses to light intensity of rod and cone cells in the retina must provide the information that we perceive as brightness, but it has proven difficult to identify any specific brain area where this processing takes place.³¹ Furthermore, researchers have discovered that there is no simple relationship between the brightness we perceive and light intensity. Areas of our visual cortex appear to respond more to contrasts between light and dark rather than light intensity. While this emphasis on the brightness of different parts of an image in relation to one another may seem strange at first, one can infer that such information has over evolutionary time proven to have greater usefulness in interpreting the real world than that provided by a simple light-meter approach to light intensity.

Colour

From the primary visual cortex (V1), information goes to a number of separate areas in the vicinity of V1, each of which is involved in the processing of different aspects of sight. With regard to colour, three different types of cone photoreceptor cells can be distinguished, each possessing a different kind of pigment molecule that responds to distinct (but overlapping) ranges of light frequency in the visible spectrum with high, medium, and low frequencies corresponding to blue, green, and red. This information comes from the retina, via V1 to V4 where further processing takes place, enabling us to perceive colours as we do. As with brightness, our perception of particular colours is not precisely related to the wavelengths of light energy being emitted but is influenced by the context of other colours surrounding them.³² Colour blindness is linked to defects in the genes that code for one or more of these pigment molecules in the retina.

Objects

The perception of the form of objects presents another problem for our visual cortex, and some progress has been made in understanding how we do this. All sensory neurons have what are termed receptive field properties; that is, a certain set of characteristics of a particular stimulus (light, sound, etc.) to which they respond. Of relevance to our perception of form is the discovery of neurons in V1 that respond to line segments of a particular orientation or length; such neurons can be highly specific in their preference. For example, differences in orientation of several degrees can be distinguished. Lines, of course, can only be the beginning of a process of carving up into identifiable objects the millions of neuronal signals that come from an image on our retina. We must also detect curves and, significantly, neurons in visual area V2 have been found that respond to borders. Psychological experiments have revealed that we work with shapes and parts of shapes using rules that emphasise certain aspects of form such as parallel elements, convexity, and symmetry but with sufficient flexibility to allow for change in shapes as objects move about in space.³³ All these things work in concert with brightness, colour, and texture to bring out and isolate objects from the general background.³⁴ One can only marvel at how we manage to do all this virtually instantaneously. The inescapable conclusion in this, as in other sensory perceptions, is that we must interpret inherently ambiguous information conveyed through our senses and thereby construct a reality to serve our purposes.

Dimensions and Depth Perception

There is a fundamental ambiguity in the lines that we make use of in our synthesis of shapes and their parts. We cannot know the exact length of any linear feature in the real world projected onto the retina because we do not have any information about how far away it is or its orientation in the third dimension of depth. Nevertheless, we are adept at perceiving the world around us as three-dimensional despite the fact that we must work from a retinal image that has only two dimensions. To some extent, we can work this out from the position of objects; those partly obscured by something else must be further away. It also helps to know something about the actual sizes of things so that, for example, when they appear smaller than we know them to be, they must be at some distance. Most useful of all is the fact that our two eyes facing forward and separated by a small distance enable us to see the same thing from two slightly different angles. Our visual system is able to compare these two viewpoints and compute from them a single three-dimensional image.³⁵ Since it is known that information from the left and right eyes is kept separate in the thalamus, this fusion must take place at some higher cortical level and area V2 appears to have a role to play in this.³⁶

A profound insight into how we construct a 3-D image from 2-D information was gained by Renaissance painters through the working out of the rules of perspective. Though there may have been some early attempts at dimensionality (e.g. Roman murals), from the time of the early art of civilisations such as the Egyptian up until the early Renaissance, artistic images essentially presented their subject matter in two dimensions and have therefore always appeared to us as rather flat and lacking in depth. The remarkable thing that painters such as Piero della Francesca were able to do was to employ the same rules of perspective that we unconsciously use to make our world three-dimensional and thus enable us to see their paintings in the same way.

Motion

As important as the aspects of visual perception considered above undoubtedly are, having the ability to interpret information about movement in our environment must be regarded as absolutely vital. Anything in motion has the potential to impinge upon us with adverse consequences. Similarly, anything desirable that possesses mobility can only be obtained if we have a finely honed sense of what direction it is moving in and how fast. Systems for perceiving and interpreting information about motion are not unique to our species and, therefore must have been developed in the distant evolutionary past.

Just as we are limited in our perception of light to the relatively narrow band of the visual spectrum, we are only able to perceive motion within certain limits. We don’t see bullets fired from a gun, for example, nor do we perceive sequential images such as those projected in movies as continuous movement unless the interval between them is between a tenth and a half of a second. At shorter intervals, images will appear simultaneously and without any appearance of movement. At intervals greater than a half a second, the images will appear sequentially and without movement.³⁷

Because our eyes are only able to register information in two dimensions, when an object moves across our field of vision, there will be a component of movement either toward or away from us that will not be apparent. As this component increases, it will be more difficult for us to work out the direction and speed of any movement. This is why the lights of aircraft moving directly toward us at night give the appearance of motionlessness even though we know they must be moving and why tennis balls travelling directly at a player at over 100 km/hour may be difficult to return.

Though our understanding of how the brain processes information about motion in our visual field must be regarded as preliminary, it is clear that, as with colour, certain areas of visual cortex are specialised for this purpose. In particular, areas MT, MT+, and MST actively respond to moving images. Further confirmation comes from studies of individuals with damage to MT+ movement areas who have trouble perceiving motion.³⁸

Putting It All Together - Object Recognition

Although a great deal of research remains to be done before we will fully understand how the different qualities of vision (brightness, colour, form, depth, and motion) are assembled into a unified visual scene, it does appear that there are, broadly speaking, two processing pathways. These have been termed the ‘what’ pathway and the ‘where’ pathway because they lead ultimately to our ability to identify objects and to locate them in space.³⁹

These pathways originate at the very beginning of our visual system in the retina. Mention has already been made of the rod and cone receptor cells here that contain pigments and convey information about brightness and colour. There are, in addition, further layers of cells (horizontal, bipolar, and amacrine) that filter and process information before it goes via the optic nerve to the thalamus.⁴⁰ Further processing of this information takes place here in the lateral geniculate nucleus, where distinctive cell layers termed magnocellular and parvocellular (and more recently koniocellular) have been identified. The cells in parvo and konio layers appear to be specialised for colour and form information while those in the magno layer are thought to transmit information about motion, depth, and brightness. These beginnings of what ‘what’ and ‘where’ information are then conveyed in separate pathways to V1.

In V1 (and also in V2) different specialised cell types have been identified that process information about colour and form and also motion and depth; however, the linkages between these two kinds of processing are complex and as yet poorly understood.⁴¹ It may be that already at this point the integration of different visual qualities has begun. Nevertheless, two pathways remain discernible, one of which proceeds from V2 to V4, the main area for colour processing. This ventral (lower) pathway leads to the temporal lobe of the cortex and culminates in various kinds of object recognition (faces, living things, and inanimate objects). The other dorsal (upper) pathway progresses from V2 to V3 (associated with depth and distance perception), V5 (motion processing), and finally V6, where the precise position of an object is determined.⁴²

It is inevitable that, with greater understanding, the two-pathway view will turn out to be a great simplification. The different aspects of visual perception complement each other and interact in object identification, movement, and position. The information about colour, form, depth, and movement must somehow be orchestrated in what are termed our association cortices with information about each visual quality making a contribution through what is perhaps a series of iterations of increasing refinement until identification and recognition occur.⁴³ Exactly how this takes place remains something of a mystery and has been termed the ‘binding problem’ by researchers.

This visual pathway does not convey information in only one direction. Learning must also play an important role in this process. Our storage of memories which grows with experience and learning constitutes a database that can be consulted to confirm or reject various interpretations as our brains strive to identify what our senses perceive. There is some agreement among researchers that we store our long-term memories about things in areas of the cortex that were involved in the initial perception of them; thus information about what something looks like is kept in or near visual cortex, information about what something sounds like is stored in or near auditory cortex, and so on. As we will see in the chapter on memory, any one of these aspects of what we know about something, each fragment of the whole, can serve as a cue to bring back an entire memory and, with it, recognition. At the same time, the fragments of different objects of memory stored in the different sensory cortices may have some very strange bedfellows based on a passing similarity in the sound they make or an aspect of their appearance. This could account for lesions in particular brain areas leading to loss of recognition of rather bizarre combinations of things that seem unrelated in appearance or function. It appears that each of us may develop our own filing system according to what we know about different things.⁴⁴

Further evidence for higher cortical areas contributing to what we see can be found in our visual imagination. We are able to see something in our ‘mind’s eye’ without it having to be present. For this to be possible, other brain areas such as the temporal, parietal, and prefrontal cortex must be recruited. Finally, the attentional control that we exercise over what we see, dependent on higher cognitive areas, must come into play as well. This ensures that routine aspects of our environment can be ignored, while at the same time, anything out of the ordinary will immediately attract our attention.⁴⁵

Hearing

Perhaps the most important contribution hearing makes in conveying information about the world around us, at least in our recent evolutionary past, is in laying the foundation for language. It would be hard to overestimate the significance of language and the role it plays in communication with other members of our species. Music, which most probably predated language, is also of inestimable importance to our species. Why this should be so remains something of a mystery, since while music has considerable aesthetic importance for us, it is hard to see what survival value it might confer. A clue may lie in the fact that music has great power to move us emotionally⁴⁶ and it is at least possible that through music a kind of emotional communication with others was possible even before language.

Music is a controlled outcry from the quarry of emotions