09 The Cog Sci Corner

Damian mentioned how the relationship between sound and vision impacts how people experience the band’s music and videos. And he’s right, you know. We humans have evolved vision to be our dominant sense. The visual cortex is the largest system in our brains. From light signals coming in, to the cognition and application of meaning in our occipital lobe, visual signals are the most deeply processed of all our senses.

However, vision isn’t necessarily our strongest or fastest sense. The chemical senses (smell and taste) have a direct line to the source, since our brains speak in the language of chemical signals. Sound, vision and touch must first translate vibration, light and pressure into the brain’s preferred dialect.

What it lacks in speed, vision makes up for in memory. “That treadmill song?” What Damian’s describing is what we brain-nerds call synaptic strength. Pictures connect to words, emotions and meaning more powerfully than the neural connections in our other senses. We wouldn’t have language or the written word otherwise! Imagine having to learn the meaning of a word each time you see it. Vision and memory are programmed partners. It’s difficult to un-see something… and when it’s stored in memory along with another sensory input, that connection is tough to break.

Yet, for all its advantages, vision can be fallible. Our brain takes shortcuts. The interplay of vision and hearing, in particular, is fraught with processing errors and illusions. The “McGurk Effect” is a classic example. Consider the sounds “faa” or “baa.” You can easily hear the difference between the two syllables. However, if you watch a video of someone speaking these sounds with the wrong audio, your sense of vision fills in the blanks… incorrectly. You can’t overcome it—even if you know it’s happening. Vision tries to make it right, taking a shortcut to ease the load on your brain. It’s similar to hearing a snip of “Here It Goes Again”; you just can’t stop your brain from firing up the meaningful connection to that treadmill visual.

COREY BEILSTEIN

Corey is our resident “brain guy” with a Masters in cognitive/human experimental psychology.