From the New Yorker 6/30/08:
The images in our mind are extraordinarily rich. We can tell if something is liquid or solid, heavy or light, dead or alive. But the information we work from is poor—a distorted, two-dimensional transmission with entire spots missing. So the mind fills in most of the picture. You can get a sense of this from brain-anatomy studies. If visual sensations were primarily received rather than constructed by the brain, you’d expect that most of the fibres going to the brain’s primary visual cortex would come from the retina. Instead, scientists have found that only twenty per cent do; eighty per cent come downward from regions of the brain governing functions like memory. Richard Gregory, a prominent British neuropsychologist, estimates that visual perception is more than ninety per cent memory and less than ten per cent sensory nerve signals.
I'm reminded of a Visual Anthropology class I took, where we were discussing tribes who had little or no contact with Westerners. A group in New Guinea (I think) was shown a photograph, but it didn't register at all what it was (a photograph of the tribe). Later, when shown a film of the tribe, they got - a very visceral reaction.