A comment on David Chalmers’ ideas about virtual reality
Abstract: I argue that the brain’s construction of our experience is in fact a simulation, and consider some of the implications of this view.
David Chalmers has recently been noticed for his ruminations about whether we live in a virtual reality (or simulation), and whether we could tell. In an interview published in New Scientist, he claims that we could not know whether we are living in a simulation. (1) On CBC’s radio program Quirks and Quarks he was one of three people asked this question (2), and Mary Hynes interviewed him on Tapestry. (3) Several of his discussions around this topic are available online: a search on “David Chalmers virtual reality” will present a dozen or more examples.
My answer is, “Of course we live in a simulation. It’s the one created by our brains.” (4) What’s more, the “I” that experiences this simulation is itself part of the simulation. So it would be more accurate to say that “We live as a simulation created by our brains.”
As asked, the question assumes that “I” is somehow distinct from the simulation. That assumption is misleading. Its explicit formulation dates from Descartes, who assumed that mind (or perhaps soul, he’s not very clear on the distinction) and body are separate. He does this because we experience our bodies. But that experience of my body is the “I” that experiences it. There is no separate experience. That is, there is no evidence that “I” can experience anything other than what my body and brain present as reality, which includes the experience of “I”. Thus, the simulation of reality includes “I”. (5)
Elsewhere, Chalmers has claimed that consciousness is the hard problem. Yes it is, if one assumes that “I” is separate from the reality it experiences. However, if “I” and experienced reality are one, then “I” is what is simulated as the experiencer.
Each of us lives as their own simulation. We can compare those simulations by converting them into other simulations. We can talk about our experience, or make pictures, or replicate the situation in the presence of other people, or ask others to do what we did. (6) What’s important about these comparisons is that we can detect differences, and we can detect them reliably. For example, I don’t know whether you see red or green as I do, but we can tell whether or not we see the same or similar differences between red and green. I don’t know how you see Aunt Emily, but we can both recognise that a photo is or is not a picture of her. We can also tell whether we both see that one portrait is a painting and another is a photograph. And that one shows her as a girl, and the other as an old lady. (7)
Science is the attempt to describe whatever it is that the brain simulates. That is, science is an attempt to create a simulation that is the same for everyone. Thus, a scientific theory is an attempt to eliminate the differences between our individual simulations. Suppose I can’t see the same red/green difference that you see, yet by some method we can both detect the same difference between red and green. One method would be to measure the frequency of red and green light, and agree that we observe the same measurements. By this method we have filtered out those idiosyncratic differences caused by the differences in our retinal cells and brains. We have created a simulation (the measurements) that we have in common. We have replaced our individual “subjective” simulations with a common “objective” one. We infer that the shared method of seeing red/green difference must therefore be closer to whatever it is that our simulation simulates. We call that whatever-it-is “reality”. But in fact all we ever do is compare simulations.
Mathematics works by abstraction. Moreover, it does not describe content, but structure. A mathematical statement expresses a relationship between two or more entities. The red/green difference can be expressed as a difference in wavelength: red is larger than green. This statement says nothing about the experience of red and green that we may or may not share, nor does it say anything about the quality we label “colour”. It states only that one aspect of our experience of the colours red and green is constant. It also states that we must take care to arrange our experience so that this constancy is revealed.
The consequence is that the closer science gets to a universal simulation, the more abstract that simulation becomes. It is finally pure structure. The most abstract simulation is that created by physics. Einstein’s relativity theories are descriptions of structure. Special relativity describes how the shape of one person’s experience can be precisely transformed into the shape of another person’s experience, given that we know their relative velocities. General relativity goes a step further and describes our experience of reality in terms of its space-time structure. Quantum physics describes interactions, that is, the behaviour of entities that behave differently (for example like waves or particles) depending on context. Hence what we can know about them is context bound. What’s more, context defines events and vice versa. Thus, quantum physics describes reality in terms of event-contexts.
But neither theory describes whatever it is that we label reality. They describe structures, the structure of space-time in the one case, and the structure of event-contexts in the other.
Both theories are highly abstract. They are highly reliable and precise in predicting how we will experience those abstract aspects. Hence the belief that these more abstract descriptions are truer descriptions of reality than the more concrete subjective simulations of reality that our brains create. I don’t think that belief is justified. What’s more, I think the question of what’s real is an unanswerable one. We know our own experience, because we are that experience. We can know some of what each other’s experiences have in common, because we can talk about them, or make pictures of them, or express aspects of them in music and dance, or describe them using mathematics.
In the New Scientist interview, Chalmers says “I think, at the very least, virtual worlds [created by virtual reality devices] provide a particularly pure illustration of Descartes’s problem.” Descartes had a problem because he assumed not only that body and mind are separate entities, but that one is physical and the other is not. (11) Chalmers perpetuates Descartes error by asking whether we can know whether we “live in” a simulation. That question makes sense only if “I” and the simulation are ontologically distinct entities. (12) Assuming that distinction begs the question: If “I” is not an essential part of the simulation, then of course we can know “I” is “living in” that simulation. The question is interesting, that is productive, if and only if “I” is an essential part of the simulation.
The brain creates the experience of reality, including “I” as the experiencer of that reality. We can know no other. It is the brain’s creation of our experience that enables “virtual reality” devices. What’s significant about these devices is that they present the “first-person” viewpoint. That is, they present the same structure as the subjective reality that they imitate. They are incomplete, however, because they do not simulate the proprioception necessary to experience the 1st person viewpoint as “I”. (13)
Chalmers ends his New Scientist interview with “A sort of structuralist conception of reality – that the world isn’t intrinsically the way we thought it was, but still has a similar sort of structure – is very strongly suggested by modern science.” I think he’s right. The brain’s simulation of reality is enough like reality, whatever reality “really is”, that we survive quite well. But exactly how much it is like reality can’t be decided. The theories of physics are highly abstract descriptions of some aspects of the structure of our experience of reality, and that is the best we can do.
Notes
(1) Interview with David Chalmers in New Scientist: https://www.newscientist.com/article/mg25333710-900-david-chalmers-interview-virtual-reality-is-as-real-as-real-reality/
(2) Broadcast 2022-03-05. https://www.cbc.ca/listen/live-radio/1-51-quirks-and-quarks/clip/15898721-could-living-computer-simulation-and-were-tell
(3) Broadcast 2022-08-23 https://www.cbc.ca/radio/tapestry/are-we-living-in-a-simulation-look-to-free-guy-not-the-matrix-for-answers-says-david-chalmers-1.6393525
(4) Here’s one explanation of how the brain does this; an online search will garner many more: https://neuroscience.stanford.edu/news/reality-constructed-your-brain-here-s-what-means-and-why-it-matters
(5 ) Every creature with a brain computes some simulation of reality. The bar for accuracy and completeness is rather low. The simulation needs only to be good enough to raise the odds that the creature will survive long enough to reproduce. For example, a frog reacts to a moving fly-size blob, but not to a static one. It will try to catch a raisin tossed past it, but will ignore a dead fly lying near it.
(6) The arts can create simulations of realities that don’t exist, and even of realities that can’t exist. We call these simulations “fictions”. Simulations of reality are called “history”, “physics” “reports”, “portraits”, "descriptions", etc. We have many terms for simulations, because we can make so many different kinds of them.
(7) These comparisons depend on our memories of Aunt Emily. That is, we compare one simulation (the photo of Aunt Emily) with another (our memories of Aunt Emily).
(8) http://www.hep.upenn.edu/~johnda/Papers/wignerUnreasonableEffectiveness.pdf
(9) Wigner writes “My principal aim is to illuminate it from several sides. The first point is that the enormous usefulness of mathematics in the natural sciences is something bordering on the mysterious and that there is no rational explanation for it.” It seems to me that this statement encapsulates Wigner’s puzzlement. I think he fails to notice that mathematics is a component of language. It is not a separate language.
All known languages include ways of expressing or describing number, size, shape, spatial and temporal relations, collections of things (sets), kinship (classification), rates of change, etc. These terms label mathematical concepts. The number and sophistication of mathematical terms or concepts varies between cultures, but all cultures have mathematical concepts. These concepts are used to communicate individual experience just as the rest of language is used.
Mathematics begins as an attempt to regularise trade, property, kinship obligations and rights, etc. That is, mathematics is part of the attempt to find those common elements of our individual experience that satisfy our desire for justice, fairness, equity, connection, community, etc.
Mathematics as a method for enabling and enforcing justice, fairness, equity and so on was formalised in rules of kinship rights and obligations in pre-literate tribal societies, and such rules still make up an essential part of what we experience as our way of life. Literate societies wrote down these rules, and added recipes for calculating the requisite quantities.
Mathematics as a discipline begins with Euclid’s attempt to organise the concepts into a logical structure. This logical structure translates exactly from one language to another.
(10) Linguists refer to “idiolects”, our idiosyncratic versions of our common language. That idiosyncrasy is in fact the definition of “style”. To understand someone is to translate their idiolect into one’s own, a process we perform almost entirely without conscious awareness. That unconscious translation is one source of mutual misunderstanding. See Steiner (1975).
(11) Another problem generated by the mind-body dichotomy is expressed as “How can the non-physical mind cause physical effects?”
(12) Another version of this problem is whether we would ever know whether we were minds uploaded into a computer.
(13) It’s not clear how a virtual reality systems could provide proprioception. Some kind of body-suit could provide external sensory inputs, but it couldn’t provide internal ones. Perhaps inputs through the brainstem could do it, but experimentation would raise interesting ethical problems. The study of the sensations produced by training simulators is instructive: Our brains can and often will construct more complex sensory experiences than the actual sensory data provide.
Bibliography
Many thinkers have stimulated my ideas. I am grateful to them all. The following represents a small selection of the sources most relevant to this paper.
Damasio, A: Descartes' Error: Emotion, Reason, and the Human Brain (1994; revised 2005)
Damasio, A: The Feeling of What Happens: Body and Emotion in the Making of Consciousness (1999)
Gelernter, David H: The Muse in the Machine: Computerizing the Poetry of Human Thought (1994)
Hawking, Stephen: The Theory of Everything (2002)
Hofstadter Douglas & Dennett, Daniel: The Mind's I: Fantasies and Reflections on Self and Soul (1981)
Hofstadter, Douglas: Gödel, Esher, Bach: an Eternal Golden Braid (1979)
Hofstadter, Douglas: I Am a Strange Loop (2007)
Kraus, Lawrence: A Universe From Nothing (2012)
Norman, Donald A.: The Psychology of Everyday Things (1988)
Rees, Martin: Just Six Numbers (1999)
Rosenfeld, Israel: The Strange, Familiar, and Forgotten. (1991)
Sacks, O: The Man Who Mistook His Wife for a Hat (1985
Sacks, O: The Island of the Colorblind (1997)
Sacks, O: The Mind's Eye (2010)
Steiner, George: After Babel (1975)
2022-04-14
2 comments:
Since we can never be an external observer of this simulation we are all in, this is an interesting intellectual exercise, but likely can never be more than that. My personal view is that we do live in reality, but are limited by our senses and brain to only see 3 dimensions plus time. All other dimensions, stated mathematically, are simply beyond our comprehension.
If everyone is in a simulation, it blows my mind that we all agree enough on those simulation parameters that we can play a musical piece as an orchestra and all agree on timing sufficiently to make it listenable. This level of consistency in interpretation of this simulation my ALL (or perhaps almost all) minds implies to me that any differentiation between a simulation and “a partial 4 dimension reality” is inconsequential. It is at most an interesting philosophical question for a while, until we move on to other curiosities….the original thesis of “do we exist in a simulation” is unanswerable.
The fact that we can in fact compare our individual versions of reality, and note where they are similar and where they differ, IMO means that the simulation is accurate. Well, at least in terms of thge sensory data we use. We know that bees can detect UV light, that many mammals can detect sound frequencies beyomd our ewars' ability,etc.
All which means that living creatures do create whatever version of reality meets their needs (the ones that don't, die).
As for playing music together: "Now"s" lags reality by about 1/10th of a second. The brain actually computes where the object will be about 1/10th of a second from "now" plus the time it takes to move your hand into position to catch the ball. It takes humans several years to train the brain to do this.
I think the same kind of prediction occurs when playing music with other people. The one (and only) time I really jammed at a party, I _knew_ where the music was going. Only (prediction + preparation for the next move) can account for this.
Post a Comment