TLDR: Joscha Bach - Artificial Consciousness and the Nature of Reality

A summary of the June 13, 2020 episode of Lex Fridman's AI Podcast.

This is a summary of episode #101 of Lex Fridman’s AI Podcast — a conversation with Joscha Bach titled ‘Artificial Consciousness and the Nature of Reality’. Do check it out, as this is not an exhaustive summary. I only covered the content that was interesting to me — which as it turns out is still a lot! So strap in, this is quite long for something called a TLDR.

A list of topics; feel free to browse in any order:

• Gödel, Turing and Wittgenstein

• General Intelligence and the Turing Test

• We Exist in a Story That the Brain Tells Itself

• Describing Reality From Inside a Mandelbrot Fractal

• Only a Simulation Can Be Conscious

• What is Consciousness?

• Everything Else Is Downstream From Controlling Attention

• Ego Death

• Access to Internal States and Emotional Coupling

• We Perceive Things That Are Improbable Yet Possible

• The Older We Get, the More Boring Our Dreams Become

• Spirit

• We Only Appear to Be So Complex to Each Other Because We Are So Stupid

• Emotions Get Projected Into Your Body Map

• Eastern vs Western Culture

• Happiness Is a Cookie That Your Brain Bakes for Itself

• What is Life? / The Meaning of Life

• God

Some quotes are slightly edited to make them clearer — but the timestamps are there, so you can go to the source.

---

Gödel, Turing and Wittgenstein

“The last big thing that we discovered was constructivist mathematics — so, to understand that the parts of mathematics that work are computation. That was a very significant discovery in the first half of the 20th century. And it hasn't fully permeated philosophy and even physics yet. Physicists checked out the code libraries from mathematics before constructivism became universal.”

“Gödel himself, I think, didn't get it yet. Hilbert could get it. Hilbert noticed that with the current semantics, we cannot build a computer in mathematics that runs mathematics without crashing. And Gödel could prove this. What Gödel could show is that using classical mathematical semantics, you run into contradictions. And because Gödel strongly believed in these semantics — more than in what he could observe — he was shocked. Basically it shook his world to the core, because in some sense he felt that the world had to be implemented in classical mathematics. For Turing it wasn't quite so bad. I think Turing could see that the solution is to understand that mathematics was computation all along. For instance, π in classical mathematics is value. It's also a function, but it's the same thing. And in computation, a function is only a value when you can compute it. And if you cannot compute the last digit of π, you only have a function. You can plug this function into your local sun, let it run until the sun burns out, and this is it. This is the last digit of π you will know. But it also means that there can be no process in the physical universe or on any physically realized computer that depends on having known the last digit of π. Which means there are parts of physics that are defined in such a way that cannot strictly be true, because assuming that [they] could be true leads into contradictions.” (14:00)

There is one central insight in Wittgenstein’s Tractatus:

“There is a way in which mathematics and philosophy ought to meet. Mathematics tries to understand the domain of all languages by starting with those that are so formalizable that you can prove all the properties of the statements that you make. But the price that you pay is that your language is very, very simple. So it's very hard to say something meaningful in mathematics. It looks complicated to people, but it's far less complicated than what our brain is casually doing all the time to make sense of reality. And philosophy is coming from the top — so it's mostly starting from natural languages, which vaguely define concepts — and the hope is that mathematics and philosophy can meet at some point. And Wittgenstein was trying to make them meet. He already understood that, for instance, you could express everything with the NAND calculus — that you could reduce the entire logic to NAND gates, as we do in our modern computers. So in some sense, he already understood Turing universality before Turing spelled it out.” (18:00)

General Intelligence and the Turing Test

“I suspect that general intelligence is the result of trying to solve general problems. So intelligence, I think, is the ability to model. It's not necessarily goal directed rationality or something. Many intelligent people are bad at this. It's the ability to be presented with a number of patterns and see a structure in those patterns, and be able to predict the next set of patterns — to make sense of things. And some problems are very general. Usually intelligence serves control, so you make these models for a particular purpose of interacting as an agent in the world and getting certain results. Intelligence itself is instrumental to something. But by itself, it's just the ability to make models. And some problems are so general that the system that makes them needs to understand what itself is and how it relates to the environment.” (25:30)

“The Turing test, in a way, is: You ask a system "what is intelligence?" If that system is able to explain what it is and how it works, then you should assign it the property of being intelligent in this general sense. So the test that Turing was administering — I don't think that he couldn't see it, but he didn't express it yet in the original 1950 paper — is that he was trying to find out whether he was generally intelligent, because in order to take this test you need to be able to understand what the system is saying. We don't yet know if we can build an AI. We don't yet know if we are generally intelligent. Basically, you win the Turing test by building and AI. It's a test on us. The Turing test basically a test of the conjecture whether people are intelligent enough to understand themselves.” (26:30)

We Exist in a Story That the Brain Tells Itself

“What you notice is that the only thing that is real to you and me is this experiential world in which things matter, in which things have taste, in which things have color, phenomenal content and so on. And this is distinct from the physical world in which things have values only in an abstract sense and you only look at cold patterns moving around. So how does anything feel like something? This connection between the two things is very puzzling to a lot of people, and of course to many philosophers. So idealism starts out with the notion that mind is primary. Materialism thinks that matter is primary. For the idealist, the material patterns that we see playing out are part of the dream that a mind is dreaming, and we exist in a mind on a higher plane of existence, if you want. And for the materialist, there's only this material thing that generates some models and we are the result of these models. I don't think that we should understand materialism and idealism as a dichotomy, but as two different aspects of the same thing. So the weird thing is we don't exist in the physical world. We do exist inside of a story that the brain tells itself.” (37:00)

Aside: I wrote a post about this — ‘The Locked Room’.

One aspect that we're missing [in the modern field of AI] is unified learning. When we learn, we at some point discover that everything that we sense is part of the same object — which means we learn it all into one model, and we call this model the universe. So experience of the world that were embedded in is not a direct via to physical reality. Physical reality is a weird quantum graph that we can never experience or get access to. But it has this property that it can create certain patterns at our systemic interface to the world. We make sense of these patterns, and the relationships between the patterns that we discover is what we call the physical universe. So at some point in our development as a nervous system, we discover that everything that we relate to in the world can be mapped to a region in the same three-dimensional space, by and large. We now know that in physics this is not quite true — the world is not actually three-dimensional. But the world that we are entangled with at the level at which we are entangled with it is largely a flat three-dimensional space. And so this is the model it our brain is intuitively making. And this is, I think, what gave rise to this intuition of res extensa — of this material world, this material domain. It's one of the mental domains, but it's just the class of all models that relate to this environment, this three-dimensional physics engine in which we are embedded. (30:00)

Aside: I feel that this, and many other quotes here, touch on Hoffman’s Interface Theory of Perception.

Describing Reality From Inside a Mandelbrot Fractal

“Imagine you look at a Mandelbrot fractal. This famous thing that Benoît Mandelbrot discovered. You see an overall shape in there, right? But you know, if you truly understand it, that it's two lines of code. It's basically a series that is being tested for complex numbers in the complex number plane for every point. And for those [points] where the series is [converging] you paint it black and where it's [diverging] you don't. And you get the intermediate colors by checking how fast it diverges. This gives you the shape of this fractal. But imagine you live inside off this fractal and you don't have access to where you are in the fractal, or you haven’t discovered the generator function even. So what you see is this spiral, and the spiral moves a little bit to the right. Is this an accurate model of reality? Yes, it is. It is an adequate description. You know that there is actually no spiral in the Mandelbrot fractal. It only appears to like this to an observer who is interpreting things in a two-dimensional space and then defines certain regularities in there at a certain scale that he currently observes. If you zoom in, the spiral might disappear and turn out to be something different — at the different resolution, right? So at this level, you have the spiral, and then you discover that the spiral moves to the right, and at some point it disappears. So you have a singularity. At this point, your model is no longer valid. You cannot predict what happens beyond the singularity. But you can observe again and you will see that it's another spiral. And at [a later] point, it also disappears. So maybe now we have a second order law. And if you make 30 layers of these laws, then you have a description of the world that is similar to the one that we come up with when we describe the reality around us. It's reasonably predictive. It does not cut to the core of it. It does not explain how it's being generated; how it actually works. But it's relatively good to explain the universe that we are entangled with.”

How to figure out the generator function:

“You try to figure out how [the Mandelbrot fractal] works, and you somehow have a Turing machine with enough memory to think. As a result you, come to this idea that it must be some kind of automaton, and maybe you just enumerate all the possible automata until you get to the one that produces your reality. So you can identify necessary and sufficient conditions. For instance, we discovered that mathematics itself is the domain of all languages. And then we see that most of the domains of mathematics that we have discovered are in some sense describing the same fractals. That's what category theory is obsessed about — that you can map these different domains to each other. So there's not that many fractals. And some of these have interesting structure and symmetry breaks, so you can discover what region of this global fractal you might be embedded in from first principles. But the only way you can get there is from first principles. So basically, your understanding of the universe has to start with automata, and then number theory, and then spaces and so on.” (40:20)

Aside: This is basically what Stephen Wolfram is trying to do with The Wolfram Physics Project — figuring out the generator function of physics.

Only a Simulation Can Be Conscious

“Some people think that a simulation can't be conscious, and only a physical system can, but they got it completely backwards. A physical system cannot be conscious, only a simulation could be conscious.”

“The mind is basically the principles that produce the simulation. It's the software that is implemented by your brain, and the mind is creating both the universe that we're in and the self; the idea of a person that is on the other side of attention and is embedded in this world. The self is basically a result of the purpose that the mind has. It's a tool for modeling, right? We're not actually monkeys. We are side effects of the regulation needs of monkeys. The monkey has to regulate the relationship of an organism to an outside world that is large part also consisting of other organisms. And as a result, it basically has regulation targets that it tries to get to. These regulation targets start with priors — they're basically like unconditional reflexes that we are more or less born with, and then we can reverse-engineer them to make them more consistent. And then we get more detailed models about how the world works and how to interact with it. And so these priors that you commit to are largely target values that our needs should approach — set points — and this deviation to the set point creates some urge — some tension. We find ourselves living inside of feedback loops. Consciousness emerges over dimensions of disagreements with the universe. Things where you care, where things are not the way they should be, that you need to regulate. And so in some sense, the self is the result of all the identifications is that you're having. And that identification is a regulation target that you're committing to. It's a dimension that you care about, that you think is important. And this is also what locks you in — if you let go of these commitments, of these identification, you get free. There's nothing that you have to do anymore. And if you let go of all of them, you're completely free and you can enter nirvana because you're done.” (44:00)

Aside: This related to Daniel Kahneman’s idea of System 1 and System 2 from his book Thinking, Fast and Slow. Basically, System 1 is your autopilot (walking, breathing, 2+2, etc) and System 2 is deliberate attention (ordering food, solving hard math problems, writing essays, etc).

What is Consciousness?

“Consciousness is largely a model of the contents of your attention. It's a mechanism that has evolved for a certain type of learning.”

“What we do is attention based learning. We pinpoint the probable region in the [neural] network where we can make an improvement and then we store this binding state, together with the expected outcome, in a protocol. This ability to make indexed memories for the purpose of learning to revisit these commitments later — this requires a memory of the contents of our attention. Another aspect is when I construct my reality and make mistakes. So I see things that turn out to be reflections or shadows and so on, which means I have to be able to point out which features of my perception gave rise to the present construction of reality. So the system needs to pay attention to the features that are currently in its focus. And it also needs to pay attention to whether it pays attention itself, in part because the attentional system gets trained with that same mechanism (so it's reflexive), but also in part because your attention lapses if you don't pay attention to the attention itself. So — is the thing that I'm clearly seeing just a dream that my brain has spun off into some kind of daydream, or am I still paying attention to my percepts? So you have to periodically go back and see whether you're still paying attention. And if you have this loop and you make it tight enough — between the system becoming aware of the contents of its attention and the fact that it's paying attention itself and makes attention the object off its attention — I think this is the loop over which we wake up.” (56:30)

Everything Else Is Downstream From Controlling Attention

“I think that meditation is essentially just a bunch of techniques that let you control attention. And when you can control the attention, you can get access to your own source code (hopefully not before you understand what you're doing). And then you can change the way it works, temporarily or permanently. But the entire thing is that you learn to control attention. Everything else is downstream from controlling attention.” (2:03:00)

Ego Death

“You can turn off your self, you know? You can basically meditate yourself into a state where you are still conscious, where things are still happening, where you know everything that you knew before, but you no longer identify with changing anything. And this means that your self in a way dissolves. There is no longer this person. You know that this person construct exists in other states, and it runs on this brain of Lex Friedman, but it's not a real thing. It's a construct; it's an idea. And you can change that idea. And if you let go of this idea — if you don't think that you are special — you realize it's just one of many people. And it's not your favorite person, even. It's just one of many. And it's the one that you are doomed to control for the most part. [The self] is basically informing the actions of this organism as a control model, and this is all there is.” (2:01:30)

Access to Internal States and Emotional Coupling

“There are internal states that are less accessible for me — that you have, and my model [of you] might not be completely adequate. There are also things that I might perceive about you that you don't perceive. But in some sense, both you and me are some puppets — two puppets — that enact a play in my mind. And I identify with one of them because I can control one of the puppets directly. And with the other one, I can create things in between. So, for instance, we can go on an interaction that even leads to a coupling — to a feedback loop — so we can think things together in a certain way or feel things together. But this coupling is itself not a physical phenomenon — is entirely a software phenomenon. It's the result of two different implementations interacting with each other.” (48:00)

Aside: I’m not quite sure what Joscha means by coupling here — is it just basic human interaction, and things like emotional contagion? Or is it something more subtle — like the studies done on eye contact, or Marina Abramović’s The Artist Is Present project? If you have any thoughts about this please reach out!

We Perceive Things That Are Improbable Yet Possible

“Our perceptual models of the world are not probabilistic but possibilistic — which means that you should be able to perceive things that are improbable but possible. Perceptual state is valid not if it's probable, but if it's possible; if it's coherent. So if you see a tiger coming after you should be able to see this, even if it's unlikely. And probability is necessary for convergence of the model. So given the state of possibilities, that is very, very large, and a set of perceptual features, how should you change the states of the model to get it to converge with your perception?” (1:07:00)

The Older We Get, the More Boring Our Dreams Become

“The older we get, the more boring our dreams become because we incorporate more and more constraints that we learned about how the world works. So many of the things that we imagine to be possible as children turn out to be constrained by physical and social dynamics and as a result, fewer and fewer things remain possible. It's not because our imagination scales back, but the constraints under which it operates become tighter and tighter.” (2:06:00)

Spirit

“For me, a very interesting discovery in the last year was the word spirit, because I realized what spirit actually means: it's an operating system for an autonomous robot. And when the word was invented, people needed this word. But they didn't have robots that built themselves yet, so the only autonomous robots that were known for people, animals, plans, ecosystems, cities and so on. And they all had spirits. And it makes sense to say that the plant is an operating system, right? If you pinch the planned in one area, then it's going to have repercussions throughout the plant. Everything in the plant is in some sense connected into some global esthetics, like in other organisms. An organism is not a collection of cells — it is a function that tells cells how to behave. And this function is not implemented as some kind of supernatural thing — like some morphogenetic field. It is an emergent result of the interactions of each cell with each other cell.” (1:14:30)

We Only Appear to Be So Complex to Each Other Because We Are So Stupid

“How much common sense knowledge do we actually have? Imagine that you are really hard working for all your life and you form two new concepts every half hour or so. You end up with something like a million concepts because you don't get that old. A million concepts — that's not a lot.”

“If you think just about the numbers, you don't live that long. If you think about how many cycles do your neurons have in your life, it's quite limited. You don't get that old.”

“In some sense, I think of concepts as the address space for our behavior programs. And the behavior programs allow us to recognize objects and interact with them (including mental objects). A large part of that is the physical world that we interact with — which is this res extensa thing — which is basically navigation of information in space. And basically it's similar to a game engine. It's a physics engine that you can use to describe and predict how things look in a particular way, feel when you touch them in a particular way, the law of proprioception, auditory perception and so on. So basically the geometry of all these things. And this is probably 80% of what our brain is doing — it's dealing with this real time simulation. And by itself a game engine is fascinating, but it's not that hard to understand what it's doing. Our game engines are already, in some sense, approximating the fidelity of what we can perceive. So if we put on an Oculus Quest, we get something that is still relatively crude with respect to what we can perceive, but it's also in the same ballpark already. It's just a couple orders of magnitude away from saturating our perception. In some sense, it's reasonable to say that the computer you can buy and put into your home is able to give a perceptual reality that has the detail that is already in the same ballpark as what your brain can process. And everything else is ideas about the world. And I suspect that they are relatively sparse.”

“We only appear to be so complex to each other because we are so stupid — it's a little bit depressing!” (2:10:30)

Emotions Get Projected Into Your Body Map

“The experience of emotion is a feeling, and in this sense, what you feel is an appraisal that your perceptual system has made of the situation at hand. And it makes this based on your motivation, and on your estimates — not your, but of the subconscious geometric parts of your mind that assess the situation in the world — with something like a neural network. And this neural network is making itself known to the symbolic parts of your mind — to your conscious attention — via mapping them as features into a space. So what you will feel about your emotion is a projection, usually, into your body map. So you might feel anxiety in your solar plexus. And you might feel it as a contraction. Which is all geometry, right? Your body map is the space that is always instantiated and always available, so it's a very obvious cheat — if the non-symbolic parts of your brain try to talk to the symbolic parts of your brain — to map the feelings into the body map. And then you perceive them as pleasant or unpleasant, depending on whether the appraisal has a negative or positive valence, and then you have different features of them that give you more knowledge about the nature of what you're feeling. For instance, when you feel connected to other people, you typically feel this new chest region — around the heart — and you feel this as an expensive feeling in which you're reaching out. It's very intuitive to encode it like this.” (2:39:00)

Eastern vs Western Culture

“Eastern European society have a taboo against [showing emotion] and especially Russia, the further you go to the east. In the US, it's the opposite. You are expected to be hyper animated in your face, and you are also expected to show positive affect. If you show a positive effect without a good reason in Russia, people will think that you are a stupid, unsophisticated person. [In the US, positive affect] is the default. It's expected. Everything is amazing.”

“There's a deeper aspect with this. It's also how we construct meaning. In the US, you usually focus on the positive aspects and you just suppress the negative aspects. And in our Eastern European traditions, we emphasize the fact that if you hold something above the water line, you also need to put something below the water line because existence by itself is at best neutral. There are moments of beauty, but these moments of beauty are inextricably linked to the reality of suffering. And to not acknowledge the reality of suffering means that you are really stupid and unaware of the fact that basically every conscious being spends most of the time suffering.” (2:43:00)

Aside: I wrote my very first post about this!

Happiness Is a Cookie That Your Brain Bakes for Itself

“Happiness itself is not important. Happiness is like a cookie. When you are a child, you think cookies are very important and you want to have all the cookies in the world. You look forward to being an adult because then you have as many cookies as you want, right? But as an adult you realize the cookie is a tool. It's a tool to make you eat vegetables. And once you eat your vegetables anyway, you stop eating cookies for the most part, because otherwise you will get diabetes and will not be around for your kids.”

“Happiness is a cookie that your brain bakes for itself. It's not made by the environment. The environment cannot make you happy. It's your appraisal of the environment that makes you happy. And if you can change your appraisal of the environment — which you can learn to do — then you can create arbitrary states of happiness. Some meditators fall into this trap. They discover the room, this basement room in their brain where the cookies are made, and they indulge themselves, and after a few months it gets really old and the big crisis of meaning comes. Because they thought before that their unhappiness was the result of not being happy enough. So they fixed this, right? They can release the neurotransmitters at will, if they train. And then the crisis of meaning pops up a deeper lair. And the question is: why do I live? How can you make a sustainable civilization that is meaningful to me? How can I insert myself into this? And this was the problem that you couldn't solve in the first place.” (2:48:00)

What is Life? / The Meaning of Life

“I think that if you look at the meaning of life, you look at what the cell is. Life is the cell. It's this self organizing thing that can participate in evolution. In order to make it work, it needs

1. a self replicator,

2. a negentropy extractor,

3. and a Turing machine.

If any of these parts are missing, you don't have a cell, and [the system] is not living. Life is basically emergent complexity over that principle. Once you have this intelligent supermolecule — the cell — there's very little that you cannot make it do. It's probably the optimal computronium, especially in terms of resilience. It's very hard to sterilize a planet once it's infected with life.”

“The cell is able to do something called existential design. In technical design, we go from the outside in. We work in highly controlled environments in which everything is deterministic, like our computers, or our labs, or our engineering workshops. Then we used this determinism to implement a particular kind of function that we dream up and that [function] seamlessly interfaces with all the other deterministic functions that we already have in our world. Biological systems design from the inside out. A seed will become a seedling by taking some of the relatively unorganized matter around it and turn it into its own structure — and thereby subdue the environment. Cells can cooperate if they can rely on other cells having a similar organization that is already compatible. But unless that's there, the cell needs to divide to create that structure by itself. So it's a self-organizing principle that works on a somewhat chaotic environment. And the purpose of life, in a sense, is to produce complexity. And that complexity allows you to harvest negentropy gradients that you couldn't harvest without the complexity. In this sense, intelligence and life are very strongly connected because the purpose of intelligence is to allow control under conditions of complexity. So basically, you shift the boundary between the ordered systems into the realm of chaos. You build bridgeheads into chaos with complexity. And this is what we're doing. This is not necessarily a deeper meaning. Outside of our priors there is no meaning. Meaning only exists if the mind projects it. [The meaning that we have priors for] is probably civilization. I think that what feels most meaningful to me is to try to build and maintain a sustainable civilization.” (2:50:00)

God

“I don't think that God has anything to do with creation. I think it's a mistranslation of the Talmud into the Catholic mythology. I think that Genesis is actually the childhood memories of a god. It's basically a mind that is remembering how it came into being. We typically interpret Genesis as the creation of a physical universe by a supernatural being. And I think when you read it… there is light and darkness that is being created. And then you discover sky and ground — create them. Then you construct the plants and the animals, and you give everything their names and so on. That's basic cognitive development. It's a sequence of steps that every mind has to go through when it makes sense of the world. And when you have children, you can see how initially they distinguish light and darkness, and then they make out directions in it and they discover sky and ground, and they discover the plants and the animals, and they give everything their name. It's a creative process that happens in every mind, because it's not given, right? Your mind has to invent these structures to make sense of the patterns on your retina.” (2:55:00)

Aside: During cognitive development, we create higher and higher level ‘chunks’ in order to better make sense of the world. Douglas Hofstadter has an excellent paper on this — ‘Analogy as the Core of Cognition’.

“What we perceive as the higher being that we’re part of is civilization. It's the thing to which we have a similar relationship as the cell has to our body. And we have this prior because we have evolved to organize in these structures. So basically, the Christian God, in its natural form — without the mythology, if you undress it — is basically the platonic form of a civilization. It's this ideal that you try to approximate when you interact with others, not based on your incentives but on what you think is right.” (2:57:00)

---

If you liked this post, feel free to share it with your friends! If you have any feedback or if I got anything wrong, please let me know!

savsidorov.com | @savsidorov