Wow! Thanks, all, for the responses to my blog post, “Fundamental Limits to Virtual Reality”, on the subject of RR (real reality) vs. VR (virtual reality).
I think that’s the most comments I’ve ever gotten on a single post, and they’re still trickling in. It’s nice to know that there I do have readers out there in RR, even if you don’t always agree with me. And it helped that BoingBoing mentioned my post.
I’m not exactly “against the singularity.” It’s rather that I’m against one particular postsingular notion, that is, the idea of grinding up Earth to make a VR Vearth. If I have any beef with the singularity it’s simply with how rapidly the public discussion of it has narrowed down to a very few ideas that are already cliches. And that’s what I’m trying to shake free from in my Postsingular series. Being a Hegelian, I like to leapfrog discussions, moving to the next idea down the line.
I’ll post some more pictures from the Pinedale, Wyoming, environs today, along with some back-atchas. I wrote this up somewhat hastily, so I apologize to those I left out, and to those whom I’ve misquoted or misinterpreted.
Jeff Saltzman : “Still, virtual realities are very real, they just have different qualities from RR– isn’t a 3D multiplayer game a virtual reality? … I think the only content with staying power will be captured reality– people you know, telepresence at sporting events or concerts or plays, the ability to have a simulated window that looks out on a real_ vista.”
Rudy: In terms of actual applications, I think “mixed reality” software is going to be very big. Things like overlays on actual scenes and live video blogs. I wrote about these “vlogs” in Mathematicians in Love.
Jeff Taylor: “In the virtual worlds that we now inhabit in games, the 3d environment is rendered only when it needs to be … with a bit of imagination isn’t it possible to make a long and dubious leap to theorizing that the universe is tracking what is being observed? And once that theory has been thrown out there, couldn’t a system that tracks observation also be rendering reality in a finite bubble around all ‘observed’ particles?”
Rudy: The solipsistic version is that the narrator is the only real character and everything is simply being brought into view as he or she (but usually its a he) looks around. Fiction dramatizes our own psychodramas. Solipsism is an immature worldview, not the view of someone who’s been around the block, loved people, fought enemies. But you’re suggesting something different, that maybe only certain points in the universe are conscious and those are the zones being decohered. My current thinking is that consciousness is universal, so that everything is being filled in.
abend : “If you can simulate a person on a computer sufficiently well that they don’t notice the difference, then you can do things like send copies of them to other planets, keep backups in case of damage, optimize their thinking to eliminate cognitive biases, eliminate the need for sleep, etc.”
ChristopherMoody:“The only difference between me in a biological substrate and me in a utility fog is that the former is limited in its capabilities and extraordinarily fragile whereas the latter is infinitely mutable and near immortal. This is a life-affirming notion not a life-hating one. I love all of the zany, unpredictable complexity of life more than anything which is why I don’t ever want it to end. Cobb Anderson seemed to understand that soul=software as well as the desirability of continuing to run that software indefinitely. …Did I completely miss the mark in your writings?”
McBob: “The life-hating bit though…personally, if my mind could stay intact, I’d like to see the stars. What’s so magical about your body that you’d lose your joy in life by changing it? The techie dream here is simple – I am me, no matter what body or form I have.”
Rudy: These responses make the point that, given that our bodies do seem doomed to decay, might it not be wise to seek out digital immortality? I’m all for immortality—at least on my good days—but there’s no particular reason to think we’ll find it in digital form. My sense is that the digital age is going to be over in another couple of hundred years. We’ll be back to analog matter biocomputations. Rather than going all around Robin Hood’s barn to put yourself into a buzzing beige box, you’ll just grow yourself a new meat body.
I’m still somewhat comfortable in saying that soul = software, but I now realize that the software goes down much deeper than I’d realized before. You need to carry over the machine code and the ROM code and the laws of physics… And, again, I think that modeling all this digitally is a dead end.
Steve H: “Erasing this world to make another one would be nuts. Making another one just like it would be cool, though. Turn Jupiter into computronium, or Mercury, and make a really hardcore Second Life.”
Rick York: “Isn’t the point of creating VR is to make it different? I love RR, but I want to bend the rules.”
Jeff: Where do you get the energy to turn conventional matter like silicon or iron into raw, unprogrammed computronium?”
Rudy: Certainly if you want to crunch up a junker world like Neptune (but what do the Neptune crystal creatures think of that?) and make a Vearth, that is less objectionable. And making a world quite different from Earth seems interesting, that’s what we do when we write novels, too.
I take objection to Stross’s notion of smashing things up to make computronium. Remember, guys, ‘computronium’ is just a word that Charlie made up. Matter doesn’t need to be optimized for computation. It’s just that we don’t yet have the I/O, the input output.
[Correction form a late-breaking comment by Andrew Doull: “Computronium” predates Stross.]
paradoctor: “My own fantasy isn’t Vearth but Earth 2.0. Keep nature, just add to it. For instance, how about giving every cell on Earth a molecule-sized computer to use? The difference between Vearth and Earth 2.0 is that on Vearth the nanocomputers own the cells, and on Earth 2.0 the cells own the nanocomputers.”
Rudy: This is exactly the idea that underlies my ending to Postsingular and the whole underlayment of my novel Hylozoic, which I’m just finishing.
someguy: “Maybe the best explanation for plank time / length and all the other quantum weirdness is that we are in a simulated universe and we are viewing the computational limits of that simulation.”
Rudy: My friend John Walker has been talking about this, too. It’s a great SF concept. This said, I don’t see our universe as a simulation inside something else. It’s a simulation that’s simulating itself. It’s the hardware and the software.
Jason Blum: “VR will very soon fly straight past rR (with a small r.) It doesn’t need to match nature – it need only match your perception of nature, which ain’t much.”
I dunno: “All you have to do is fool the entities in the Vearth … That said, the whole concept is silly to me. I don’t get why anyone would want to.”
mahalis: “I think it’s reasonable to say that …it would be possible to set up a virtual space that was real enough.”
Rich: “There are huuuuuuuuuge amounts of detail we can’t perceive that doesn’t need to be rendered, which is computational power which can be used for other things. Like laserdragons.”
Vidar Hokstad: “We can’t predict the arrangement of individual atoms in a large object. Why would a simulation even try? If someone do point an electron microscope at an object in the simulated world, the simulator can pick any random arrangement and we wouldn’t know better.”
Sam Walker: “Quantum computing, could duplicate pretty much the whole universe in something like 10^300 fully entangled qubits.”
Rudy: My whole point is to wake people up to the fact that the physical, daily world is inconceivably rich. It’s something that we forget if we look at screens all the time. And I think it’s safe to say that the rulers and merchants in our society are interested in blurring this distinction. If you’re outside looking at a cloud, you’re not spending money and you’re not thinking about what the politicians are up to.
And if you really think that there is some shortcut desktop way of simulating reality in full, I’m sorry, but you’re just factually wrong. Not that 10^300 fully entangled qubits is a desktop system! That’s, like, the universe. One remark re. quantum stuff— it may very well be that quantum mechanics is just a high-level approximation, and matter is in fact infinitely, or even transfinitely, divisible.
The notion of leaving the details up to randomness is an interesting move. But maybe they aren’t random. Wolfram sometimes claims the whole kaboodle comes out of some, like, ten-bit rule that’s run for a really large number of cycles. Here’s the number of cycles that’s the thing that won’t fit on your desk.
When people talk about a substitute being “just as good,” I think of the Who song.
You think we look pretty good together
You think my shoes are made of leatherBut I’m a substitute for another guy
I look pretty tall but my heels are high
The simple things you see are all complicated
I look pretty young, but I’m just back-dated, yeahSubstitute your lies for fact
I can see right through your plastic mac
I look all white, but my dad was black
My fine looking suit is really made out of sackI was born with a plastic spoon in my mouth
…
Substitute me for him
Substitute my coke for gin
Substitute you for my mum
At least I’ll get my washing done
OmgWtfBBqNewFag: “What happens after the singularity level event is speculation. … My favorite is that the godlike AI computer mind could reorganize all matter in the universe and that the dimension containing our universe would recognize our universe not as an ignorant blob but something like a cute puppy.”
Rudy: The universe as a whole having a mind is a nice concept. These days I’m leaning towards the notion that the universe is larger than transfinite in size, so that there truly is no final “One” thing. There’s always a higher level.
MB: “One wouldn’t have to recreate the whole Earth, but only its surface down to a certain depth (say 100km) and then use some sufficiently realistic model for the interior. The resulting savings in material would be enormous and would allow for even more than slight inefficiencies in matter usage.”
Jeff: “There’s a lot of planetary volume that we don’t need to simulate at the atomic level. Do we really need to simulate the geophysical dynamo at the earth’s core that generates the magnetic field that creates polar auroras?”
Rudy: That’s a move worth mentioning, that you can just make a Hollow Earth replica. But this only what a computer scientist would call a linear speed-up, and it’s a logarithmic speed-up that you’d need to make computing Vearth feasible.
Anton Bijgaarden: “There is no reality out there, just interpretations, perceptions, a collective hallucination tinted by emotions and interests that we agree upon only to fulfill the practical business of living. Therefore, every reality is a virtual reality, never reaching the actual thing. … “
Auer Westinson: “ We already live in a simulation. Every one of us. We already have a simulated VR presentation of the world — running on the cybernetically feedback-looped brain wetware supercomputer. I am interested in getting to the bottom of what it is we already are experiencing—not building another simulation.”
Evan : “When you consider how rich and detailed and complex the world around us is, it’s staggering to imagine how much more so the real one must be…”
Alek Traunic: “Does the human brain already have all the software needed to create a 100% believable VR?”
Rudy: These conversations are good in that they do get us to thinking about Ultimate Reality or, maybe, the Ultimate Morass of Reality Layers. I like the notion that, since we dream convincingly, our brains can simulate reality. This said, I don’t usually see very realistic things in my dreams.
outerjoin: “Don’t forget that Vearth would actually require dev, test, and staging environments to enable all phases of the testing cycle … Post-release bugs are what we refer to as the “paranormal”, and people like Jesus are no more than test user accounts with admin rights.”
Kelson:“Do you suppose that Nature might have some sort of auto-immune response should any one part of herself start tinkering too much?”
Rudy: Some great SF ideas here. I love it. I may shamelessly use one of these…
Mantissa128: “It is amazing that matter is performing computation while just sitting there, but we don’t have access to it, we have no way of making it do our bidding except by blunt force that applies to trillions of atoms. We need to get down there, have our conscious decisions exert a causative force at an atomic level.”
Sam Walker: “In the nanomachine world you describe, the advantage to replacing the blade of grass with pseudograss is that pseudograss is programmable.”
Rudy: This is a key point. As I mentioned above, I don’t think we need to go to computronium. I think matter, just as it is, does as much rich computing as we want. The catch is the I/O problem: how do we get at it? How do we hack matter?
In my novel in progress Hylozoic, the characters do it via a kind of telekinesis. They’re able to telepathically merge into an atom, say, and reprogram it. I don’t have a super-good explanation of how this works, my current one is based on some handwaving talk about unfurling the eighth dimension; I say a bit about this my Psipunk essay.
What is reality?
March 6th, 2008 at 8:21 am
You guys are on fire! This is great stuff.
March 6th, 2008 at 9:36 am
This and the previous post are fabulous. Rudy, I agree with you wholeheartedly. I have an unhealthy obsession with the idea of the Singularity, as expressed fictionally by Stross and Vinge and yourself, but I think the ultrageeks you mention are missing the point. The point–which Stross hinted at in Accelerando–is that uploading ourselves would result in an essentially empty existence. It’s the ultimate in escapism–not just escaping reality, but destroying it and replacing it with something entirely unreal.
Now, me, I’m not interested in replacing reality, but in adding to it. We’ve already done that in much of the physical world, probably as much as we can and have the planet remain habitable. But now, with things like GPS, mobile phones, and Google Earth, we’re adding information layers as well–what you call “mixed reality” and I call “augmented reality”–and this is where the fun is really going to be, IMHO.
(And, of course, the most exciting stuff in science fiction is dealing with this. Stross’s Halting State, Gibson’s Spook Country, etc.–though, as usual, Vinge got there first, with Rainbows End.)
I do think long and hard about the nature of consciousness in the universe we seem to be discovering, though. (Getting mystical here; pure materialists may skip this paragraph.) If consciousness is universal, then what exactly is its nature to physical reality? My own notion, which I spend my more whimsical moments trying to hash out, is that consciousness is something fundamental that condenses or coalesces at points where the local information density is high enough–like in the brain. But this is pure crazy speculation on my part.
Anyway, I’m really enjoying this. Can’t wait to read Hylozoic!
March 7th, 2008 at 7:21 am
You want VR right now? give me today’s tools and I can give you an easy 2-5 minutes of reality that you can’t tell the difference.
I get a kick that people say we are soo special that there is no way we can recreate the universe.
Well duh. what we don’t want to admit is that we can recreate what we see/feel/taste whatever with a machine and fool us. Impossible to fool our magnificent brains!
All the richness what we see..oh please. we see practically nothing. stop and just stare and the the number one and “look” without moving your eyes.
1
Do you see all the wonderfull detail of your sleeves? the richness of all detail of the world beyond the monitor?
Jeesh could you even read half of the sentences what I typed below and above the number one?
Do you know how easy it is to make such blurry shapes with today’s computers? we can render in our brains only about two letters wide before it all falls to crap. A simple eyetracking and displaying only what you are looking at on a screen and barely rendering anything else would blow the reality off games off the roof. Right now we can do this…
I knocked you out and you awoke with a VR helmet on your head. I limited your movement by having your head strapped and your body, strapped to a chair. I had a bright warm light shining on you. I mixed up some pungent oceany smelling water and had a fans blowing on you periodically. your shoes were on those squishy bead travel pillows. I gave you some rigmarole that your pupils were super dilated from the knock out drug and this is the only way you could see at the moment. I had beach sounds being piped in and people talking secretively in the background.And then I told you we were here at a beach in a secret location and of course, I am from the government.
I guarentee you I could fool you with a supercomputer for quite a while. Simulate the beautiful, oh so complex “pirates of the Caribbean graphics” waves breaking off in the distance. Make you experience the trillions of grain of sands that are around you.
Ok yeah, i cant simulate that we are in a busy city/or I can’t have you walking around. What I am saying is that I can partially recreate your universe now and fool you by restricting your senses just a little. So to say that computers won’t be able to do this at all…gimme a break you divine humans.
I could strap you on a chair and on a movable platform and beam a remote tracking 2-camera setup on a sightseeing bus of the pyramids. Blow hot air and some particles of sand. Have on cue someone put in your mouth a cake as the tour guide walks up to you and places the same thing he is holding up the “number 5 is alive” looking setup on the bus. it soon would feel real. very real.
I am spending to much time on this. time to get back to exploring the websites/video/universe thru my laptop. Then clean my crap small place. yeah, reality is so great for some of us!
The singularity is coming. Yippee!!
March 7th, 2008 at 7:55 am
W00t!
What would the Big Bang look like from a transfinite perspective? Harg. I suppose from the transfinite perspective, it would depend at what level you’re on at the time…
March 7th, 2008 at 10:49 am
Wally M, the fact that your examples all involve strapping someone down and/or drugging them don’t inspire much confidence in your notions about great VR…
I’d suggest that you get a geranium or philodendron plant to help make your room a more enjoyable reality!
March 8th, 2008 at 6:15 am
“..I can partially recreate your universe now and fool you by restricting your senses just a little. So to say that computers won’t be able to do this at all..”
I think you are missing the point, I am not saying it would be pleasant, I’m just saying “reality=VR” can be done for a short moment , right now.
But yes grinding up the earth for VR is…just silly.
To capture all the systems the Earth would have to be ground INSTANTLY including the impossible knowledge of place and direction/speed quantum thingy . By taking any time eveything would be disjointed and not copied correctly. ground/sliced my spaggetti while I was eating it you would have a plate that is half there half not. Grind the radiation belt and the photons from the sun? I guess you could import the whole model and then start when it was all completed but it would be a time-mass stretched out mess of nonsense..as far as I can see it is a dumb concept with sooo many holes and people should probably just drop it.
March 8th, 2008 at 8:25 am
Building just a bit from Brian’s remarks above (“I’m not interested in replacing reality, but in adding to it”), the notion of replacing something real in contrast to creating a representation of something has been a long discussion.
N. G. Chernyshevsky (Н. Г. Черныше́вский) wrote about art and Art and Reality back in the 1800’s, and argued that a painting of an apple isn’t an apple. Magritte’s artwork emphasized this theme in his famous “This is not a pipe” painting and others. Replacing Art with VR can get very similar discussions.
While the currently impossible process of recreating an exact duplicate of a living thing (Star Trek transporters–still impossible today, right?) seems exciting, I also think that there are more possibilities in exploring a VR that isn’t an exact replica. After all, we already know the real world. Where’s the fun in that?
I was pondering duplicated environments and Lifeboxes this week when I got my new computer. My initial thought was to exactly duplicate my old environment on the new machine, but then I had a change of heart. I brought over the files I currently use, left the old stuff on the old computer, and now I’ve got two similar environment that are changing in their own directions. It would be nice to have them both on one machine (as Virtual Machines running on a VMM) so that I could watch them both over time, but I’ll need a bigger machine still for that to happen. How interesting it would be to have two Lifeboxes with slightly different threads that I could maintain…or three, or more.
Anyway, thanks for the VR posts, Rudy–it made me exercise my brain the way it should be exercised.
Scott
March 10th, 2008 at 4:35 am
Rudy,
You said “The notion of leaving the details up to randomness is an interesting move. But maybe they aren’t random”
But the entire point is that for a simulation _it doesn’t matter_. You won’t know the difference. It only matters from a philosophical goal of a “perfect simulation”. But what is the value in a perfect simulation vs. one that is “good enough” that you can’t tell it apart from the real thing?
If I present you with 1000 pictures of snowflakes, you will be completely unable to tell me how many of them I’ve reused multiple times unless you were allowed to zoom in to a great level of detail and study them for a long time. _Unless you zoom_ (use a microscope, or whatever) there’s no reason to take the computational cost of making a vast number of different versions.
I’ve seen human game players be blatantly fooled by completely trivial tricks in that respect into thinking they are dealing with a far richer environment than they are. Human brains have evolved into highly sophisticated pattern matching machines where it is safer to find a pattern where there is none (prey moving about rather than the wind rustling the trees) than miss a pattern where there is one (the prey gets away because you don’t see it, and you go hungry). As a result we are VERY easy to fool.
We frequently “see” things that are not there because pattern matching kicks in. Our memory is woefully incomplete (Try a simple experiment: draw your shoes without looking at one, then put them in front of you and draw them again. Most people, regardless of skill level, will draw an “idealized” version of their shoes the first time, lacking a lot of details not because they can’t draw them, but because their memory of it only contains the important features) Our reading comprehension is terrible – just try to have a group of five people read a paragraph and then summarize it 30 seconds later and see how different they are (and how none of them are likely to match it completely, unless they’re told in advance they’ll have to memorize or summarize it)
In short, we’re lossy machines to start with, designed to fill in details based on similarity and experience rather than reality. Present us with a lossy world, and, as long as it takes the “right” shortcuts, we won’t be able to tell the difference.
We’ll assume that the leaves on the ground have been blown around by the wind if we even notice they’ve shifted somewhat. If we raked them together in a nice pile we’ll notice if the pile is gone completely, but not if it’s subtly changed because the location of individual leaves was picked randomly when you returned.
We can fool a human in this way _without_ access to thought processes, but a simulation of the world including our brains would be able to tell exactly what we remember of what we’ve seen and experienced, and can selectively cull details our brains deemed unimportant rather than make guesses. Fooling us when you can check the simulation of our brain and adjust the randomized model until it is consistent with expectations would be trivial compared to doing the simulation in the first place.
In other words, a simulation can make changes we _can’t_ catch out unless we actively try to. Even then it could make it extremely hard by limiting changes possible changes that had a high enough probability of happening naturally.
We might even be able to make it impossible to catch. Worst case, if we found it acceptable, the simulation could alter our neural response to match the changes, but even to me that’s one step too far.
Yes, I hold a fairly mechanistic view of nature. I do believe we’re all a bunch of machines. Of course, we don’t _know_. It’s possible that consciousness can’t be captured by a simulation. But until we have something that indicates otherwise, that’s my basic assumption, and that boils it down to making the inputs match at the right stage.
I get the feeling from what you write that your starting point is very different, and that that is the basis for bringing up a “complete” simulation.
To me, a complete simulation is wasteful, and waste is terribly inelegant when I could instead use those cycles to “add features” – I’d rather have a virtual reality that augments what we have today than one that aims to perfectly duplicate the real thing. If it’s just a duplicate, then what’s the point?
For a virtual reality to make sense, it would have to offer vastly more than “real” reality.
(While we’re on this subject, btw., whether you agree with the starting assumptions or not, everyone should read this: http://www.simulation-argument.com/)
March 11th, 2008 at 4:18 pm
Wally M: your “simulation” would give me motion sickness. And how would you blank out the smell of the room and the background noise of your VR gear? Or make the indoor lighting feel like tropical sun? Raise the temperature without coating the VR helmet in sweat? Nice try 🙂
April 8th, 2008 at 2:47 pm
wau geile seite.schöne bilder…respect
May 14th, 2008 at 11:46 pm
Rudy: My whole point is to wake people up to the fact that the physical, daily world is inconceivably rich. It’s something that we forget if we look at screens all the time.
And maybe we should add beautiful as well 🙂
I think that summarises your point. Get out more people. And great pictures Rudy, they really make the point.
May 19th, 2008 at 2:11 pm
While I do appreciate the intent of your post, Rudy (and I particularly enjoyed the clouds this weekend!), your argument is defined by an intrinsic assumption that our universe is somehow “perfect.” When stating the pointlessness of simulating an atom with another atom, you are assuming that there is simply no way to BETTER optimize the components of our reality than in the form that it currently resides in. This feels rather shortsighted to me (if not a bit pseudo-religious as well). It seems entirely possible that there are “realities” which utilize matter in much better ways than our own.
Here’s a rather simplistic hypothetical: what if we discovered a way to simulate atomic structure using subatomic particles? Material substance, when reconfigured, could then express an exponentially larger amount of pseudo-atomic information than its atomic predecessor. It would then seem an imperative (amorally speaking) to reconstruct the universe to this more optimal state.
And couldn’t we call this reconfigured substance, say… computronium?
May 20th, 2008 at 6:38 am
Noahsam: I guess you have a point. Maybe there are some subsdimensional levels that could simulate our reality more compactly. That’s a good SF idea, actually. A handheld device with the Earth in it, and no cheap-jack Flinstones-animation thing, an actual eidetic image of the planet, running on a Subdee chip. This calls for a thought experiment…
December 2nd, 2008 at 9:50 pm
I am currently reading “The Lifebox, the Seashell, and the Soul” and I’d like to propose a method of measuring the computation power of a device:
“The rate at which a device can simulate an equivalent volume of reality”
A cube of sea-water would have a value of exactly one – it can simulate itself in real-time, but no more.
A modern intel CPU can barely simulate a handful of atoms. If the CPU is made up of 10^20 atoms and it can simulate 100 atoms (10^2), then it would have a value of 1 / 10^18. An alternative way of expressing this would be say that it has a value of “18”, and the scale is inverse exponential.
So then a hypothetical future quantum computer that can see into the future would simulate reality 100x faster than real-time – it would have a value of 1 / 0.01 or 1 / 10^-2 or just “-2”.
June 22nd, 2011 at 2:31 pm
What I’ve not seen addressed here (maybe I missed it) is the possibility of simulating the world much more slowly. I agree that turning the world to computronium doesn’t cut it, but if we run it 10x, 100x, 1000000x more slowly, it seems we could go as deep as we want.
There’s no hurry when you’re a simulation … you don’t experience the time slicing.
June 22nd, 2011 at 7:07 pm
Ron, it’s nice to have a comment on this thread again. I myself don’t remember everything that’s in the posts and comments anymore. You suggest using a very long-running engine to generate a deeply detailed simulation.
To some extent this is feasible, but we the exponential time issue is more of a problem than one immediately thinks. Suppose you want to simulate some region for one second with a level of detail down to 1/N the size of the region. So the amount of time it takes to do this is going to be some function of N. The problem is that the time isn’t going to vary like, say, 100 * N, or like N^3. It’s going to vary exponentially, that is, like 10^N.
And exponential computation times are kind of hideous. Suppose N is 100. Then you’re looking at the mighty googol, that is, 10^100.
And the computational work might also be exponential in the length of the time that you want to simulate as well.
So it seems kind of unfeasible, even for SF. This said, I can vaguely imagine a somewhat comic or satiric story involving the problem. A galaxy-spanning civilization works flat out for a trillion years just to simulate some guy eating a hard-boiled egg. “The Egg.”