Lightfields
We stradle two seemingly separate universes: the inner world that we percieve with our emotions, thoughts, and mind's eye, and an external world that we percieve with our vision, touch, sound, and other external senses. You could describe consicousness as being the interplay between the internal and external universes that we mediate between. I'm not going to try and describe where I think these universes come from, or whether our current external universe is a simulation (it is), but at the very least it does seem evident that I create an internal projective representation of the external universe into my own. Virtual Reality offers an opportunity to bring the external perceived world much closer to the internal one we create, and blend them in a manner similar to a dream that we can freely explore. It allows us to more closely examine and explore what consciousness is by offering us the ability to project our inner world back out into the simulated one. Furthermore, virtual reality offers us the opportunity to communicate our inner worlds to each other in a way that was not previously possible. Let me elaborate...Today when we communicate we encode our experience into a set of symbols that are then decoded by someone else such that they create an internal represenation of your message in their head. For example, I think of a banana and then I use the word (symbol) for banana, and since we have a common set of symbols you can take the encoded symbol and understand the message. We do this with language, but we also do this with images, and even video. There's the expression "a picture is worth a thousand words," but what's the conversion ratio from a picture to a video? What about from a video to actually being there?
Even within a video there's a loss of information, you're only sampling a real-life scene from one point. Light is only being captured from one point, sounds as well... You're encoding a 3D scene from a single point. Furthermore, you're taking the 3D scene and then decoding it onto a 2D screen. There's infomation loss at the encoding step and the decoding step. Watching a video only gives you an *idea* of what it would be like to actually be there. Of course, you don't actually feel like you're there, though.
What excites me about vitual reality is that for the first time there's a medium that is fully able to recreate what it feels like to "actually be there." The medium has 6 degrees of freedom. This just means you can rotate and move your head in 6 degrees of freedom (pitch, yaw, roll, x, y, z), just like you can in real life. Real life is 6dof. This means there's no inherent loss of information at the decode step, we're rendering the message in a medium that has the same dimensionality as real life. If we want to trick someone into believing they're reliving someone's experience, we need 6dof video representations. If we want to create our own simulation, we need 6dof video.
The medium for experiencing 6dof video exists, but there are very few people with the capabilities to capture 6dof video or images. In other words, encoding the message without any loss of information. 6dof capture is still in its infancy, but I think once we get to there it will have profound effects on our lives. (Note: Creating purely virtual 6dof environments is another possibility. Photorealistic rendering techniques are getting better and better, but we'll have to pass through an uncanny valley for a while until we get something that truly feels real, especially with people. I think methods like light field capture will probably beat out purely virtual 6dof video, at least with my current understanding... although GANs can do some crazy things, I'd imageine that a 6dof capture rig would incorporate both virtual and real components).
A related area of interest is Augmented Reality, AR glasses are getting better and better. An AR display is simply superior to a regular 2D display, you can do so much more with it. It is my strong belief that AR glasses will replace the 2D monitor display we use for computers. Soon we won't be using macbooks, but instead glasses. An area of interest to me is how we will interface with something like AR glasses. We'll want a more mobile version of a keyboard. Human-computer interfaces are a special area of interest for me... Especially Brain Computer Interfaces.
Back