Mixing Reality with Virtual Reality

Virtual reality is a pretty magical experience when it comes to making art. However, if you have friends in the room watching you, the magic is lost on them. They can only see the experience by looking at a distorted preview of the player’s perspective on a computer monitor.

A friend was recently painting a 3D submarine in Tilt Brush. “Look at this periscope!” she said. I told her to look closer at it, and I leaned into my computer monitor to get a glimpse of what she made.

We can do better than this. Why do I have to get up off the couch to see what my friend is creating? Why can’t I just lean back and see the art floating in the middle of the room?

Until recently, such magic would have been impossible. That is, until Microsoft released development kits of their new mixed-reality HoloLens glasses. I’m fortunate to have access a couple units, and I really wanted to use mixed reality to share in a VR experience.

So I spent a week making a proof of concept to feel that out. It runs on the HTC Vive, a VR system that includes two positionally-tracked controllers. Here’s what that looks like:

Read this article