PictureWindow - Origin Story

I saw a couple of tweets with people outside of the HMD interacting with the virtual world and had been playing around with mirrors using the Vive Stereo Rendering Toolkit when I had the idea of using the tracked controller to determine what was being displayed on the mirror so that it would effectively be what you have here.

Needless to say that didn't work as anticipated, but like most experimental projects it led down a very interesting rabbit hole.

Now to be clear, I know this is not a new idea. In fact, I would say if anything this project is a bit of an homage to one of the first times I really saw the potential of positional tracking.

Just a day after getting started I saw another tweet that was getting a lot of attention which showed a similar concept (demonstrated entirely within TiltBrush) so that had me motivated to see this thing through.

While doing research on it I found many many many different instances of people exploring this in various ways.

I'm not going to lie, I haven't read all of them and I am sure there would have been some help in there, but I wasn't here to read, I was here to hack at code.

Everything actually went together pretty quickly and I had something working within the first week, but then I ran into some performance issues that ended up causing some direction changes and delays.

The SteamVR_TrackedObject problem:

First, I had experimented with having this working with both the SteamVR Player prefab and the SteamVR [CameraRig] prefab so people could use the one of their choice. This is when I noticed a problem with SteamVR_TrackedObject. It seemed that because it was only updating the position and rotation on new pose events, that it wasn't updating 1:1.

You can reproduce this yourself by toggling off 'Use Custom Tracked Object' on the PictureWindowControl component.

If you turn that off with the debugging box enabled you will see that the boundaries of the screen do not stay fixed to the boundaries of the box. Using the modified version (which just takes the position and rotation update code from the Hand component of the Player prefab) this no longer jostles around.

I don't know if this is because of something I am doing wrong with the [CameraRig] prefab or the way I'm using TrackedObject or if it's a general bug in the tracking performance, but if anyone has any idea I would be more than happy to hear it. I really don't like relying on my own TrackedObject here.

The Player prefab problem:

In addition to the TrackedObject problem, I also had an issue with the Player prefab not being performant for me at all. I should note that I was building this all on my (under spec) portable rig which is using a 4690k and a GTX 770.

If I create a new project with a new scene:

Because of that I opted to release this as [CameraRig] only. If anyone knows why I'm getting those frame rate drops I would be more than happy to hear that as well.

Final thoughts:

I had an idea and wanted to get some more practice trying solve a problem, so I did, but it was all worthless if no one ever uses the thing, so please, feel free to use this for whatever you want. Nothing would make me more happy than to see some interesting uses of this.

Some ideas I've had that might get you started:

In the future I'd like to add some new features like being able to move the window around in the virtual space, but there's no timeline on that right now.

Anyway, if you made it this far thanks for checking it out!