2D Viewer Apps Are Not Virtual Reality

January 4, 2016 by Terry

If you have gotten your hands on a Google Cardboard or Gear VR or similar virtual reality (VR) headset, you likely followed up by downloading a bunch of VR demos to your smartphone. Common types of demos are 360-degree photos, 360-degree videos, and scenes rendered with 3D graphics. Unfortunately, much of the 360 photo and video content is a feeble pile of me-too apps. You can find some reasonable theories about the nature of the problem, such as the poor field-of-view or poor framerates and motion sickness. However, I have not seen anyone asserting the biggest problem of all: much of the photo and video content is 2D.

Nearly any definition of VR that you can find requires the simulated environment be made to appear real. So any hardware or software that doesn’t bother to simulate 3D stereo vision, the way the vast majority of sighted humans see the world, simply isn’t VR. The main offenders in this category–and the focus of this article–are some image and video viewers. Apps that draw everything with computer graphics are generally better because they almost all have a quality 3D effect, though some of them still find ways to strain your eyes. You could also reject 360 photo and video viewing based on their lack of interactivity, but let’s stick to the 3D deficiencies for this article.

2D vs. 3D

Figure 1 shows what a smartphone screen should look like when content is presented properly to generate a believable 3D effect. Figure 2 shows what many VR apps actually look like.

Figure 1

Figure 1. A 3D image pair, photos taken 6 centimeters apart just like your eyes. Note that you can only see the yellow side of the cube from your left eye.

Figure 2

Figure 2. A 2D image pair, two crops of the same photo.

Do you see the difference? Both figures contain images for the left and right eye that look slightly different. Figure 1 contains images taken from two different locations, simulating the parallax that your eyes capture in real life (you can only see the yellow side of the cube from one eye). Figure 2, on the other hand, contains two views of the same image. The views are framed with a small horizontal shift relative to one another in an attempt to prevent double-vision and make everything look very far away. If the horizontal shifting is done poorly, the user may be required to go cross-eyed or wall-eyed (the opposite of cross-eyed) for his eyes to converge on the images. Many users will not figure this out and see a confusing (or painful) double-image. Either way, the 2D image pair does not simulate the way your eyes actually see the world.

The Scattered State of Mobile VR

So why do so many app creators present 2D content in VR headsets that are designed for displaying 3D content? One possibility is that they want to make 3D content and have no idea what they are doing. Another possibility is that they are simply too lazy to make the content correctly. Yet another is that they want to ride the latest VR craze with me-too apps and think their viewers are too stupid to spot the difference between 2D and 3D. The main reason to show 360 imagery in a VR headset is so your phone can track your head rotations and display different parts of the imagery accordingly, but when the imagery is 2D your VR headset becomes little more than a magnifying glass. You may be better off holding the phone in front of your face without a headset so you can enjoy the full resolution of your screen.

Some 2D apps, such as NYT VR, at least attempt to show their imagery at a proper field-of-view for each eye. A Google Cardboard-style headset usually has a QR code on it that can be scanned by the Cardboard app. NYT VR uses this information to calibrate the imagery on your smartphone for your particular headset. Google even provides a QR code generator so you can calibrate your headset if the manufacturer didn’t already provide a QR code. NYT VR received criticism for giving people double-vision. This was likely due to phones not being calibrated for the correct headset.

Worse are apps that ignore or misuse the information provided in the QR code. RYOT – VR and Ascape Virtual Travel & Tours lazily present two identical images without even compensating for lens warp. With these you’ll need to go cross-eyed to view images that appear warped. Even worse is Vrideo, which does correct for lens warping but makes you go painfully wall-eyed by applying too large a horizontal shift to each eye’s view. (I used a 5-inch screen for all these tests. Your experience may be better or worse with a different screen size.)

If you are interested in seeing the VR industry take off, you should be bothered by all of this. Most people’s first experience with VR will be with a mobile headset. If the first app they see shows wall-eyed 2D imagery, they will likely dismiss VR as underwhelming or even annoying.

On the bright side, there are some apps that present 360 video in 3D. Jaunt VR and Vrse – Virtual Reality provide video with the proper parallax to simulate stereo vision, the way we actually see. Cardboard Camera will let you take 360 photos that also have a believable 3D effect.

The Displaced is a video shown in 3D on Vrse and 2D on NYT VR. I don’t know why NYT VR takes quality content and reduces it to 2D. Perhaps the creators of NYT VR wanted smaller downloads, or perhaps they think their viewers won’t notice. In any case, if you’re going to call it VR you can at least attempt to make it feel real by using 3D imagery.

Wrap-up

The best 3D you’ll get in a VR headset will be done with real-time computer graphics because every image is rendered for your exact eyepoint, allowing you to rotate your head into any orientation. 3D photos and video are a good alternative if they accurately simulate stereo vision. The 3D effect is baked into the imagery instead of computed in real-time as with computer graphics, so it will generally only work correctly if you keep your head level and look toward the horizon in any direction. Again, try Jaunt VR or Vrse – Virtual Reality for good examples. Lastly, 2D “Virtual Reality” is a cop-out forged by charlatans. If someone tries to show you 2D VR, do him a favor and show him some quality VR.

The state of VR articles and reviews is as random as the quality of smartphone VR. Top-10 lists are overflowing with all the apps mentioned in this article, including the bad ones. If you’re a journalist, please take the time to understand the differences between quality content and all the me-too nonsense. Maybe the situation will improve after the release of several high-end headsets (HTC Vive, Oculus Rift, PlayStation VR) in 2016, but most people’s introduction to VR will probably still be with hit-or-miss smartphone VR.

(This article has been Android-centric. If you have related experiences with iOS-based VR, please describe them in the comments.)

3 Responses to “2D Viewer Apps Are Not Virtual Reality”

  1. Tom

    non-stereoscopic content is far easier to capture. For non-stereoscopic photos / videos, you can just grab a Ricoh Theta S, press a button and you’re done. For stereoscopic content you need an array of cameras (think Google’s Jump or Samsung Project Beyond) and some intelligent stitching so the photos / videos look correct for each eye with matching, sane white balance and exposure settings.

    Also, as you mention, stereoscopic content takes double the bandwidth. I personally feel this is more of an issue for videos than for photos.

    For a good 360 photo / video experience, you need more than just stereoscopic around a single point. Better would be taking position into consideration. Your head/neck are always moving at least slightly and should be taken into account. To do that, you need to do something along the lines of a 360 lightfield capture like Lytro has recently done. With that, you can move your head/neck around a little bit, but not so much you can walk around. It’s still a sitting experience.

    There is big concern in the VR community that the experience that Cardboard provides is really going to poison the well and turn off the masses. I think Google has helped lower expectations by making the thing literally out of cardboard. Hopefully that is setting the expectation that what people are about to experience is not a full VR experience, but just a limited taste of what could be. I might be being too optimistic though.

    • You are correct, 3D content is harder to produce. For those who can’t or won’t produce it, I think the honest solution is to stop calling their apps VR.

      Head movement and and stereo vision are both good for giving a virtual environment a sense of depth and scale. If I can only have one, I’ll take stereo vision. I’m looking forward to 360 light field cameras too. Jaunt is also making one. I still haven’t seen a light field demonstration that is anywhere near commercially viable, so hopefully these guys will amaze us.

      As for cardboard experiences, I’d like to get my hands on some Wearality Sky lenses. Didn’t have a smartphone when they were doing their Kickstarter or I might have gotten a pair. Sucks for me.

  2. I have been in the middle of this debate since the last “VR merry-go-round” 25 years ago. If it’s NOT stereoscopic, it’s not “virtual” anything. With low resolution and a limited field of view, the whole thing is a waste of time and money. They had a much more exciting path the last time this failed. If they have not been successful selling 3D TV’s (LG makes the best with “passive” Cinema 3D), and they haven’t been successful with stereoscopic gaiming (though this is incredible) adding a “cell phone”to a 19th century stereoscope (and not calling it a “Viewmaster”) is the dumbest thing since the recent Democratic Presidential cadidates. So go ahead, waste your money. I produce fun stereoscopic 3D videos compatble with the “cell phone VR” but which look much better on the big screen. Speaking of which, since WIN 8 and above have a built-in stereo mode, why are all the screensavers still 2D?

Leave a Reply