2D Viewer Apps Are Not Virtual Reality

January 4, 2016 by Terry

If you have gotten your hands on a Google Cardboard or Gear VR or similar virtual reality (VR) headset, you likely followed up by downloading a bunch of VR demos to your smartphone. Common types of demos are 360-degree photos, 360-degree videos, and scenes rendered with 3D graphics. Unfortunately, much of the 360 photo and video content is a feeble pile of me-too apps. You can find some reasonable theories about the nature of the problem, such as the poor field-of-view or poor framerates and motion sickness. However, I have not seen anyone asserting the biggest problem of all: much of the photo and video content is 2D.

Nearly any definition of VR that you can find requires the simulated environment be made to appear real. So any hardware or software that doesn’t bother to simulate 3D stereo vision, the way the vast majority of sighted humans see the world, simply isn’t VR. The main offenders in this category–and the focus of this article–are some image and video viewers. Apps that draw everything with computer graphics are generally better because they almost all have a quality 3D effect, though some of them still find ways to strain your eyes. You could also reject 360 photo and video viewing based on their lack of interactivity, but let’s stick to the 3D deficiencies for this article.

2D vs. 3D

Figure 1 shows what a smartphone screen should look like when content is presented properly to generate a believable 3D effect. Figure 2 shows what many VR apps actually look like.

Figure 1

Figure 1. A 3D image pair, photos taken 6 centimeters apart just like your eyes. Note that you can only see the yellow side of the cube from your left eye.

Figure 2

Figure 2. A 2D image pair, two crops of the same photo.

Do you see the difference? Both figures contain images for the left and right eye that look slightly different. Figure 1 contains images taken from two different locations, simulating the parallax that your eyes capture in real life (you can only see the yellow side of the cube from one eye). Figure 2, on the other hand, contains two views of the same image. The views are framed with a small horizontal shift relative to one another in an attempt to prevent double-vision and make everything look very far away. If the horizontal shifting is done poorly, the user may be required to go cross-eyed or wall-eyed (the opposite of cross-eyed) for his eyes to converge on the images. Many users will not figure this out and see a confusing (or painful) double-image. Either way, the 2D image pair does not simulate the way your eyes actually see the world.

The Scattered State of Mobile VR

So why do so many app creators present 2D content in VR headsets that are designed for displaying 3D content? One possibility is that they want to make 3D content and have no idea what they are doing. Another possibility is that they are simply too lazy to make the content correctly. Yet another is that they want to ride the latest VR craze with me-too apps and think their viewers are too stupid to spot the difference between 2D and 3D. The main reason to show 360 imagery in a VR headset is so your phone can track your head rotations and display different parts of the imagery accordingly, but when the imagery is 2D your VR headset becomes little more than a magnifying glass. You may be better off holding the phone in front of your face without a headset so you can enjoy the full resolution of your screen.

Some 2D apps, such as NYT VR, at least attempt to show their imagery at a proper field-of-view for each eye. A Google Cardboard-style headset usually has a QR code on it that can be scanned by the Cardboard app. NYT VR uses this information to calibrate the imagery on your smartphone for your particular headset. Google even provides a QR code generator so you can calibrate your headset if the manufacturer didn’t already provide a QR code. NYT VR received criticism for giving people double-vision. This was likely due to phones not being calibrated for the correct headset.

Worse are apps that ignore or misuse the information provided in the QR code. RYOT – VR and Ascape Virtual Travel & Tours lazily present two identical images without even compensating for lens warp. With these you’ll need to go cross-eyed to view images that appear warped. Even worse is Vrideo, which does correct for lens warping but makes you go painfully wall-eyed by applying too large a horizontal shift to each eye’s view. (I used a 5-inch screen for all these tests. Your experience may be better or worse with a different screen size.)

If you are interested in seeing the VR industry take off, you should be bothered by all of this. Most people’s first experience with VR will be with a mobile headset. If the first app they see shows wall-eyed 2D imagery, they will likely dismiss VR as underwhelming or even annoying.

On the bright side, there are some apps that present 360 video in 3D. Jaunt VR and Vrse – Virtual Reality provide video with the proper parallax to simulate stereo vision, the way we actually see. Cardboard Camera will let you take 360 photos that also have a believable 3D effect.

The Displaced is a video shown in 3D on Vrse and 2D on NYT VR. I don’t know why NYT VR takes quality content and reduces it to 2D. Perhaps the creators of NYT VR wanted smaller downloads, or perhaps they think their viewers won’t notice. In any case, if you’re going to call it VR you can at least attempt to make it feel real by using 3D imagery.

Wrap-up

The best 3D you’ll get in a VR headset will be done with real-time computer graphics because every image is rendered for your exact eyepoint, allowing you to rotate your head into any orientation. 3D photos and video are a good alternative if they accurately simulate stereo vision. The 3D effect is baked into the imagery instead of computed in real-time as with computer graphics, so it will generally only work correctly if you keep your head level and look toward the horizon in any direction. Again, try Jaunt VR or Vrse – Virtual Reality for good examples. Lastly, 2D “Virtual Reality” is a cop-out forged by charlatans. If someone tries to show you 2D VR, do him a favor and show him some quality VR.

The state of VR articles and reviews is as random as the quality of smartphone VR. Top-10 lists are overflowing with all the apps mentioned in this article, including the bad ones. If you’re a journalist, please take the time to understand the differences between quality content and all the me-too nonsense. Maybe the situation will improve after the release of several high-end headsets (HTC Vive, Oculus Rift, PlayStation VR) in 2016, but most people’s introduction to VR will probably still be with hit-or-miss smartphone VR.

(This article has been Android-centric. If you have related experiences with iOS-based VR, please describe them in the comments.)

3 Responses to “2D Viewer Apps Are Not Virtual Reality”

Leave a Reply to Terry

   Click here to cancel reply.