Skip to main content
GameDev.net gamedev.net
Using GameDev.net for your class this semester?
Learn more →
🔒 Locked

Mirroring a VR view

Started by Josh Klint Aug 5 at 4:12 PM 7 replies 450+ views
Original Post
Josh Klint
Josh Klint

I was wondering how best to display the VR view to a window, for the purpose of recording videos? Showing two eye views is very distracting. Rendering a third camera placed in front of the user's head will slow the framerate. Is there some known method of combining the two eye views into something that looks good enough for non-VR viewing?

Aressera
Aressera

Why not just display one of the eyes? The difference between the left/right eyes is not very large except for very close objects. It should be fine for videos to use either one arbitrarily. I can't imagine how you could combine two views without artifacts, maybe some kind of TAA-like reprojection of one view onto the other, but I wouldnt even attempt it.

frob
frob

In the major engines there are settings to show the scene on the main screen. We've used it on several VR-enabled titles I've worked on.

For MOST games it is fine. Yes it drops performance slightly so it needs an option to enable/disable the view, but the performance penalty has always been pretty minor for my experience except on low end machines.

In both Unity and Unreal the rendering/camera views are easily adjusted. You can reuse one of the VR eyes or trigger a third render pass with an independent camera view. If you choose to pull from one of the VR eyes you can choose the distorted or undistorted view that it renders, or see it letterboxed or cropped to screen dimensions, or just render it to a texture for use in whatever you want.

SBD
SBD

I think what Frob was getting at is that this is how some big engines handle what you're asking about, maybe one of those approaches might work for you.

As far as "combining two eye views", I don't think that makes much sense, I'm not even sure what you'd expect there visually? If your main issue with just rendering one of the eye views is that, depending on the VR hardware, the viewport/texture is somewhat oblong vertically, then you could either just do another render pass for that eye with a standard viewport/render target size, or just do a "center eye" position and do the same thing (although, as mentioned above, the variance between left and right eye is visually quite tiny in the vast majority of cases).

In my custom engine, I simply present the left eye view on screen and call it a day, as I don't want to incur additional overhead rendering the scene again.


JoeJ
JoeJ

Well, i remember Crytek worked on the opposite problem - generating 2 VR images from a single rendered image. (maybe in Crysis 2)
I could not find any links quickly, but likely some reprojection approach. The same would work the other way around, with even less artifacts and easier to implement i guess. With two images there should be almost no missing information, but ofc. a scene like many fine branched bushes partially occluding each other along distance would show some artifacts.

A basic algorithm would be: For each pixel of the target image, take many samples around that spot in the two source images, and weight them by angle to the target ray, times some exponential distance term to prefer samples closest to the eye. This should preserve initial image quality and give smooth temporal behavior. But maybe a TAA pass for the generated center image would be worth it too.

Another related technique is the 'back projection' (iirc) step used in older soft shadow map approaches. Here they reprojected a kernel of SM texels to get the approximated occlusion from a shifted light source position. That's the same problem i think, so worth to look up. (Sadly i don't remember any more search terms to find papers.)

I would be optimistic, expecting good results after putting some time in.
But you could also spare the time and instead shifting marketing strategy, by saying: 'Watching two eyed VR videos on YT is cool, because if you pinch your eyes you get true stereo depth impression!'
This is actually true and works. I always do this for such videos, making sure they are just 10 cm wide on screen, so i can pinch my eyes wide enough. It does not work for fullscreen. (But it would work for a mobile game!)
It's really cool to watch VR videos this way, but sadly i see such videos only super rarely. For me, there should be more of them, not less.

frob
frob

Josh Klint wrote:

I didn't ask for advice on how to use someone else's engine, I am asking if there is a known technique to combine two eye views that looks good enough to use?

Simply: you don't combine the views.

What I listed IS the known technique, that's what other games do, it is what the engines do, it is the current best practice among the games that implement the displays. The typical solutions are to either take the rendered image from one of the eyes, or to generate a third, spectator view.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.