Oculus Rift and Super Resolution

This post is written in a way that makes it hard to figure out the subject ...

Are you saying that the pixel density is too low for Oculus Rift due to how close the screen is to the eyes ?

He's commenting on an effect that he's noticed with the Rift, and wants to know other people's experiences/thoughts on it.

Basically, think of looking through a screen or fence slats- if you're standing still, they block your vision and you can only see part of whats past them. If you're moving, though, they basically disappear, because your mind pieces together what you're seeing to form a complete image of whats behind the fence/screen.

He's noticed a similar effect with the Rift. The game world seems to be at a much higher resolution than it actually is, because the brain is piecing together all the pixelated images (that are slightly different because the head is moving, however subtly) to form a near-perfect, "complete" image of the game world that is at a "super-resolution". However, aspects like UI that are stuck to one point in the screen can't be improved by this effect, so UI text was almost impossible to read.

I don't have a Rift, so I can't comment on whether or not this happens to me. However, if this is generally the case, it does offer a way to improve VR UI. I know people are saying to have stuff like ammo count and health bar as part of the world, but if you did want it as floating UI for whatever reason, it'd be best to still have it fixed (more or less) in world, so that the brain can piece a better image of it together.

I never noticed that effect on my rift. I will look for it next time. The only time the low resolution was a problem for me was in half life when I was trying to fire at an enemy that was far away. So perhaps I was experiencing that super resolution effect since the resolution doesn't seem to bother me.

Which rift did you use? I got to try one out the other day, and the resolution was painfully low. The screen door effect was terrible and the latency was way too high.

However, that was the original rift.

Everything I've heard/read says that the DK2 version is a vast improvement on both counts.

It seems that you like the screen door effect. ;) The thing it reminds me of is when you walk past a fence. On average you can see only 20% through many fences (e.g. the gaps), but as long as you keep moving your brain stitches those glimpses together and you feel like you can see through the fence.

It seems that you like the screen door effect. ;) The thing it reminds me of is when you walk past a fence. On average you can see only 20% through many fences (e.g. the gaps), but as long as you keep moving your brain stitches those glimpses together and you feel like you can see through the fence.

It's simple motion blur and depth of filed blur what you are describing with that fence thing.

It seems that you like the screen door effect. ;) The thing it reminds me of is when you walk past a fence. On average you can see only 20% through many fences (e.g. the gaps), but as long as you keep moving your brain stitches those glimpses together and you feel like you can see through the fence.

It's simple motion blur and depth of filed blur what you are describing with that fence thing.

No it isn't; you're actually getting more information about what's behind the fence if the background is moving relative to the fence. Each pixel in the Oculus (or any screen) is only a single data point, but naturally, if you sample the same pixels repeatedly over time, the pixels from one frame could give you data that falls "between" pixels in another frame. It does seem that the brain is remarkably good at figuring out where in space those pixels belong as well.

I'm actually very surprised that, to my knowledge, no one has used this principle for up-scaling video resolution. Obviously it'd only work in situations where automatic motion tracking be performed, but motion tracking works well enough that it can be used for 3D reconstruction, camera tracking, and stabilization, so my intuition says that it would work for this as well.

I've used the original Oculus development kit, and I've noticed the same thing, but I think with improved latency and frame rates the effect would be much stronger.

I'm actually very surprised that, to my knowledge, no one has used this principle for up-scaling video resolution

It's used in a lot of video-games. Every frame you shift the screen projection by a fraction of a pixel, which results in edges maybe shifting by a pixel, or not. When combined with MSAA and temporal reprojection, nVidia uses the TXAA buzzword to refer to it

It's also an old-school way that people used to use to render out "magazine quality" screenshots from game engines. You'd take thousands of sub-pixel shifted screenshots and then average them, which gives you a screenshot that you can say is "in engine", but looks pre-rendered

I'm actually very surprised that, to my knowledge, no one has used this principle for up-scaling video resolution

It's used in a lot of video-games. Every frame you shift the screen projection by a fraction of a pixel, which results in edges maybe shifting by a pixel, or not. When combined with MSAA and temporal reprojection, nVidia uses the TXAA buzzword to refer to it

It's also an old-school way that people used to use to render out "magazine quality" screenshots from game engines. You'd take thousands of sub-pixel shifted screenshots and then average them, which gives you a screenshot that you can say is "in engine", but looks pre-rendered

Yeah, that's pretty close to what I'm referring to, but I was imagining using that principle to actually increase the number of pixels (versus just increasing the number of "samples" averaged to form each pixel) by reconstructing the amount of shift after the fact based on only the video data.