Frames Per Second Question

Quote:

Original post by xtr33me Please forgive me if this is the wrong area to post this. I wasn't sure if I should post under game programming, but I figured it wasn't directly related so I am posting here. Question is, Why is it that 30 fps in a movie looks great, however 30 fps in a game starts looking like junk? The response was something I am not sure is true but it made sense. He stated that it is most likely to the natural motion blur acheived when filming video. Now I have seen this when playing around with a lightsaber scene in premiere. Now if this is the case, I was curious, could you acheive a smoother looking game by somehow creating one motion blur frame in place of a few rendered clear frames. This would cut down on the number of frames needing to be rendered, essentially giving the look of smoother video. I am new to programming so I have not truly started playing with game programming fully yet, but I am just curious on all your thoughts regarding this.

For what it's worth, it's important noting that in a game, you are in control. And that'sa Huge factor. 30 fps is much more bearable if you are watching your friend play.

Also, the 30 FPS in movies includes effects, 100% realistic motion-blur, and other realities that trick your eyes into seeing more than just a jumpy 30 frames. Your eyes compensate for the 'missing' frames.

the most convincing reason I've heard for this is something similar to what you hint at with motion blur.

The idea is that while computers generate discrete snapshots of a simulated world, cameras operate more similarly to the eye, because the exposure takes a small amount of time, capturing a range of motion for each frame. So even if a movie is at 24 fps it looks as good as the real thing because the camera sees how we see and displays that approximation. And I think 24 fps (maybe even lower) is about the threshold for people before they start to blur frames themselves.

I did read an article that explained it as follows.....

It is to do with the way monitors/TVs display their images compared to cinema. A film is a sequence of still frames. The projector displays one single stationary image, then the shutter closes as the film moves (so you don't see the frame scrolling off screen and the new one scrolling on), then the shutter opens again to show another complete, static image. You are in reality seeing a sequence of still image, blank screen, still image, blank screen, still image which the brain blends together.

On a monitor you don't have the blank screen part of the equation. Instead of a whole static image being projected to the screen the monitor screen is updated line by line so you are actually seeing the current image being over-written, which is more confusing for the brain.

I think that was the gist of it. I will see if I can hunt down a link.

Thanks everyone! A lot of good info.

Besides the human perceptive elements and the reasons listed above, don't forget that the 35mm film is being blown up to a rather gigantic size. Also, the very nature of photography lends itself to this phenomena. Most importantly: depth of field.

By the way, you might notice that when watching a film like King Kong (or a new Star Wars for that matter) the movie and CG "look" much "crappier" than they did when you saw them in theatres. There's a few technical reasons for that, but you can basically attribute size as a fundamental factor.

There are a number of reasons for this, but the most important is what game designers refer to as 'motion blur', and what photographers would refer to as shutter speed.

But first, to clear up some misconceptions.

Movies, that you see at the theater, are shot at 24 frames per second.

NTSC TV, however, is diaplyed at 60 fields per second. A field is half the vertical resolution of the entire frame.

The way photography basically works is this: There is a bunch of light out in the world, and you need to get it to correctly strike some photo-sensitive surface, like film.

So first you need a series of lenses to focus the incoming light properly onto the film. One you have that, you basically have a constant stream of light flowing into the camera, and onto the film.

However, too much light and you'll over expose the film, too light. To little light and it will be under-exposed, too dark

So a camera has another mechanism, called the shutter. Basically the shutter controls the ammount of light passing through the lens onto the film surface. Normally the shutter is completely closed until you want to expose the film, then it opens for a fraction of a second then closes again. Under this controlled enviroment, you can properly expose a photo-senstivie surface.

With the shutter you can control two variables that adjust the ammount of light that passes through it: how large the opening is (apature), and how long the shutter is open (f-stop). These two are inversely related, so as you increase the size of the opening, you have to decrease the time its open, and vice versa.

So what does this all mean?

Well, film is mechanically shot at 24 frames per second, usually shot with a shutter speed of 1/48.

So every 1/24th of a second the shutter will open to a specific size and then close 1/48th a second later.

During that short time, objects that are moving will have changed their position while the shutter was open. Causing the light to shift and change as it passes through the lens and onto the film surface, causing 'motion-blur'. This is why objects that are moving fast appear blury and transparent, because light from 'behind' the object is being exposed to the film surface as the object itself moves.

When displayed back, because object that are moving are 'blurred' in the direction that they are traveling, the mind can blend the images togeather a lot easier, causing the illusion of motion where there is in fact none.

And to answer your question, yes. By re-creating motion blur on a frame would allow you cut down the number of frames rendered. However, to calculate motion blur (in this case, determining how an object would look as it moved 1/48th of a second) would be more processor invensive rather than rendering 100 fps.

The simplest way to achieve motion blur is to over-render the image, basically for every frame you render 12 or 18 'extra frames' that at different 'times' during the 1/48th of a second when the shutter should be 'open' and then blend them together. At 30fps * 12 = 360 fps.

[Edited by - kryat on April 20, 2006 12:00:44 PM]

kryat, thanks for that explanation. Would that mean you can achieve this motion blur thing in pre-rendered scenes? And is that actually done or would that have benefits?

Motion blur is done in rendered images normally. When you watch a movie on TV that has CG in it they use motion blur. CG doesn't look natural if you don't.

Quote:

Original post by jfclavette For what it's worth, it's important noting that in a game, you are in control. And that'sa Huge factor. 30 fps is much more bearable if you are watching your friend play.

I've noticed this. Why is it better if you're in control?

It may also have to do that what you see on the screen, has already a precedent in your reality- i.e. you've seen most TV 'objects' and scenes before in your daily reality, your mind has only to fill in the missing parts. What I mean is that if you see someone fighting with a sword in reality, it's relatively easy to fill in the missing parts (memory!) , later, when you see a sword fighting on your TV.

In my 2d game, movement becomes 'realistic', only if the frame rate is 60 Hz. + Indeed, 30 fps looks like junk. I.m.o.-- in 3D this factor becomes way less important, as we humans are used to percieve things from this perspective.

If you study the way that the brain/eyes perceive images then it can get quite facinating how we can notice such subtle differences and instantly reject them as fake.

A couple of years ago I implemented a simple form of motion blur in Direct3D - the idea being to mimmick the shutter speed / exposure method used in traditional photography/film.

It worked really well. By doing a "diff" on the non-blurred images the difference was relatively small (1-2px or so). It's difficult to describe, but when I showed it to a few friends (note: not regular gamers/techies..) they found it much better - easer to perceive what was happening and so on...

I've been meaning to get around to updating it for the shader generation and adding some DOF. Maybe I'll get time over the summer.

Jack

Just another aside, cartoons and hand drawn animation in general was often drawn at half of the target frame rate. In the early days of animation, it wasn't uncommon to animate at 12fps, and its fairly common for saturday morning cartoons to be running at 15fps (displaying each frame twice).

However, animation traditionally employs the technique of "squash and strech" where the actual object is exaggerated and changes its shape as it moves. Its animations attempt to replicate and implement motion blur.

Early animators realized that if an object was moving quickly where the edges of the object did not overlap with its previous position, its motion appeared jerky or the object would appear to strobe.




to "fix" the inherrent problem of discrete frames with no "blur" they started using squash and stretch.




The faster an object moves, the "strechier" it becomes:



Squash and strech also applies to other dynamics of motion, but thats a definate aside.

I think another factor is probably that in games framerate doesn't tend to be constant. While your game might run at 30 fps overall, what this probably means is that at some points it is running at say 40 fps and at others 20 fps. I would imaging that this constant change makes the issue more noticable.

Never knew that about animation... Very interesting post kryat - ratings++ [smile]

Cheers,
Jack

If you haven't seen it yet, Showcan has an impossibly real look to it (almost 3D). Showscan is 60Hz, "hi-def" film.

I worked on early Virtual Reality projects (head mounted displays, motion trackers) and it was discovered that people will get "motion sick" if the display is not precisely synchronized to the head (inner ear "biological motion tracker"). The theory is that when the visual and inner ear brain systems don't match, the body recognizes this "disconnect" as an effect learned from evolution: the body must be poisoned. The response is to empty the stomach (limit/reduce more poison from entering the blood).

Interestingly, with 60Hz video games and fast FPS POV motion, some people will get sick just watching (and/or playing) a video game. Here the brain believes the body is moving, but the inner ear is not. A "work around" would be to ask users to turn their head's really fast as they play your game (if they are prone to motion sickness in this way). Synchronization might be tricky, though [wink].

While doing frame synchronized tests with my current physics-based game at 60-120 Hz, I could see a major improvement in perceptual smoothness when going from 100Hz physics (with or without frame locking at 60Hz) to 60/120Hz frame-locked video/physics. I thought, wow, that's the way to go, it looks much smoother. However, any time the physics load was heavy, and a frame was missed (could not run at exactly 60 or 120Hz), the hiccup was very noticeable (and perhaps annoying). During (very) late night testing, another interesting phenomenon was observed: the super-smooth frame-locked physics/video mode could induce motion sickness. Quite surprising!

Thus, I settled on 100Hz physics, which is out of phase with 60Hz video of most LCD's/TV's (a CRT might be running at 100Hz or more). The out-of-phase effective jitter appears to make the resulting motion look more like film (as opposed to videotape), better hides frame hiccups (physics-based game environments are unpredicatle), and appears to reduce motion sickness (would need a larger statistical sample set to substantiate).

Quote:

Original post by John Schultz If you haven't seen it yet, Showcan has an impossibly real look to it (almost 3D). Showscan is 60Hz, "hi-def" film. I worked on early Virtual Reality projects (head mounted displays, motion trackers) and it was discovered that people will get "motion sick" if the display is not precisely synchronized to the head (inner ear "biological motion tracker"). The theory is that when the visual and inner ear brain systems don't match, the body recognizes this "disconnect" as an effect learned from evolution: the body must be poisoned. The response is to empty the stomach (limit/reduce more poison from entering the blood). Interestingly, with 60Hz video games and fast FPS POV motion, some people will get sick just watching (and/or playing) a video game. Here the brain believes the body is moving, but the inner ear is not. A "work around" would be to ask users to turn their head's really fast as they play your game (if they are prone to motion sickness in this way). Synchronization might be tricky, though [wink]. While doing frame synchronized tests with my current physics-based game at 60-120 Hz, I could see a major improvement in perceptual smoothness when going from 100Hz physics (with or without frame locking at 60Hz) to 60/120Hz frame-locked video/physics. I thought, wow, that's the way to go, it looks much smoother. However, any time the physics load was heavy, and a frame was missed (could not run at exactly 60 or 120Hz), the hiccup was very noticeable (and perhaps annoying). During (very) late night testing, another interesting phenomenon was observed: the super-smooth frame-locked physics/video mode could induce motion sickness. Quite surprising! Thus, I settled on 100Hz physics, which is out of phase with 60Hz video of most LCD's/TV's (a CRT might be running at 100Hz or more). The out-of-phase effective jitter appears to make the resulting motion look more like film (as opposed to videotape), better hides frame hiccups (physics-based game environments are unpredicatle), and appears to reduce motion sickness (would need a larger statistical sample set to substantiate).

Out of curiousity, how do you manage timing / animation in your game? Are the physics / visuals in seperate threads? I'm trying to figure out the smoothest way to accomplish it in my game, which currently consists of running the physics at a specific resolution, and interpolating between samples for the actual position that is rendered on a given frame.

Quote:

Original post by skjinedmjeet Out of curiousity, how do you manage timing / animation in your game? Are the physics / visuals in seperate threads? I'm trying to figure out the smoothest way to accomplish it in my game, which currently consists of running the physics at a specific resolution, and interpolating between samples for the actual position that is rendered on a given frame.

I performed threaded physics tests and found a 2x improvement on an older PIII 700 dual processor.However, on a modern X2 4800, GFX was the bottlekneck. I dropped threaded physics for the first release (added complexity was not adding enough value). The threaded test interpolated positions/orientations linearly (lerp and slerp). IIRC, running the physics in a single thread at exactly the same rate as the GFX (frame locked), or a multiple thereof (60,120,180,240, etc.), produced the smoothest looking motion (physics motion is typically non-linear; it may be the case that small linear interps don't look as good).

The current version runs single threaded at 100Hz (with or without frame locking), with no interframe interpolation. When I run the simulation at 120Hz (video is 60Hz), there is a slight but noticeable improvement in perceptual smoothness, but not enough to be worth the extra 20Hz, nor the increased possibility of motion sickness). I'll allow users to select interframe interpolation (if requested) in a later release. You'll be able to play with the demo shortly (I'll post info in the Announcements forum and on my website when it's available).