🔒 Locked

Float problem

Started by jammy5202 Apr 13, 2006 at 10:33 AM 3 replies 450+ views

Original Post

Apr 13, 2006 at 10:33 AM

I have never had a problem with using floats before now. I my program i have created a float like this float fNum = 7.1f; I would expect it to store 7.1 but it stores 7.099999999. I have know i dea why it is doing this as tried it in another program and it works fine. Could it have anything to do with directx.

Evil Steve

Apr 13, 2006 at 10:38 AM

D3D changes the FPU precision. You can tell it not to do that by a flag to CreateDevice: D3DCREATE_FPU_PRESERVE.

However, you should still expect this sort of error. A float can't hold a number exactly, only an approximation. It might appear to work for one number, but if you chaneg it slightly you could be off.

Oluseyi

Apr 13, 2006 at 10:41 AM

It is the nature of floating point objects to have precisely that sort of imprecision. This is why it is recommended that you never test floats for equality; instead you use an epsilon or margin of error:

float f = GetSomeFloatValue();const float epsilon = 0.0001;if(f == 0.001) // virtually guaranteed never to work{  ExplicitTestWorked();}if(f - 0.001 < epsilon){  ApproximatelyEqual();}

It all stems from the way that floating-point numbers are represented, according to the IEEE-754 Standard for Binary Floating Point Arithmetic.

Emmanuel Deloget

Apr 13, 2006 at 10:58 AM

clicky linky for a very good report on the subject.

jammy5202

Apr 13, 2006 at 11:01 AM

if i what to test 2 float point value are the same i use this method. like 10.05 and 10.5 where the same how would i do that as both would not pass either test. Ok just seen the post above was posted will i wrote this message

[Edited by - jammy5202 on April 13, 2006 11:01:45 AM]

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.