Skip to main content
GameDev.net gamedev.net
🔒 Locked

double number subraction error

Started by Craigjw123 Dec 21, 2001 at 10:07 AM 4 replies 300+ views
Original Post
Craigjw123
Craigjw123
When I subtract a double from another, when the result should be zero it isn''t, ie: /////// code /// double A1 = 0.1; double result = 0.7; for( int i = 0; i < 7; i++ ) { cout << "Result: " << result << endl; result -= A1; } cout << "...Result: " << result << endl; ///// end of code //////// A should be 0.00.... However, it displays this: 1.3878e-016 or something just as small, but still if I was working to 20 decimal places, it would screw things up. Any suggestions please. Craigjw123@aol.com
AdamMB
AdamMB
What you are seeing is simply floating point round off error in action. This is the reason if you try to use comparison statements such as IFs, you have to use a tolerance such as

if( a < 0.001 && a >= 0.0) { CODE }

As far as I know, there is not a way to solve that problem, but I haven''t had to work with the FPU too much. If there is a solution, it''s probably going to be changing something in the FPU status register or something similar to tell it to use more accurate calculations at the cost of speed, but as I said, I''m not sure that this is even possible. The only other work-around I can think of would be to go in and write your own code to do all the calcs by hand, which would be horribly ugly and very, very, very long because most likely, it''d have to be in ASM. No matter what you do though, you will probably never get something to come out exactly to 0.0

-Adam
Craigjw123
Craigjw123
Writing the FPU sounds to hairy to me! If I do the subraction on anything other than zero, it works fine, it is only when it gets to the zero it decides that there should be error, and all of a sudden the error is multiplied upto e-16, or however many times I do the calculations.

Craigjw123
Craigjw123
Might there be an error option in Vis C somewhere ???

Oluseyi
Oluseyi
quote:
Original post by Craigjw123
Might there be an error option in Vis C somewhere ???

Why should there be? C/C++ use the IEEE floating-point representation, and there are inherent issues in manipulating them. The responsibility lies with the programmer to avoid the potential pitfalls.

[ GDNet Start Here | GDNet FAQ | MS RTFM | STL | Google ]
Thanks to Kylotan for the idea!
Craigjw123
Craigjw123
I don''t think that I am questioning the IEEE stds, but rather an issue on output stream precision, I have tried to use
cout << setw(5) << setprecision(3) << ios::fixed << mydouble etc.
and to no avail, it still likes the old 3.1e-16 format.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.