🔒 Locked

Javascript math problem.

Started by Xest Jan 22, 2006 at 2:54 PM 7 replies 2.2k views

Original Post

Jan 22, 2006 at 2:54 PM

Hi there, I've got the user inputting a number into a dialog and setting it to a variable by just using myvar = window.prompt('blahblah','') When I then multiply the variable by another number I get some odd results, for example if myvar is set to 1.40 and I multiply it by 23 I get a result of: 23 multiplied by 1.40 is 32.199999999999996 When writing it out to my document. According to my calculators (I've checked on a couple!) it should be 23.2 however. I tried using parseFloat to ensure that myvar was correctly being set as a floating point number however that made no difference. Just to confirm that I wasn't being totally ignorant to basic programming stuff here I gave the same thing a try in a quick C program: printf("Output: %f\n", (1.40 * 23)); The result was as expected: Output: 32.200000 I may be overlooking some extremely simple basic math here because I'm partially asleep but I'm just having mental block right now and can't for the life of me figure out what's causing this result. Can anyone explain this anomaly for me ?

sit

Jan 22, 2006 at 3:15 PM

1.40 = 1 + 40 / 100 = 1 + 4 / 10 = 1 + 2 / 5

well, 1 is easy to represent in binary, how do you represent 2 / 5 in binary then?

0.0110011001100110011001100110011001100110011001100110011001100110... (0110 repeats forever)

so the problem is that you're not rounding after the calculation before displaying. Your conversion got as close as it could [just a tad short] and by multiplying the difference became larger.

your C program is rounding, not sure why the other one is not.

Prototype

402

Prototype

Jan 22, 2006 at 3:38 PM

javascript suffers from horrible rounding errors, I've encountered that many times. You also have to realize that all computers basically have this problem with floating point numbers; after all the number of digits is limited so not every number can be pecisely represented.

More

Xest

Author

Xest

Jan 22, 2006 at 4:14 PM

Thanks for the replies - I'm just not used to seeing this kind of problem in C, I realise floating point numbers can cause problems, particularly where a recurring value exists, but in this situation there is no recurring value which is what's got me puzzled somewhat.

markr

159

markr

Jan 22, 2006 at 4:16 PM

This has absolutely nothing to do with javascript, and is inherent in finite-precision binary maths systems such as floating point. It will happen in C and C++ too. The solution is to round the answer as necessary. The answer will be as precise as it's possible to represent in whatever precision javascript's Number has (Which I suspect is somewhat implementation-dependent). Have a look at the toFixed() property of Number Mark

Xest

Author

Xest

Jan 22, 2006 at 4:31 PM

But why in a sum where only 1 decimal place of precision is needed for a precise exact answer does javascript fall over where C doesn't? The answer in this case should be 32.2, not 32.2111111 recurring or 32.1999999 recurring which is why I'm a little puzzled, put simply javascript seems to have done the maths wrong in this scenario. If I trim down to 2 decimal places I'm going to end up with an answer of 32.19 which is incorrect, I suppose the only solution is to round to 2 decimal places here.

markr

159

markr

Jan 22, 2006 at 4:35 PM

Quote:
Original post by Xest
But why in a sum where only 1 decimal place of precision is needed for a precise exact answer does javascript fall over where C doesn't? </td></tr></table></BLOCKQUOTE> They both fail to have the "right" answer, or rather, they both suffer from the same problem, which doesn't mean it isn't right, because it's a well-known and totally documented behaviour. <BLOCKQUOTE>Quote:<table border=0 cellpadding=4 cellspacing=0 width="95%"><tr><td class=quote> The answer in this case should be 32.2, not 32.2111111 recurring or 32.1999999 recurring which is why I'm a little puzzled, put simply javascript seems to have done the maths wrong in this scenario. </td></tr></table></BLOCKQUOTE> The same reason that any floating point app would. If you want the right answer in every case, use an arbritary precision maths library. If the problem doesn't show up in C, it's because the way you're printing out the number must be rounding it. Mark

nicksterdomus

Jan 22, 2006 at 4:58 PM

I changed the C program to

printf("%.15f\n", 23*1.40);

output: 32.199999999999996

Now it prints out the same amount of digits as the javascript one, and you get the same answer also. So pretty much the printf function was rounding when it outputted the answer, but the actual value that was computed was not exact.

Xest

Author

Xest

Jan 22, 2006 at 6:09 PM

Thanks for the explanations all ;)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.