wierd math

The literals 800 and 10240 are integers. When you divide two integers you get an integer as a result. 800/10240 is equal to the integer 0.

position.x * (800/10240) 

That line of code will result in

position.x * 0

I will assume that position.x is a float. When you multiply a float and an integer literal you get a float. When you divide a float by an integer literal you also get a float.

position.x * 800 / 10240

That line of code is the same as

(position.x * 800) / 10240

Which evaluates to a float divided by an integer, which results in a float.

Another way to get the correct result would be to replace the integer division by float division in the following manner.

position.x * (800.0f/10240.0f)

Quote:

Original post by daniel_i_l If I do: position.x*800/10240; I get the right answer but if I do position.x*(800/10240); I get 0. (Discovered this after lots of hairpulling debugging) Why? Thanks.

Not sure what type position.x is, but it most likely has to do with type promotions. In the latter example the integer division yields 0, thus the erroneous result. In the former example (presumably) the integer literal '800' is promoted to a floating-point type, and so in turn is the value '10240'.

You can identify a literal as type double by including a decimal place:

position.x*800.0/10240.0;

And as type float using the suffix f:

position.x*800.0f/10240.0f;

'Hope that helps.

[Edit: too slow.]

It doesn't matter much whether 800 is promoted to float, for instance if position.x is 200:
200*800/10240
160000/10240
15
Where floating point numbers would have given you ~15.625