Original Post
whenever i pass 1.25 into a function as a parameter, it turns into 1.249999999! is there a way to prevent this?
Quote:
Original post by Yohomyth
i think Fruny is right. i've tried 1.25, 1.250, 1.25f, 1.250f, even 1.25d out of curiosity, but that caused a compiler error.
Quote:
Original post by Alpha_ProgDes
o for octals
h for hexadecimal (?)
s for character strings
Quote:
Original post by Yohomyth
whenever i pass 1.25 into a function as a parameter, it turns into 1.249999999! is there a way to prevent this?
Quote:
Or the conversion from binary to decimal for displaying the number is wrong (i.e. in your debugger or in printf)
Quote:
Original post by Yohomyth
i'm not using printf
Quote:
Original post by ZQJ
some routine equivalent to printf("%f", 1.25).
Quote:
Original post by ZQJ
Second digit found to be 2
0.25 - 0.20 = 0.05
BUT in binary 0.20 is not exactly representable - it's 0.0011001100110011 etc.)
str_t WrReaVal( real_t val, int_t dig ){ str_t ret; real_t test; real_t larg; int_t len; int_t c; int_t cur; int_t i; bool_t neg; // negative? if( val < 0 ) { neg = TRUE; val *= -1; } // not negative else neg = FALSE; // count left digits if( val > 0.0 ) { // find largest digit test = 1; while( test <= val ) test *= 10; test /= 10; larg = test; // get string length if( larg >= 1 ) len = Zeros( test ) + 3 + Larger( 1, dig ); else len = ( 3 + Larger( 1, dig ) ); } // negative? if( neg ) len++; // allocate string ret = MemAlloc( len ); c = 0; // negative? if( neg ) { ret[0] = '-'; c++; } // above while( larg >= 1 ) { ret[c] = '0' + ( cur = (int_t) val / (int_t) larg ); val -= (real_t) cur * larg; c++; larg /= 10; } // decimal ret[c] = '.'; c++; // zero right digits? if( dig == 0 ) { ret[c] = '0'; c++; } // right digits else { for( i = 0; i < dig; i++ ) { ret[c] = '0' + ( cur = (int_t)( val / larg ) ); val -= (real_t) cur * larg; c++; larg /= 10; } } // return ret[c] = 0; return ret;}Quote:
Original post by Yohomyth
it just pushes 0x3FF40000 or something onto the stack. it doesn't change. the debugger displays it as 1.2499999999. my converter also makes it turn out the same way.
00111111 11110100 00000000 00000000 01111111 11101000000000000000000s eeeeeeee mmmmmmmmmmmmmmmmmmmmmmmsign bit: : 0 (positive)exponent: : 01111111 == 127float exponent bias : 127actual exponent : 127 - 127 == 0mantissa: : 11101000000000000000000normalized mantissa : 1.11101000000000000000000mantissa : 1 + 1/2 + 1/4 + 1/8 + 1/32 = 1.90625f = (-1)<sup>0</sup>*2<sup>0</sup>*1.90625 = 1.90625This topic has been locked by a moderator. New replies are not allowed.
With your permission, GameDev.net uses analytics cookies to understand how people use the platform. You can accept analytics or continue with necessary cookies only. Learn more