Added by: ~guest, 2015 III 20

Added by: ~guest, 2015 III 20

using your Online-Calculator with "Big precision - 2048 bits mantissa, 256 bits exponent"

"2375346.6 * 5540" gives 13159420163.9~ period

"23753466 * 5540 / 10" gives 13159420164

i expect the second result for both - is the mantissa still too small?

any way to calculate the needed bits (if that is the problem)

Added by: tomek, 2015 III 24, Last modified: 2015 III 24

The problem here is conversion from decimal to binary format.23753466 and 5540 and 10 has a final binary representation but 2375346.6 gives something like that:

1.0010000111110101100101001100110011.....*10^10101

So when you enter 2375346.6 the real value the calculator sees is only an aproximation, let we say something like 2375346.59999999999999...

more info about floating point accuracy:

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problem s