Least Significant Digits Error
Entering the expression "mod( 1.23456789E36; (2*pi) ) - (2*pi)" into both TTCalc and Big Online Calculator returns a result with the 35 or 36 least significant digits being incorrect.
Using TTCalc set to medium precision, the above expression gives a result of:
-0.429902122666408133629338511837051281544042148281442325979 451679332963508128029785681
Using big precision, the result is:
-0.429902122666408133629338511837051281544042148281424617916 850946376963393013262932538459140668800808171227460809013829 767121050503589151228780064735344110949132295842863619282083 048321630363305931694981093771748238093870849896771478144408 766957176130433190794
Note that the 35 least significant digits in the medium precision result are different than the corresponding digits in the big precision result.
Big Online Calculator gives similar results.
Regards,
Chuck
cos(1.5707963267948966192313216916397514420985846996876)retu rns a result with similar errors.
The problem with floating point numbers is that they are only an approximation to the real value. Sometimes
the approximation differs only at last digit sometimes there are more inaccurate digits. In expression "mod(
1.23456789E36; (2*pi) ) - (2*pi)" the most inaccurate function is 'mod'. This function makes larger
mistake when the first argument is relatively large comparing to the second. If you try "mod( 10000; (2*pi)
)" then the result would be (ttcalc medium precision):
3,4521762772779152118687544046218224846069711884132776577263 07276528195197282966365607 and the last three digits
are incorrect, if you change "10000" to "10000000" then more digits would be inaccurate. The
'mod' function is calculated from such an expression:
mod(x1; x2) = x1 - int(x1 / x2) * x2
maybe in the future it will be changed but at the moment there is no a simple remedy to the accuracy.
Similar is with the 'cos' function. 'Cos' is calculated by using Taylor series. The most inaccurate places are where the 'cos' has about 1, 0 or -1 values. It is in: "k * (pi/2)" where k is integer. 1,5707963267948966192313216916397514420985846996875 is very closed to pi/2 and you should assume the result to be a zero.
You should read more about precision in floating point numbers, you can start from wikipedia ("Accuracy
problems" topic):
http://en.wikipedia.org/wiki/Floating_point