:

Geographic conversion

Added by: ~cuzza_fin, 2014 IX 15

Added by: ~cuzza_fin, 2014 IX 15

Hi all,
The conversion between decimal degrees and degrees minutes seconds is not equal or incorrect. Can anyone help or explain? I'm assuming it revolves around recurring numbers.
E.g.:
Converting 22°15'45.7" to decimal degrees using this online calculator, bigint ticked with the following formula:
((((45.7)/60+15)/60)+22) = 22.262694* (*4 recurs 608 times followed by a 5).

Using this number in reverse formula to get the seconds: ((22.262694-22)*60-15)*60 = 45.70* (*0 recurs 609 times followed by 205). This is not equal to 45.7.

Thanks in advance for your help,
Michael

Added by: ~t, 2014 IX 22, Last modified by: tomek, 2014 IX 22

> This is not equal to 45.7.

because ttmath uses floating point numbers:
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problem s