Possible Rounding issue in NAV

jordi79

Hi,

Code to perform a rounding as as follows:

Dec1 := 14.32455;

Precision := 0.01;

MESSAGE(FORMAT(ROUND(Dec1, Precision, '=')));

Once code is executed, the result is: 14.32

I would have expected the output to be 14.33. Is my assumption correct? This code is executed in NAV ver 5Sp1. The same result is observed in ver NAV 3.10.

Belias

no, the result is mathematically correct, if you round to the second dec, you have to take into account only the third decimal. example:
2,444444444444444444445 rounded to the second dec is 2,44

jordi79

Hi Belias,

You are correct. The result IS mathematically correct. What was I thinking! Must be all the hours spent staring at the computer have fried my brains.

Belias

Honestly, after reading your post, i have had 10 seconds of doubt about it, too