Rounding

[Lima]

Value.d = 21.505

Debug "Result: " + StrD(Value, 2)
Debug "Result: " + FormatNumber(Value)


; Result: 21.50
; Result: 21.50

The result of the above code should not be 21.51 ?

Thank you for any help.

[skywalk]

Code: Select all

Procedure.s StrDr(X.d, nPlaces.i=2)
  Protected.i Scale = Pow(10, nPlaces)
  ProcedureReturn StrD(Round(X*Scale, #PB_Round_Nearest)/Scale, nPlaces)
EndProcedure

Value.d = 21.505
Debug "Result: " + StrD(Value,  2)
Debug "Result: " + StrDr(Value, 2)

[Lima]

Thanks a lot for the help.
But I repeat the question: should not the StrD() and FormatNumber()
do that ?
It is a mistake, PB failure?
Thank you.

[skywalk]

There is a Round() function to allow user selection of Up,Down,Nearest.
You could make a feature request to include that functionality in FormatNumber() and StrD() as optional parameters.

[kenmo]

It's not a bug or a missing feature

It's the usual fact that floats/doubles cannot store every number exactly as written.
21.505 in the source code is not exactly 21.505 when stored in a double variable.

Whereas 21.605 rounds how you expect, for example.

Code: Select all

Value.d = 21.605
Debug Value
Debug StrD(Value, 2)

Debug ""

Value.d = 21.505
Debug Value
Debug StrD(Value, 2)

EDIT: In case your results are different, mine are:

21.605
21.61

21.504999999999999
21.50

[skywalk]

Yes, but this begs the question of utility?
Is StrD() used by anyone to display floating point data without rounding?
If so, they must know it is not rounded.

[wilbert]

If so, they must know it is not rounded.

It is rounded

[skywalk]

You are correct, but the banker disagrees.
Of course, Integer math is required for financial calc's.

Code: Select all

Procedure.s StrDr(X.d, nPlaces.i=2)
  ; Round Double to nPlaces. StrD() does not perform rounding.
  Protected.i Scale = Pow(10, nPlaces)
  ProcedureReturn StrD(Round(X*Scale, #PB_Round_Nearest)/Scale, nPlaces)
EndProcedure
Define.d x = ValD("3.1415926535897931")
Define.i nPlaces = 7
Debug x
Debug "StrD(x,"+Str(nPlaces)+"):  " + StrD(x, nPlaces)
Debug "StrDr(x,"+Str(nPlaces)+"): " + StrDr(x, nPlaces)
nPlaces = 2
x = ValD("2.505")
Debug x
Debug "StrD(x,"+Str(nPlaces)+"):  " + StrD(x, nPlaces)
Debug "StrDr(x,"+Str(nPlaces)+"): " + StrDr(x, nPlaces)
; 3.1415926535897931
; StrD(x,7):  3.1415927
; StrDr(x,7): 3.1415927
; 2.5049999999999999
; StrD(x,2):  2.50
; StrDr(x,2): 2.51

[Blue]

Duplicating skywalk's logical process, but skipping the string conversions.
We get the same results, with a faster process.

Code: Select all

Procedure.d DRound(x.d, nPlaces.i=2)
  nPlaces = Pow(10, nPlaces)
  ProcedureReturn Round(x*nPlaces, #PB_Round_Nearest)/nPlaces
EndProcedure

Define.i nPlaces = 7
Define.d x = 3.1415926535897931

Debug x
Debug "StrD(x,"+Str(nPlaces)+"):  " + StrD(x, nPlaces)
Debug "dRound(x,"+Str(nPlaces)+"): " + dRound(x, nPlaces)
Debug ""

nPlaces = 2
x = 2.505

Debug x
Debug "StrD(x,"+Str(nPlaces)+"):  " + StrD(x, nPlaces)
Debug "dRound(x,"+Str(nPlaces)+"): " + dRound(x, nPlaces)

;- Results : 
; 3.1415926535897931
; StrD(x,7):  3.1415927
; DRound(x,7): 3.1415927
; 2.5049999999999999
; StrD(x,2):  2.50
; DRound(x,2): 2.51

Thanks for the algorithm, skywalk.
Just what i was looking for.

[Machupichu]

Hi, friends.
I was checking doubles.
(I am newbie )
Is this a bug?

Define x.d=678437923546269.58
Debug x ; 678437923546269.62 <<<--- Error?? '.62'

[Demivec]

Hi, friends.
I was checking doubles.
(I am newbie )
Is this a bug?

Define x.d=678437923546269.58
Debug x ; 678437923546269.62 <<<--- Error?? '.62'

No it is not a bug. It is due to limitations on the binary precision of the decimal portion (i.e. binary representation is limited on what values it can represent precisely, the rest are approximations) as well as the scale of the number in your example (i.e. large values will have less precision in the fractional decimal portion). If you reduce the scale of the value by a factor of ten by remove a digital from the left side of the decimal you will see the fractional part is a closer approximation of the decimal you entered.