Page 2 of 2

Re: StrD and Rounding

Posted: Wed Feb 21, 2024 9:55 pm
by Olli
We should do a best of all the times we break the ghoul in this floating binary format...

Re: StrD and Rounding

Posted: Wed Feb 21, 2024 10:03 pm
by Tenaja
You could multiply by 10, or 100, or 10000 it whatever precision you want, then do your rounding. Then divide back, if you want decimals (or do it in the string).

Re: StrD and Rounding

Posted: Thu Feb 22, 2024 9:50 am
by Cyllceaux
My solution. Not good, but worked:

Code: Select all

Define val1.d=ValD("83.23195000000001")
Define val2.d=ValD("83.23195")

Debug StrD(val1,4) ; 83.2320
Debug StrD(val2,4) ; 83.2319

Debug FormatNumber(val1,4) ; 83.2320
Debug FormatNumber(val2,4) ; 83.2319

Debug "--------------------------------------"

Procedure.d RoundToDecimalPlaces(value.d, decimalPlaces)
  Protected multiplier.d = Pow(10 , decimalPlaces)
  Protected roundedValue.d = Round(value * multiplier,#PB_Round_Nearest) / multiplier
  ProcedureReturn roundedValue
EndProcedure

Define val1b.d=RoundToDecimalPlaces(val1, 4)
Define val2b.d=RoundToDecimalPlaces(val2, 4)

Debug StrD(val1b,4) ; 83.2320
Debug StrD(val2b,4) ; 83.2320

Debug FormatNumber(val1b,4) ; 83.2320
Debug FormatNumber(val2b,4) ; 83.2320
Thx @all