StrD and Rounding

[Cyllceaux]

Code: Select all

Define val1.d=ValD("83.23195000000001")
Define val2.d=ValD("83.23195")

Debug StrD(val1,4) ; 83.2320
Debug StrD(val2,4) ; 83.2319

Debug FormatNumber(val1,4) ; 83.2320
Debug FormatNumber(val2,4) ; 83.2319

both should rounding up

[STARGÅTE]

both should rounding up

No, they shouldn't. No bug!
The number created by ValD("83.23195") is 83.2319499999999980, because 83.23195 cannot be represented in as binary number.
Rounding here at the fourth digit gives 2319.

Code: Select all

Define val1.d=ValD("83.23195000000001")
Define val2.d=ValD("83.23195")

Debug StrD(val1,16) ; 83.2319500000000120
Debug StrD(val2,16) ; 83.2319499999999980

Debug StrD(val1,4) ; 83.2320
Debug StrD(val2,4) ; 83.2319

[Cyllceaux]

Sooo....

How Can I make them round up... I mean a mathematic round?
I cant use the pb function "round", cause it hat no decimal places.

So... maybe no bug, but a missing feature?


And... 0.049 should still rounding up. 49 -> 5 and 5 is rounding up

[STARGÅTE]

The "problem" is that your number 83.23195 do not exist in the computer system, just in you mind.
At the moment you enter this number, the number is already "clipped" to 83.231949999999998, because it is the nearest number to 83.23195 which can be represented by a computer (for 64 bit floats).

The only chance is, to perform the rounding you wish in the string representation.

[Fred]

One trick to round up is to add 0.5 so it might also work for lower decimals (but I can't warranty it):

Code: Select all

Define val1.d=ValD("83.23195000000001")
Define val2.d=ValD("83.23195")

Debug StrD(val1+0.00005,4) ; 83.2320
Debug StrD(val2+0.00005,4) ; 83.2319

Debug FormatNumber(val1+0.00005,4) ; 83.2320
Debug FormatNumber(val2+0.00005,4) ; 83.2319

[Cyllceaux]

Maybe StrD and FormaNumber should get a rounding Flag.

The numbers are from a customer project to calculate various liquids. This can be quite dangerous in chemistry and biology. We just came across it by chance.
The numbers are absorbance values of a skin sample via a MALDI mass spectrometer. They are read in as strings. This is the top data. Evaluations are then carried out.

This Workaround workes, but it is still a workaround.

[DarkDragon]

And... 0.049 should still rounding up. 49 -> 5 and 5 is rounding up

Wrong. You don't apply rounding recursively from the least significant to most significant digit.

[skywalk]

This is a problem of scale.
You should use fixed point math for picoliters(e-12) and milliliters(e-3), etc.

[STARGÅTE]

Maybe StrD and FormaNumber should get a rounding Flag.

What kind of flag do you mean?

In principle StrD works correct. If you have a number like 83.03125, this number is exact, also in binary format.
Therefore StrD(Val, 4) gives correctly 83.0313 (rounding up).

Code: Select all

Define Val.d = 83.03125

Debug StrD(Val, 4)

In programming language it is not always possible to perform "correct math".
Here an illustrative examples. Let be Val.d = 0.4 and then perform 3*Val - 0.8
You might think, 3*0.4-0.8 = 1.2-0.8 = 0.4, so performing this again and again should always result in 0.4.
But this isn't the case:

Code: Select all

Define Val.d = 0.4

For n = 1 To 35
	Debug StrD(Val)
	Val = 3*Val - 0.8  ; 3*0.4 - 0.8 = 1.2 - 0.8 = 0.4
Next

The calculation drifts away, because 0.4 as well as 0.8 cannot be represented in binary format and the error potentiates each time.

Therefore skywalk is right, and precise calculations should be performed in a different unit.

[skywalk]

Interesting example though I have not seen runaway as high as that with my typical floating point math.
I do get burned when doing lazy trigonometry on small shapes on a canvasgadget.
Scaling really saves the day.
Simple scaling applied to STARGATE's example:

Code: Select all

#ML_SF = 10000
Define.i i
Define.d v   = 0.4
Define.d vx  = 0.8
Define.q vi  = 0.4 * #ML_SF
Define.q vxi = 0.8 * #ML_SF
Debug "-- doubles --"
For i = 1 To 32
  Debug Str(i) + ", " + StrD(v)
  v = 3*v - vx     ; 3*0.4 - 0.8 = 1.2 - 0.8 = 0.4
Next i
Debug StrD(v)
Debug "-- scaled --"
For i = 1 To 32
  Debug Str(i) + ", " + StrD(vi)
  vi = 3*vi - vxi  ; 3*0.4 - 0.8 = 1.2 - 0.8 = 0.4
Next i
Debug StrD(vi/#ML_SF)

[jacdelad]

@Cyllceaux: How about using a Quad and move the comma?

[Cyllceaux]

My problem is... the "old" systems (Delphi, python and java) don't have problems with double. OK... java has the same problem, but there I can change to BigDecimal and this works. But I don't want to use java.

So I think I will create a Module for doing this.
@jacelad: I thought about it and I will try something
@skywalk and STARGATE: thx for the clearing. My problem is, the old systems have no problems with converting and Math-Rounding. And the new Software has the have the same results. So yeah, I will try to build something.

[STARGÅTE]

My problem is... the "old" systems (Delphi, python and java) don't have problems with double.
[...]
@skywalk and STARGATE: thx for the clearing. My problem is, the old systems have no problems with converting and Math-Rounding. And the new Software has the have the same results. So yeah, I will try to build something.

They haven't?
The math of a computer is universal. All languages use the same number system, when they use IEEE doubles.
Here the same "issue" in Python:

Code: Select all

val = 83.23195
print(val)
print(format(val, ".4f"))
print(round(val, 4))

83.23195
83.2319
83.2319

Also the Pure Basic community has published some includes and modules for math with fixed-point-numbers or for currency values.

[Cyllceaux]

I don't know how they do this. I'm the new one to replace these systems

[Marco2007]

Code: Select all

Define Val.d = 0.4

For n = 1 To 35
	Debug StrD(Val)
	Val = 3*Val - 0.8  ; 3*0.4 - 0.8 = 1.2 - 0.8 = 0.4
Next

The calculation drifts away, because 0.4 as well as 0.8 cannot be represented in binary format and the error potentiates each time.