Float bug, how can I quickly fix it please ? :(

[Phlos]

Dare, I wanted the integer part, so when I want the integer part of 2.7, I will not get the accurate result with your +0.5 trick

I repeat for everybody !
When I do this :

Code: Select all

Test.f = 1.50
For i = 1 To 5
  Test + 0.30
Next i

You agree that Test should be equal to 3.00 right ? (1.50 + (0.30 * 5) = 3.00)
In reality it is equal to 2.99999... because float doesn't have infinite precision (stored on 32 bits, so data loss, so bad accuracy) : more you do operations, more you will get a bad result (not accurate)

So when I ask the integer part of 2.9999 (for cpu -3.00 for me-)... Round() or even the Int() function returns "2" instead of "3" : this is a good result for cpu...but wrong result for me because the float should be 3.00, so the integer part should be "3"

Derek, Round() can be used to take integer part with the option "0", so this is exactly the same as Int() but it returns a float value (which is what I want). So Int() doesn't fix it

Psychophanta, thank you for your fix, but there is still a bug.
If I have the number 2.99, it will round to "3" instead of "2", so that's still not accurate : you cannot add a constant value to fix this bad accurary, that's totally wrong.

My tricky fix is the best atm :
Round(ValF(StrF(Float.f)), 0)

I think Fred corrected the bad float accurary in StrF() function, so I get an accurate float string...and when I use it in ValF() I get back the accurate float value

[Dare]

Hi Phlos

I admit I am now a bit confused about what you are actually after (or if you have found your fix).

If you are looking for an integer part only then don't add 0.5

The 0.5 trick suggested by Thade is correct and works if you want to round up or down depending on whether the fractional part is half (and higher) or less than half.

You would use a smaller correction if you are looking to round up only when the decimal fraction is higher. So for example if you only wanted to round up values with ?.75 or higher fractional parts you would use 0.25.

Therefore to correct the tiny differences that come from inherent floating point inaccuracies use a tiny fraction.

So something like Round(num + 0.00005, 0) should do it for 4 byte floats (generally accurate to 7 digits) and an even smaller fraction should work for doubles (generally accurate to 14 or 15 digits).