Page 1 of 2
Float bug, how can I quickly fix it please ? :(
Posted: Wed Oct 24, 2007 6:55 pm
by Phlos
Code: Select all
Test.f = 1.50
For i = 1 To 5
Test + 0.30
Next i
Div.f = Round(Test / 1.00, 0) * 0.05
Debug Div ; Should be 0.15 instead of 0.10 ((3.0 / 1.0) * 0.05 = 0.15) ...)
How can I get accurate value in div please ?
Running PB 4.10 b4
Posted: Wed Oct 24, 2007 6:58 pm
by Derek
Change to doubles instead of floats.
Code: Select all
Test.d = 1.50
For i = 1 To 5
Test + 0.30
Next i
Div.d = Round(Test / 1.00, 0) *0.05
Debug Div
You will still get some small errors in accuracy but it will be a lot closer to what you are expecting.
Posted: Wed Oct 24, 2007 8:18 pm
by Psychophanta
See comments here in the snipet:
Code: Select all
Test.f = 1.50
For i = 1 To 5
Test + 0.30
Next i
;test.f is now 2.99999....
Div.f = Round(Test, 0) * 0.05 ; Round(2.999999999999999999999999999999,0) is 2.0 , And then 2.0 * 0.05 is 0.1
Debug Div; Correct result is 0.1
Posted: Wed Oct 24, 2007 8:55 pm
by Derek
But, of course, the 2.999999999 part is wrong as it should total 3.0 which it will if you use doubles.
Posted: Wed Oct 24, 2007 8:56 pm
by Psychophanta
Derek wrote:But, of course, the 2.999999999 part is wrong as it should total 3.0 which it will if you use doubles.
Agree, agree
Posted: Wed Oct 24, 2007 11:21 pm
by Phlos
I am not a newbie, I knew why it was bugged Psychophanta
Derek, I cannot use double for 2 reasons :
- my project is big (5000+ lines) and I use float since the beginning (so many many stuffs to change, and maybe new bug
)
- double doesn't really fix the issue, it hides it better
So here is my tricky solution :
Code: Select all
Test.f = 1.50
For i = 1 To 5
Test + 0.30
Next i
Div.f = Round(ValF(StrF(Test / 1.00)), 0) * 0.05
Debug Div ; Should be 0.15 and it is 0.15 now :D
Posted: Thu Oct 25, 2007 1:31 am
by Thade
Round is wrong named ... it replaces Floor and Ceil of other languages but does not round in the actual sense.
To get a rounded result always use + 0.5
Posted: Thu Oct 25, 2007 8:36 am
by Psychophanta
Thade wrote:Round is wrong named ... it replaces Floor and Ceil of other languages but does not round in the actual sense.
To get a rounded result always use + 0.5
Exactly!
So this is perhaps the more elegant solution to take:
Code: Select all
Test.f = 1.50
For i = 1 To 5
Test + 0.30
Next i
Div.f = Round(Test.f+0.5,0) * 0.05
Debug Div ; Should be 0.15 and it is 0.15 now :D
Posted: Thu Oct 25, 2007 9:21 am
by Derek
Yep, Thade is right, the round function doesn't work like you might of learned about rounding but instead rounds up or down depending on your choice so added 0.5 and rounding down works.
Posted: Thu Oct 25, 2007 4:50 pm
by Demivec
Wouldn't it be simpler to use Int() instead of Round()?
Posted: Thu Oct 25, 2007 7:04 pm
by Psychophanta
Yes, but it depends on what Phlos want to do for his program. Round() returns a float while Int() returns a long.
Posted: Sat Oct 27, 2007 2:03 am
by Phlos
I am sorry guys, but your fix seems to be wrong
If I have the float value 2.7 and I want the integer part "2", doing Round(2.7 + 0.5, 0) will return me "3" instead of "2"
You can't fix the float bad accuracy by adding a constant value
Posted: Sat Oct 27, 2007 4:50 am
by Dare
Phlos wrote:I am sorry guys, but your fix seems to be wrong
If I have the float value 2.7 and I want the integer part "2", doing Round(2.7 + 0.5, 0) will return me "3" instead of "2"
You can't fix the float bad accuracy by adding a constant value
Hi Phlos.
As 2.7 plus 0.5 is 3.2, Round(2.7 + 0.5,0) = Round(3.2,0) = 3 and so is giving you an accurate result, no?
Edit:
By adding 0.5 and rounding down (flooring) you ensure you get anything with a fractional part of .5 or higher to the next whole number and then truncate the digits. Whereas anything with a decimal fraction part under 0.5 the next whole number is not reached and thus is truncated down to the current whole number. This, of course, within the vague-eries of how floating point numbers are stored (an industry standard).
Posted: Sat Oct 27, 2007 9:31 am
by Derek
Phlos wrote:If I have the float value 2.7 and I want the integer part "2", doing Round(2.7 + 0.5, 0) will return me "3" instead of "2"
If you just want the integer part then you should use int() not round() as round() implied that you wanted the
nearest whole number which is what the +0.5 was obtaining.
Posted: Sat Oct 27, 2007 12:25 pm
by Psychophanta
Phlos wrote:I am sorry guys, but your fix seems to be wrong
If I have the float value 2.7 and I want the integer part "2", doing Round(2.7 + 0.5, 0) will return me "3" instead of "2"
You can't fix the float bad accuracy by adding a constant value
At the moment may be this workaround can help you, even it is not very elegant, it helps if you need floats and no doubles:
Code: Select all
Test.f = 1.50
For i = 1 To 5
Test + 0.30
Next i
Div.f = Round(Test.f+0.01,0) * 0.05
Debug Div ; Should be 0.15 and it is 0.15 now :D