[5.60] ValF( "10.24") = 10.23999977111816

[Yuri_D]

Hi!

I faced a problem that any float values except .0 and .5 looks like in subj
Is it a bug?

[kenmo]

Floats can't store most numbers exactly, regardless of programming language.
http://en.wikipedia.org/wiki/Floating-p ... y_problems

It only approximates it, such as x ~ 1/2 + 0/4 + 1/8 + 1/16 + ...
For displaying to the user, you can use some rounding tricks.

Doubles are closer, but still not guaranteed exact.

A comparison in C:

Code: Select all

void main(void) {
  float  f = 10.24;
  double d = 10.24;
  printf("%.14f \r\n%.14f \r\n", f, d);
}

Code: Select all

10.23999977111816 
10.24000000000000

[Yuri_D]

I use Floats very rarely so I forgot about this limitation...(
Besides the PB help confused me:

Code: Select all

...
Remarks

Strings holding an integer can also be converted with Val() and 64-bit floats with ValD() (with more accuracy than ValF()). 
Example

  Debug ValF("10.24")     ; will display 10.24
...

[tj1010]

Even the latest MSVC and GCC have this problem. This is why people in finance and simulation have to manually handle precision in their tools.. It's an annoying old problem. I know from binary analysis PB uses a unique flow with integer to float conversion that no other language uses too.

[Dude]

Debug ValF("10.24") ; will display 10.24

That example in the manual is wrong. It displays 10.23999977111816 instead. Using ValD("10.24") displays 10.24, though.