Math of decimal point!!

[ankachen]

Hi guys, I got a little problem with math.

I wonna have the result of "a.f=1/10".

I got "0.10000000149012".

If I wonna get jst "0.1", how could I gonna do?

any suggestions will be appreciated.

[Matt]

Check out what the manual says:

A floating point number is stored in a way that makes the binary point "float" around the number, so that it is possible to store very large numbers or very small numbers. However, you cannot store very large numbers with very high accuracy (big and small numbers at the same time, so to speak).

Another limitation of floating point numbers is that they still work in binary, so they can only store numbers exactly which can be made up of multiples and divisions of 2. This is especially important to realise when you try to print a floating point number in a human readable form (or when performing operations on that float) - storing numbers like 0.5 or 0.125 is easy because they are divisions of 2. Storing numbers such as 0.11 are more difficult and may be stored as a number such as 0.10999999. You can try to display to only a limited range of digits, but do not be surprised if the number displays different from what you would expect!

This applies to floating point numbers in general, not just those in PureBasic.

Like the name says the doubles have double-precision (64 bit) compared to the single-precision of the floats (32 bit). So if you need more accurate results with floating point numbers use doubles instead of floats.

[Kale]

You can mitigate the wierd results a little by using doubles. This is about as accurate as floating point numbers get i'm afraid.

Code: Select all

a.d = 1 / 10
Debug a

[Dreamland Fantasy]

Hi guys, I got a little problem with math.

I wonna have the result of "a.f=1/10".

I got "0.10000000149012".

If I wonna get jst "0.1", how could I gonna do?

any suggestions will be appreciated.

If you know the number of decimal places that you need and just want to display the number you can use something like:

Code: Select all

a.f = 1 / 10
Debug StrF(a, 1)

Kind regards,

Francis

[jack]

here a bcd mat dll that you can use depending on your needs as bcd is slow. http://www.crbond.com/extended_precision.htm