ValF() Issue

[PB]

These are bugs, surely?

Code: Select all

n$="1234567890"
Debug ValF(n$) ; Returns 1234567936.0

Code: Select all

n$="1234567890"
n.f=ValF(n$)
Debug n ; Returns 1234567936.0

[Deeem2031]

Try doubles if floats are too inaccurate for you. No bug.

[PB]

Yes, doubles work, but 1234567890 isn't a large number, so I would've thought
that it'd fit easily into a float? How many digits is a float accurate too, then?
Is it best just to ditch floats altogether and just rely on doubles for everything?
Because that's how it comes across.

[freak]

Floats have a precision of about 7 decimal digits, doubles have a precision of 16 digits.

[PB]

But there's no decimal points in 1234567890, though. Well, to be technical
it's 1234567890.0 then, but that's only 1 decimal point. What am I not
understanding here?

[rsts]

He might mean digits, not points.

cheers

[Demivec]

But there's no decimal points in 1234567890, though. Well, to be technical
it's 1234567890.0 then, but that's only 1 decimal point. What am I not
understanding here?

If there is precision to only 7 digits the 1234567890.0 shows which digit is the last accurate one. So if you convert the string "1234567890" you will get precision in the first 7 digits and the rest are less precise (garbage) 1234567936.0 . The less precise ones are due to limitations converting binary to decimal, i.e. 4 binary digits %0100 are only accurate to 1 decimal digit because they represent values of 0 to 15. So can only represent the first decimal place accurately (0 to 9), the second decimal place can only be represented from 0 to 5 and so it is inaccurate. If you increase the # of bits, and include an exponent you can see why there is a limitation with the floats and doubles.

[PB]

> He might mean digits, not points

Ah, that's where my confusion comes from. Thanks for clarifying!

Now for all my apps:

Search: .f
Replace: .d