Page 2 of 2
Re: Decimal point problem (was: Very noob doubt is driving me crazy)
Posted: Sun Sep 01, 2024 9:17 am
by wilbert
PBJim wrote: Sun Sep 01, 2024 8:19 am
For floating-point, type .f, there are usually 7-digits of precision after which an error occurs.
If you need accuracy, always use type .d (double precision).
It is much more accurate as .f
Re: Decimal point problem (was: Very noob doubt is driving me crazy)
Posted: Mon Sep 02, 2024 6:48 am
by jacdelad
..because you didn't do what Barry said!
6.59 would become 659. And you still use float, which is wrong. Use longs and when displaying
Re: Decimal point problem (was: Very noob doubt is driving me crazy)
Posted: Mon Sep 02, 2024 2:15 pm
by Mesa
Re: Decimal point problem (was: Very noob doubt is driving me crazy)
Posted: Fri Sep 06, 2024 9:00 am
by C87
For accurate mathematical calculations and everyday calculations like money and weight.
Use integer calcs. Do the calc, shift the decimal to the position to make it an integer.
Save that to the database or memory and shift the decimal for later recovery and display.
Using integer calcs means adding dozens, hundreds, or thousands of line totals for a final
total will never be a penny, or a kilo out. Totally removes any possibility of rounding errors.