Decimal point problem (was: Very noob doubt is driving me crazy)

Just starting out? Need help? Post your questions and find answers here.
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3942
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: Decimal point problem (was: Very noob doubt is driving me crazy)

Post by wilbert »

PBJim wrote: Sun Sep 01, 2024 8:19 am For floating-point, type .f, there are usually 7-digits of precision after which an error occurs.
If you need accuracy, always use type .d (double precision).
It is much more accurate as .f
Windows (x64)
Raspberry Pi OS (Arm64)
User avatar
jacdelad
Addict
Addict
Posts: 1991
Joined: Wed Feb 03, 2021 12:46 pm
Location: Riesa

Re: Decimal point problem (was: Very noob doubt is driving me crazy)

Post by jacdelad »

..because you didn't do what Barry said!

6.59 would become 659. And you still use float, which is wrong. Use longs and when displaying

Code: Select all

Debug FormatNumber(0.01*12*659,2)
Good morning, that's a nice tnetennba!

PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
Mesa
Enthusiast
Enthusiast
Posts: 433
Joined: Fri Feb 24, 2012 10:19 am

Re: Decimal point problem (was: Very noob doubt is driving me crazy)

Post by Mesa »

User avatar
C87
Enthusiast
Enthusiast
Posts: 178
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Decimal point problem (was: Very noob doubt is driving me crazy)

Post by C87 »

For accurate mathematical calculations and everyday calculations like money and weight.
Use integer calcs. Do the calc, shift the decimal to the position to make it an integer.
Save that to the database or memory and shift the decimal for later recovery and display.
Using integer calcs means adding dozens, hundreds, or thousands of line totals for a final
total will never be a penny, or a kilo out. Totally removes any possibility of rounding errors.
If it's falling over......just remember the computer is never wrong!
Post Reply