Re: ValF("2.3") sometimes returns "2.0"
Posted: Wed Sep 14, 2011 8:08 pm
Well, i'm just going on the general conversation flow here because you did not provide any real examples of what your working with. You said 2.3, and thats what I used, and didn't make any mention of such large variations as your alluding to now. Maybe you should be using doubles then, or ensure your using multiples of 2 in the float values at the very least, if your seeing such large variations, but considering that most code requirements round at some point then an accuracy of ValF("2.4") = 2.40000009536743 is still 2.4000000 default rounded for all practical purposes despite the very small inaccuracy. Are you sure your not accidently rounding up or down somewhere or adding or multiplying somewhere that would affect default rounding? I've never seen a float just simply get truncated right after the decimal point for no reason at all, of course i'm certainly no expert and have not seen it all. Your FPU on the processor isn't turned off in the BIOS is it? Its not called FPU in the BIOS if I remember correctly, its something else, so many different BIOS's and so many computers over the years I just can't remember them all. I did see a few CPU's once that were overclocked affect the accuracy of a float, your not overclocked are you? Just guessing........