Page 1 of 1

Division problems

Posted: Wed Apr 11, 2007 7:38 pm
by codefire
Here's my code:

Code: Select all

Declare TestDivision ()

Procedure TestDivision ()

  x.q = 4192456821
  y.q = x/60
  z.q = y*60
  If OpenConsole()   
    PrintN(Str(x))
    PrintN(Str(z))
    Print("Hit enter to continue")
    Input()
  EndIf
EndProcedure

TestDivision()

I get:
-102510475
-102510496
Hit enter to continue

If I change x to be 1192456821 I get:

1192456821
1192456800
Hit enter to continue

Either way that doesn't seem right. What's happening. Am I doing something wrong? Those numbers should fit happily into a quad shouldn't they?

Thanks,
Codefire

Posted: Wed Apr 11, 2007 7:57 pm
by Clutch
There are two changes you need to make in order to get this to work. One involves the fact that the value for 'x' is not evenly divisible by 60, and any remainder is being ignored. Change 'y' from a quad to a double, and the calculation will be correct. The second part has to see with using Str() on a quad value, as it is meant for a long. Use StrQ() instead.

Code: Select all

Declare TestDivision ()

Procedure TestDivision ()

  x.q = 4192456821
  y.d = x/60
  z.q = y*60
  If OpenConsole()
    PrintN(StrQ(x))
    PrintN(StrQ(z))
    Print("Hit enter to continue")
    Input()
  EndIf
EndProcedure

TestDivision()

Posted: Wed Apr 11, 2007 8:01 pm
by srod
The result of the division is truncated because y is an integral (whole number) data type. So effectively, the remainder is being discarded. Hence, the value of z could be out by as much as 59 (when dividing by 60)

e.g. if a=5, then b=a/2 will assign b the value of 2 since the remainder of 1 will have been discarded.

Posted: Thu Apr 12, 2007 7:54 am
by codefire
Yes, that's fixed it...thanks guys!