Page 1 of 1

What's up with this?

Posted: Mon Mar 21, 2011 10:48 am
by Joakim Christiansen

Code: Select all

EnableExplicit

Define string$, i, checksum.a, calculatedChecksum.a

string$="10003000FCCF3091C600089528E02EBF2FEF2DBFD2"
checksum = Val("$"+Right(string$,2))
For i=0 To 19
  calculatedChecksum + Val("$"+Mid(string$,i*2+1,2))
Next
If ~calculatedChecksum+1 = checksum
  Debug "match"
Else
  Debug "no match"
EndIf
Debug checksum
Debug ~calculatedChecksum+1
Change .a to .b and it works, can someone explain why it wont work if using unsigned bytes? :?

Re: What's up with this?

Posted: Mon Mar 21, 2011 2:15 pm
by Trond
Calculations are always done signed in PB, you have to store the value in an unsigned value to treat it as unsigned.

Code: Select all

EnableExplicit

Define string$, i, checksum.a, calculatedChecksum.a

string$="10003000FCCF3091C600089528E02EBF2FEF2DBFD2"
checksum = Val("$"+Right(string$,2))
For i=0 To 19
  calculatedChecksum + Val("$"+Mid(string$,i*2+1,2))
Next
Define checksum2.a = ~calculatedChecksum+1
If checksum2 = checksum
  Debug "match"
Else
  Debug "no match"
EndIf
Debug checksum
Debug checksum2

Re: What's up with this?

Posted: Mon Mar 21, 2011 6:25 pm
by Joakim Christiansen
Aha, so the If when comparing temporary stores the value ~calculatedChecksum+1 as a signed value when it does the comparison (if I got that right).

This also fixes it:

Code: Select all

calculatedChecksum = ~calculatedChecksum+1
If calculatedChecksum = checksum
Thanks! :)
Now...to remember this everywhere in my code :shock:

Re: What's up with this?

Posted: Mon Mar 21, 2011 6:34 pm
by Zach
Find / Replace :mrgreen: