Getting unexpected results when using HMAC SHA1

[1988reload]

I have tested the new HMAC functions in PureBasic 6.10 and am wondering why I am getting the same result for some of the different ASCII characters. The issue only appears between ASCII codes 127 and 160, but I cannot see why these should be special in any way. Can someone explain to me why this happens and if there is a workaround? Or better, is there any way to use a binary key instead of ASCII that also allows chr(0) to be used because that one is totally ignored?

Code: Select all

UseSHA1Fingerprint()

For i = 1 To 255
  Debug "ascii: "+ Str(i) + " (hex:" +RSet(Hex(i),2,"0")+")   hash: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, #PB_Ignore, #PB_Ascii, Chr(i), #PB_Ascii)
Next

here is part of the debugger output:

ascii: 127 (hex:7F) hash: e8f07370c85f1aac51374cfe63c1cefb90c91963
ascii: 128 (hex:80) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 129 (hex:81) hash: 4869fdf9cdd977ec26dbf87977fe18314e405fd7
ascii: 130 (hex:82) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 131 (hex:83) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 132 (hex:84) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 133 (hex:85) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 134 (hex:86) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 135 (hex:87) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 136 (hex:88) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 137 (hex:89) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 138 (hex:8A) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 139 (hex:8B) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 140 (hex:8C) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 141 (hex:8D) hash: bff0f2fe805ae3728f440420587f1d6fc2ccdf96
ascii: 142 (hex:8E) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 143 (hex:8F) hash: 159a136e3db1a906d49ed2ad372832dd61957537
ascii: 144 (hex:90) hash: 6816576aa1b1a143150c4900ae4c720fee631f87
ascii: 145 (hex:91) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 146 (hex:92) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a
ascii: 147 (hex:93) hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a

I have used https://www.liavaag.org/English/SHA-Generator/HMAC/ to compare the results and everytime bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a appears as hash it does not match the expected one.

[wilbert]

The issue only appears between ASCII codes 127 and 160, but I cannot see why these should be special in any way.

I think it has to do with Unicode (the output string of Chr) to ASCII conversion.
There are no visible Unicode characters in that range but C1 control codes.
https://en.wikipedia.org/wiki/C0_and_C1_control_codes

When you use #PB_Unicode instead of #PB_Ascii (at least for KeyFormat), does that give the result you are looking for?

Code: Select all

UseSHA1Fingerprint()

For i = 1 To 255
  Debug "ascii: "+ Str(i) + " (hex:" +RSet(Hex(i),2,"0")+")   hash: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, #PB_Ignore, #PB_Ascii, Chr(i), #PB_Unicode)
Next

[1988reload]

Yes, this works when the key is only one byte long. But the key I want to use has much more than one byte. I made the above example to find out why it is not working correctly and found out that some of the bytes are calculated wrongly.

Here is a longer key example with the above problem:

Code: Select all

UseSHA1Fingerprint()

Debug "calculated: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, #PB_Ignore, #PB_Ascii, Chr(144) + Chr(16) + Chr(33) + Chr(6) + Chr(222) + Chr(186) + Chr(192) + Chr(206), #PB_Ascii)
Debug "expected: 8ab8e5342a90a985311bbb296286a1ea36404251"
Debug ""
Debug "calculated: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, #PB_Ignore, #PB_Ascii, Chr(152) + Chr(16) + Chr(33) + Chr(6) + Chr(222) + Chr(186) + Chr(192) + Chr(206), #PB_Ascii)
Debug "expected: 196c887ee3ad81400044bcaa9fbc0ccebd350b64"

debugger output:

calculated: 8ab8e5342a90a985311bbb296286a1ea36404251
expected: 8ab8e5342a90a985311bbb296286a1ea36404251

calculated: 2ea46d7df8f7d53a7daeff85f5067f1cf74b2ce8
expected: 196c887ee3ad81400044bcaa9fbc0ccebd350b64

The first key works fine, but the second one with only one ASCII code difference is not.

#PB_Unicode is no solution because allowed Unicode value needs to be between 0 and $D7FF or between $E000 and $FFFF, so some values are not supported. The key I want to use is 64 bytes long and each byte can have any value between 0 and 255. For me it looks like this is not supported at all by Purebasic. All Purebasic examples I could find only seem to support text HMAC keys.