HMAC not working correctly when using #PB_Ascii mode

[1988reload]

I tried different ways to calculate a correct HMAC SHA1 hash and had no success. I shortened the code more and more to find out where the issue might be. The result was that some ASCII characters generated a bad result when used as key.

I have also asked here for help, but got no solution: https://www.purebasic.fr/english/viewtopic.php?t=84677

Code: Select all

UseSHA1Fingerprint()
For i = 125 To 165
  Debug "ascii: "+ Str(i) + "   hash: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, #PB_Ignore, #PB_Ascii, Chr(i), #PB_Ascii)
Next

As can be seen the key is different for every loop but the hash output is the same for much of them (bad hash result: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a).
The problem only happens when "HmacKeyFormat" is using #PB_Ascii mode.

[infratec]

For testing:

https://cryptii.com/pipes/hmac

The hash for i = 129 ($81) is correct

ascii: 0x81 hash: 4869fdf9cdd977ec26dbf87977fe18314e405fd7

is

4869fdf9cdd977ec26dbf87977fe18314e405fd7

$82 is wrong and $80 is also wrong,

ascii: 0x80 hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a

should be

ecb48ecfa43c92659858fa328b7dbb03df2c6075

ascii: 0x82 hash: bcc96068c1cc6ecc361a9a26e8a9282b8204cd3a

should be

c8dddb1f662096b058dff1519a150961ffb42f43

Maybe there is a padding to the one byte key, which is not correct.
Should always be padded with 00

Verfied also with:
https://www.liavaag.org/English/SHA-Generator/HMAC/

[infratec]

Same thing with FingerPrint()

Code: Select all

*Buffer = UTF8("test")
If *Buffer
  ;ShowMemoryViewer(*Buffer, MemorySize(*Buffer) - 1)
  Hash$ = Fingerprint(*Buffer, MemorySize(*Buffer) - 1, #PB_Cipher_SHA1|#PB_Cipher_HMAC, 0, Chr($80), #PB_Ascii)
  Debug Hash$
  FreeMemory(*Buffer)
EndIf

[infratec]

Btw. at the moment it is impossible to use 0 inside of the HMAC key, since it is always a string.

[infratec]

Code: Select all

UseSHA1Fingerprint()

For i = $7D To $A5
  Key$ = Chr(i)
  *Buffer = Ascii(Key$)
  ShowMemoryViewer(*Buffer, MemorySize(*Buffer) - 1)
  FreeMemory(*Buffer)
  Debug "ascii: 0x"+ RSet(Hex(i), 2, "0") + "   hash: " + StringFingerprint("test", #PB_Cipher_SHA1|#PB_Cipher_HMAC, 0, #PB_Ascii, Key$, #PB_Ascii)
Next

With a breakpoint at FreeMemory() you can see that the wrong ascii keys are all converted to $3F

[idle]

looks like it's expecting UTF8. $80 is where you switch to 2 bytes

[infratec]

It is explicit #PB_ASCII an not UTF8.

Can you give an explanation for this:

Code: Select all

Debug Hex(Asc(Chr($82)))  ; Ok 82
*ptr = Ascii(Chr($82))
If *ptr
  ShowMemoryViewer(*ptr , MemorySize(*ptr) - 1) ; not Ok 3F
  FreeMemory(*ptr)
EndIf

Btw. PB 6.11 X86 assembler WIndows 10 x64

[idle]

This shows it better.

Code: Select all

Debug Bin($81)

Debug Hex(Asc(Chr(129)))
*ptr = Ascii(Chr(129))
If *ptr
  ShowMemoryViewer(*ptr , MemorySize(*ptr) - 1)
  FreeMemory(*ptr)
EndIf

[wilbert]

I did try to explain it in the other thread 1988reload is referring to.

PureBasic nowadays is Unicode internally.
The result of Chr(130) is a Unicode string with Unicode character number 130.
That specific character is one of the C1 control codes.
https://en.wikipedia.org/wiki/C0_and_C1_control_codes
It has no Ascii equivalent so that's why it is converted into a question mark.

[infratec]

Then

Code: Select all

Debug Hex(Asc(Chr($82)))

should also show a 3F.
But it shows the correct 82.
I expect that Ascii() should do the same.
Values below 256 can not return a ? (3F)

[1988reload]

I did try to explain it in the other thread 1988reload is referring to.

PureBasic nowadays is Unicode internally.

I understood that. But if the result is wrong because of that then it is a bug, isn't it? I think #PB_Ascii should work like ASCII even if it is calculated as Unicode internally.

Btw. at the moment it is impossible to use 0 inside of the HMAC key, since it is always a string.

Yes, this is an issue for me. But description in Purebasic tells that it is working like that, so it is not a bug.

I think because of this ASCII and String issues (CHR(0), and Unicode control codes) we have "FileFingerprint" for the data input as workaround. Sadly there is nothing similar for the HMAC key.

[1988reload]

are there any news? will this issue be solved? is it a bug or wanted to be like that? not being able to use all possible ascii characters as key is not good if someone like me wants to decode data generated with a key that purebasic can not handle.

[Fred]

I guess something was overlooked when introducing the HMAC function, as not being to use 0 in a key is a also an issue. As said, Ch(130) means nothing in unicode, that's why it gets converted back to wrong ascii. So the best move I guess is to convert the string Key$ parameter to a raw memory buffer so you can put what you want in it (new signature would be StringFingerprint(String, Plugin [, Bits [, Format [, *Key, KeySize]]])

[infratec]

@1988reload
viewtopic.php?p=625418