AES-128-CBC - replicate openssl result

[firace]

I'm trying to obtain the same result as this command:

Code: Select all

echo "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt

(output should be "VRWqHjP23NtfjqhgWpVsXg==")

Can anyone see what I'm doing wrong? Thanks.

Code: Select all

;; just an educational exercise
;; echo "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt


Procedure Bytes(asString$)
  Define xAsHexString$, lengthOfRawBytes, *xAsRawBytes, k    
  
  xAsHexString$       = asString$
  
  lengthOfRawBytes =  Len(xAsHexString$)/2     
  *xAsRawBytes     = AllocateMemory(lengthOfRawBytes)
  
  
  For k=0 To lengthOfRawBytes-1 
    PokeA(*xAsRawBytes+k   ,Val("$"+PeekS(@xAsHexString$ +(k*4)   ,2)))
  Next
  ProcedureReturn *xAsRawBytes
EndProcedure


Procedure DeriveKey(pwd$) ;; openssl style for AES 128 CBC  (uses MD5!)
  UseMD5Fingerprint()
  MD5$ = StringFingerprint(pwd$, #PB_Cipher_MD5, 0, #PB_Unicode)
  *MD5Bytes = AllocateMemory(16)
  *MD5Bytes = bytes(MD5$)
  ShowMemoryViewer(*MD5Bytes, 16)
  ProcedureReturn *MD5Bytes
EndProcedure   

 

*iv= bytes("20002000200020002000200020002000")            ;; iv must always be 16 bytes
pass$="abc1234"                                           ;; key (not pass) must be 16 bytes for AES 128, 32 bytes for AES 256

;  ShowMemoryViewer(@iv$, 16)

test1$ = "SUPERBIGSECRET" 
*test2 = AllocateMemory(StringByteLength(test1$))
test3$ = Space(Len(test1$)) 

AESEncoder(@test1$,  *test2, MemorySize(*test2) , DeriveKey(pass$), 128, *iv, #PB_Cipher_CBC)

b64$ = Base64Encoder(*test2, MemorySize(*test2))

Debug b64$   ;;   should be  "VRWqHjP23NtfjqhgWpVsXg=="

[NicTheQuick]

One of the main differences would be that strings in Purebasic are UTF-16 and not UTF-8 like they are in the terminal.

Also if I run that CLI command on my system the result is: gO4ucnBRzoO2pmqNnHyaBA==
So it also differs from yours and I don't know why.

[infratec]

Use UTF8()

An then you need padding, because SUPERBIGSECRET is < 16 bytes.

Also you need to do this without the trailing 0 of the PB strings

...

It is not so easy as it looks.

[Olli]

Nutty question : What is << -nosalt >>

Code: Select all

echo -n "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -K 60616230313234 -iv 20002000200020002000200020002000 -nosalt

?

[NicTheQuick]

Nutty question : What is << -nosalt >>

Code: Select all

echo -n "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -K 60616230313234 -iv 20002000200020002000200020002000 -nosalt

?

You can find all the options and their explanations here: https://docs.openssl.org/3.0/man1/openssl-enc/#options

[Olli]

osh... I have such a headaches that I was typing a very old password I forgot long time ago, instead of my actual password. Sad world where we risk to loose ourselves, just for a biological memory exception...

Thank you nicTheQuick for the glossary. And thank you too to firace for this interesting subject.

I think, that, to reach the goal (get a standardized encryption), we should do this, step by step.

Why not start by a single upper 'A' character which could be converted to a base64 string ?

I think, that, through this humble step, all the questions about string encoding, and string termination rule should naturally appear ?

It seems (and it is to be confirmed), that the IDE uses 2 encoding modes : Unicode for displaying, and UTF-8 to save/load on a source file. Maybe I am wrong but all these details should be explicitly searched...

What do you think about this slow method I am suggesting ?

[Olli]

Example :

If two persons could test openssl to get the same result as this :

Code: Select all

Debug Base64Encoder(Ascii("A"), 1)

Which gives :

Code: Select all

QQ==

(I am already surprised by the result, but, under a headaches, everything can surprise me...)

Code: Select all

echo -n "A" | openssl -base64

[infratec]

Very strange.

For a test I use directly the iv and the key which is shown by the openssl command.
My data is correctly padded with the pkcs5 method.
I don't know why the output is .. wrong.

Code: Select all

;echo "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt -p
;enter AES-128-CBC encryption password: abc1234
;Verifying - enter AES-128-CBC encryption password: abc1234
;*** WARNING : deprecated key derivation used.
;Using -iter Or -pbkdf2 would be better.
;key=36F583DD16F4E1E201EB1E6F6D8E35A2
;iv =20002000200020002000200020002000
;gO4ucnBRzoO2pmqNnHyaBA== -> ; 80 EE 2E 72 70 51 CE 83 B6 A6 6A 8D 9C 7C 9A 04


DataSection
  iv:
  Data.a $20, $00, $20, $00, $20, $00, $20, $00, $20, $00, $20, $00, $20, $00, $20, $00
  
  key:
  Data.a $36, $F5, $83, $DD, $16, $F4, $E1, $E2, $01, $EB, $1E, $6F, $6D, $8E, $35, $A2
  
  input:
  Data.a 'S', 'U', 'P', 'E', 'R', 'B', 'I', 'G', 'S', 'E', 'C', 'R', 'E', 'T', $02, $02
  inputEnd:
EndDataSection


InputLength = ?inputEnd - ?input

*output = AllocateMemory(InputLength)

If AESEncoder(?input, *output, InputLength, ?key, 128, ?iv, #PB_Cipher_CBC)
  ShowMemoryViewer(*output, InputLength)
  
  Debug Base64Encoder(*output, InputLength)
EndIf

[NicTheQuick]

Very strange.

For a test I use directly the iv and the key which is shown by the openssl command.
My data is correctly padded with the pkcs5 method.
I don't know why the output is .. wrong.

Your Purebasic code is right but you missed one thing. The `echo` command also adds a #LF$ add the end.
If you execute an `echo -n` then it outputs the same as Purebasic:

Code: Select all

$ echo -n "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt -p
enter AES-128-CBC encryption password:
Verifying - enter AES-128-CBC encryption password:
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
key=36F583DD16F4E1E201EB1E6F6D8E35A2
iv =20002000200020002000200020002000
h4icFVDNRNzc4wyH7Brmpg==

[Piero]

You ppl desperately need to take a look at the PureMaCrypt0r®©™

It doesn't fear single nor double quotes, its output is compressed, and maybe you can call it obsolete, but at least it's salty…

To resume: #r0o⊺ Mac Rulez! ( )

PS: why -nosalt?

PPS/Note: it's just for laughs

[infratec]

Your Purebasic code is right but you missed one thing. The `echo` command also adds a #LF$ add the end.
If you execute an `echo -n` then it outputs the same as Purebasic:

Yep,

you are right

With

Code: Select all

Data.a 'S', 'U', 'P', 'E', 'R', 'B', 'I', 'G', 'S', 'E', 'C', 'R', 'E', 'T', #LF, $01

I get the same result.

The question is now ...
How I come from 'abc1234' to the key

[infratec]

With:

Code: Select all

Procedure.i DeriveKey(pwd$) ;; openssl style for AES 128 CBC  (uses MD5!)
  
  MD5$ = StringFingerprint(pwd$, #PB_Cipher_MD5)
  *MD5Bytes = bytes(MD5$)
  
  ProcedureReturn *MD5Bytes
  
EndProcedure

I get the original requested VRWqHjP23NtfjqhgWpVsXg== as output

[infratec]

My solution to reproduce the openssl example:

Code: Select all

;; just an educational exercise
;; echo "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt -p -k abc1234


Procedure.i Bytes(asString$)
  
  Protected lengthOfRawBytes.i, *xAsRawBytes, k.i
  
  lengthOfRawBytes = Len(asString$) / 2
  *xAsRawBytes = AllocateMemory(lengthOfRawBytes, #PB_Memory_NoClear)
  
  For k = 0 To lengthOfRawBytes-1
    PokeA(*xAsRawBytes + k, Val("$" + PeekS(@asString$ + (k * 4), 2)))
  Next
  
  ProcedureReturn *xAsRawBytes
  
EndProcedure


Procedure.i DeriveKey(pwd$) ;; openssl style for AES 128 CBC  (uses MD5!)
  
  MD5$ = StringFingerprint(pwd$, #PB_Cipher_MD5)
  *MD5Bytes = bytes(MD5$)
  
  ProcedureReturn *MD5Bytes
  
EndProcedure



UseMD5Fingerprint()


*iv = bytes("20002000200020002000200020002000")            ;; iv must always be 16 bytes


;*test = UTF8("SUPERBIGSECRET" + #LF$)  ;gO4ucnBRzoO2pmqNnHyaBA==   VRWqHjP23NtfjqhgWpVsXg==
;*test = UTF8("SUPERBIGSECRET56" + #LF$)  ;XmQGYcGfqz83HaCN/2jmpvaXN6sd4+nqYvRz8zERp/Y=
;*test = UTF8("SUPERBIGSECRET5" + #LF$)  ;1bH6WQblEeswo1virHW4GySYp99GniZhZeAcn6Nee2w=
;*test = UTF8("SUPERBIGSECRE" + #LF$)  ;rowX4Guqxwa+z4AiWgrH0g==

memsize = MemorySize(*test) - 1
If memsize % 16 <> 0
  *test1 = AllocateMemory(16 * (memsize / 16 + 1))
  FillMemory(*test1 + 16 * (memsize / 16), 16, 16 - memsize % 16)
  CopyMemory(*test, *test1, memsize)
Else
  *test1 = AllocateMemory(memsize + 16)
  FillMemory(*test1 + memsize, 16, $10)
  CopyMemory(*test, *test1, memsize)
EndIf

;*key = DeriveKey("abc1234")
*key = bytes("36F583DD16F4E1E201EB1E6F6D8E35A2")

*test2 = AllocateMemory(MemorySize(*test1), #PB_Memory_NoClear)

If AESEncoder(*test1, *test2, MemorySize(*test1), *key, 128, *iv, #PB_Cipher_CBC)  
  b64$ = Base64Encoder(*test2, MemorySize(*test2))
  Debug b64$   ;;   should be  "VRWqHjP23NtfjqhgWpVsXg==" "gO4ucnBRzoO2pmqNnHyaBA=="
EndIf

Onle the newer key generation is a question.

[Piero]

[NicTheQuick]

The question is now ...
How I come from 'abc1234' to the key

In the case of `-nosalt` and no `-iter` or `-pbkdf2` I think openssl falls back to an iterative MD5 based hash. But I am not totally sure. You may find more about that in its source code:
In any case you should not use that if security is what you have in mind. Better use the proper PBKDF2 algorithm using `DeriveCipherKey()` in Purebasic.

Here's an example using PBKDF2 with 101 iterations:

Code: Select all

$ echo -n "SUPERBIGSECRET" | openssl enc -aes-128-cbc -base64 -iv 20002000200020002000200020002000 -nosalt -pbkdf2 -iter 101 -p
enter AES-128-CBC encryption password:
Verifying - enter AES-128-CBC encryption password:
key=9A538EC69C90A5D1FA7B5404E9F12B4C
iv =20002000200020002000200020002000
I+82eXRpUI9eFSdM7o+35Q==

So the key is: 9A538EC69C90A5D1FA7B5404E9F12B4C
And you can do the same with Purebasic:

Code: Select all

Procedure.s HexMemory(*mem.Ascii, size.i)
	Protected buffer.s = Space(2 * size)
	Protected *c.Character = @buffer
	While size > 0
		PokeS(*c, RSet(Hex(*mem\a, #PB_Ascii), 2, "0"), 2)
		*c + 2 * SizeOf(Character)
		*mem + 1
		size - 1
	Wend
	ProcedureReturn buffer
EndProcedure

passphrase.s = "abc1234"

Define *key = AllocateMemory(16)

UseSHA2Fingerprint()
DeriveCipherKey(passphrase, "", 101, *key, 16 * 8, #PB_Cipher_SHA2)

Debug HexMemory(*key, 16)