SHA3 Module (Keccak Module)
Re: SHA3 Module (Keccak Module)
Yep, many sorry, i have oversight ...
i have set the module in my Multifunktion cryptographie and steganographie Tool QUICK-AES-256
It works fine !
Many thanks for this great work !
http://www.quick-aes-256.de
i have set the module in my Multifunktion cryptographie and steganographie Tool QUICK-AES-256
It works fine !
Many thanks for this great work !
http://www.quick-aes-256.de
Last edited by walbus on Mon Jan 05, 2015 11:38 pm, edited 9 times in total.
-
- Enthusiast
- Posts: 790
- Joined: Sat Aug 09, 2003 3:13 am
- Location: 90-61-92 // EU or ASIA
- Contact:
Re: SHA3 Module (Keccak Module)
Version: 1.4.7
- optimize the 64 bit code by wilbert
- optimize the 64 bit code by wilbert
-
- Enthusiast
- Posts: 790
- Joined: Sat Aug 09, 2003 3:13 am
- Location: 90-61-92 // EU or ASIA
- Contact:
Re: SHA3 Module (Keccak Module)
Version: 1.4.8
- another update improving speed by wilbert
- another update improving speed by wilbert
Re: SHA3 Module (Keccak Module)
Very nice work
Re: SHA3 Module (Keccak Module)
In time a change was made to the SHA3 draft from the original Keccak version.
To comply to the final FIPS202 you have to make a change to the finalize procedure.
Changeinto
This change should bring the output in line with the most recent test vectors.
For Shake128 and Shake256 the xor value should be %11111
To comply to the final FIPS202 you have to make a change to the finalize procedure.
Change
Code: Select all
*st\a[*ctx\pos] ! 1
Code: Select all
*st\a[*ctx\pos] ! %110
For Shake128 and Shake256 the xor value should be %11111
Last edited by wilbert on Sat Nov 21, 2015 7:29 am, edited 2 times in total.
Windows (x64)
Raspberry Pi OS (Arm64)
Raspberry Pi OS (Arm64)
Re: SHA3 Module (Keccak Module)
I noticed I still had a file I did a long time ago but never posted.
It's an optimized 64 bit SHA3 module in case you don't need 32 bit support.
I updated that also to be compatible to final FIPS202 specification
It's an optimized 64 bit SHA3 module in case you don't need 32 bit support.
I updated that also to be compatible to final FIPS202 specification
Code: Select all
;======================================================================
; Module: SHA3_x64.pbi
;
; Author: Wilbert, Sec
; Date: Nov 21, 2015
; Version: 1.0.3
; Target Compiler: PureBasic 5.20+
; Target OS: All (x64)
; License: Free, unrestricted, no warranty whatsoever
; Use at your own risk
;======================================================================
DeclareModule SHA3
#ModeKeccak = 2048
Structure SHA3Context
state.q[25] ; offset 0
tmpstate.q[25] ; offset 200
pos.a ; offset 400
rsiz.a ; offset 401
hsiz.a ; offset 402
xor_byte.a ; offset 403
EndStructure
;-Exported functions
Declare$ SHA3Fingerprint(*Buffer, Size, Mode=224)
Declare$ SHA3FileFingerprint(Filename$, Mode=224, Offset.q=0, Length.q=0, ProgressBarGadgetNr=-1)
Declare SHA3_init(*ctx.SHA3Context, hsiz.i)
Declare SHA3_update(*ctx.SHA3Context, *Data, len.i)
Declare$ SHA3_final(*ctx.SHA3Context)
EndDeclareModule
Module SHA3
EnableExplicit
Structure WordArray
w.w[0]
EndStructure
Structure AArray
a.a[0]
EndStructure
DataSection
_hex_bytes:
!db '000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f'
!db '202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f'
!db '404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f'
!db '606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f'
!db '808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f'
!db 'a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf'
!db 'c0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedf'
!db 'e0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff'
EndDataSection
Macro MovX(r1, r2) ; move 64 bits between r64 and xmm registers
!movq r1, r2
EndMacro
Macro PrepareC(xmm_reg, x)
!mov r8, [rdx + x * 8]
!mov r9, [rdx + x * 8 + 40]
!mov r10, [rdx + x * 8 + 80]
!mov r11, [rdx + x * 8 + 120]
!mov r12, [rdx + x * 8 + 160]
!xor r8, r9
!xor r8, r10
!xor r8, r11
!xor r8, r12
MovX(xmm_reg, r8)
EndMacro
Macro D(reg, xmm_reg1, xmm_reg2)
MovX(reg, xmm_reg1)
MovX(rax, xmm_reg2)
!rol rax, 1
!xor reg, rax
EndMacro
Macro B(xmm_reg, x, y, d, r)
!mov rax, [rdx + x * 8 + y * 40]
!xor rax, d
CompilerIf r <> 0
!rol rax, r
CompilerEndIf
MovX(xmm_reg, rax)
EndMacro
Macro E_C(xmm_reg, x, y, b0, b1, b2, rnd, rndc = 0)
!movq xmm10, b1
!pandn xmm10, b2
!pxor xmm10, b0
CompilerIf x = 0 And y = 0
!mov rax, rndc
MovX(xmm_reg, rax)
!pxor xmm10, xmm_reg
CompilerEndIf
CompilerIf rnd < 23
CompilerIf y = 0
!movq xmm_reg, xmm10
CompilerElse
!pxor xmm_reg, xmm10
CompilerEndIf
CompilerEndIf
!movq [rcx + x * 8 + y * 40], xmm10
EndMacro
Macro SHA3Round(rnd, rndc)
; Prepare C[.] (round 1 only)
CompilerIf rnd = 0
PrepareC(xmm5, 0)
PrepareC(xmm6, 1)
PrepareC(xmm7, 2)
PrepareC(xmm8, 3)
PrepareC(xmm9, 4)
CompilerEndIf
; D[.]
D( r8, xmm9, xmm6)
D( r9, xmm5, xmm7)
D(r10, xmm6, xmm8)
D(r11, xmm7, xmm9)
D(r12, xmm8, xmm5)
; B[.] (y = 0)
B(xmm0, 0, 0, r8, 0)
B(xmm1, 1, 1, r9, 44)
B(xmm2, 2, 2, r10, 43)
B(xmm3, 3, 3, r11, 21)
B(xmm4, 4, 4, r12, 14)
; E[.,.] and C[.]
E_C(xmm5, 0, 0, xmm0, xmm1, xmm2, rnd, rndc)
E_C(xmm6, 1, 0, xmm1, xmm2, xmm3, rnd)
E_C(xmm7, 2, 0, xmm2, xmm3, xmm4, rnd)
E_C(xmm8, 3, 0, xmm3, xmm4, xmm0, rnd)
E_C(xmm9, 4, 0, xmm4, xmm0, xmm1, rnd)
; B[.] (y = 1)
B(xmm0, 3, 0, r11, 28)
B(xmm1, 4, 1, r12, 20)
B(xmm2, 0, 2, r8, 3)
B(xmm3, 1, 3, r9, 45)
B(xmm4, 2, 4, r10, 61)
; E[.,.] and C[.]
E_C(xmm5, 0, 1, xmm0, xmm1, xmm2, rnd)
E_C(xmm6, 1, 1, xmm1, xmm2, xmm3, rnd)
E_C(xmm7, 2, 1, xmm2, xmm3, xmm4, rnd)
E_C(xmm8, 3, 1, xmm3, xmm4, xmm0, rnd)
E_C(xmm9, 4, 1, xmm4, xmm0, xmm1, rnd)
; B[.] (y = 2)
B(xmm0, 1, 0, r9, 1)
B(xmm1, 2, 1, r10, 6)
B(xmm2, 3, 2, r11, 25)
B(xmm3, 4, 3, r12, 8)
B(xmm4, 0, 4, r8, 18)
; E[.,.] and C[.]
E_C(xmm5, 0, 2, xmm0, xmm1, xmm2, rnd)
E_C(xmm6, 1, 2, xmm1, xmm2, xmm3, rnd)
E_C(xmm7, 2, 2, xmm2, xmm3, xmm4, rnd)
E_C(xmm8, 3, 2, xmm3, xmm4, xmm0, rnd)
E_C(xmm9, 4, 2, xmm4, xmm0, xmm1, rnd)
; B[.] (y = 3)
B(xmm0, 4, 0, r12, 27)
B(xmm1, 0, 1, r8, 36)
B(xmm2, 1, 2, r9, 10)
B(xmm3, 2, 3, r10, 15)
B(xmm4, 3, 4, r11, 56)
; E[.,.] and C[.]
E_C(xmm5, 0, 3, xmm0, xmm1, xmm2, rnd)
E_C(xmm6, 1, 3, xmm1, xmm2, xmm3, rnd)
E_C(xmm7, 2, 3, xmm2, xmm3, xmm4, rnd)
E_C(xmm8, 3, 3, xmm3, xmm4, xmm0, rnd)
E_C(xmm9, 4, 3, xmm4, xmm0, xmm1, rnd)
; B[.] (y = 4)
B(xmm0, 2, 0, r10, 62)
B(xmm1, 3, 1, r11, 55)
B(xmm2, 4, 2, r12, 39)
B(xmm3, 0, 3, r8, 41)
B(xmm4, 1, 4, r9, 2)
; E[.,.] and C[.]
E_C(xmm5, 0, 4, xmm0, xmm1, xmm2, rnd)
E_C(xmm6, 1, 4, xmm1, xmm2, xmm3, rnd)
E_C(xmm7, 2, 4, xmm2, xmm3, xmm4, rnd)
E_C(xmm8, 3, 4, xmm3, xmm4, xmm0, rnd)
E_C(xmm9, 4, 4, xmm4, xmm0, xmm1, rnd)
!xchg rdx, rcx
EndMacro
Procedure SHA3_Process(*ctx.SHA3Context)
!mov rdx, [p.p_ctx]
!lea rcx, [rdx + 200]
; store volatile registers
!sub rsp, 96
!movdqu [rsp], xmm6
!movdqu [rsp + 16], xmm7
!movdqu [rsp + 32], xmm8
!movdqu [rsp + 48], xmm9
!movdqu [rsp + 64], xmm10
!movdqu [rsp + 80], xmm11
!push r12
; process rounds
SHA3Round( 0, 0x0000000000000001)
SHA3Round( 1, 0x0000000000008082)
SHA3Round( 2, 0x800000000000808A)
SHA3Round( 3, 0x8000000080008000)
SHA3Round( 4, 0x000000000000808B)
SHA3Round( 5, 0x0000000080000001)
SHA3Round( 6, 0x8000000080008081)
SHA3Round( 7, 0x8000000000008009)
SHA3Round( 8, 0x000000000000008A)
SHA3Round( 9, 0x0000000000000088)
SHA3Round(10, 0x0000000080008009)
SHA3Round(11, 0x000000008000000A)
SHA3Round(12, 0x000000008000808B)
SHA3Round(13, 0x800000000000008B)
SHA3Round(14, 0x8000000000008089)
SHA3Round(15, 0x8000000000008003)
SHA3Round(16, 0x8000000000008002)
SHA3Round(17, 0x8000000000000080)
SHA3Round(18, 0x000000000000800A)
SHA3Round(19, 0x800000008000000A)
SHA3Round(20, 0x8000000080008081)
SHA3Round(21, 0x8000000000008080)
SHA3Round(22, 0x0000000080000001)
SHA3Round(23, 0x8000000080008008)
; restore volatile registers
!pop r12
!movdqu xmm6, [rsp]
!movdqu xmm7, [rsp + 16]
!movdqu xmm8, [rsp + 32]
!movdqu xmm9, [rsp + 48]
!movdqu xmm10, [rsp + 64]
!movdqu xmm11, [rsp + 80]
!add rsp, 96
EndProcedure
Procedure SHA3_init(*ctx.SHA3Context, hsiz.i)
FillMemory(*ctx, 200)
*ctx\pos = 0
If hsiz & #ModeKeccak
*ctx\xor_byte = 1
Else
*ctx\xor_byte = %110
EndIf
Select hsiz & 1023
Case 512
*ctx\hsiz = 64
*ctx\rsiz = 72
Case 384
*ctx\hsiz = 48
*ctx\rsiz = 104
Case 256
*ctx\hsiz = 32
*ctx\rsiz = 136
Default; 224
*ctx\hsiz = 28
*ctx\rsiz = 144
EndSelect
EndProcedure
Procedure SHA3_update(*ctx.SHA3Context, *Data, len.i)
While len
; absorb data setup
!mov rdx, [p.p_ctx]
!mov r8, [p.v_len]
!movzx r11, byte [rdx + 401] ; *ctx\rsiz
!movzx r10, byte [rdx + 400] ; *ctx\pos
!mov rcx, r11
!sub rcx, r10 ; rcx = number of bytes to absorb
!cmp r8, rcx
!cmovl rcx, r8 ; if len < absorb, absorb = len
!mov r8, [p.p_Data] ; r8 = source
!lea r9, [rdx + r10] ; r9 = destination
!add [p.p_Data], rcx ; *Data + absorb
!sub [p.v_len], rcx ; len - absorb
!add r10, rcx ; *ctx\pos + absorb
; actual xor absorb
!sub rcx, 8
!jc sha3.l_update_l1
!sha3.l_update_l0:
!mov rax, [r8 + rcx]
!xor [r9 + rcx], rax
!sub rcx, 8
!jnc sha3.l_update_l0
!sha3.l_update_l1:
!add rcx, 8
!jz sha3.l_update_l3
!sub rcx, 1
!sha3.l_update_l2:
!mov al, [r8 + rcx]
!xor [r9 + rcx], al
!sub rcx, 1
!jnc sha3.l_update_l2
!sha3.l_update_l3:
; process absorbed data if *ctx\pos = *ctx\rsiz
!xor rax, rax
!cmp r10, r11
!cmove r10, rax
!mov [rdx + 400], r10b
!jne sha3.l_update_l4
SHA3_Process(*ctx)
!sha3.l_update_l4:
Wend
EndProcedure
Procedure$ SHA3_final(*ctx.SHA3Context)
Dim hash.w(*ctx\hsiz)
Define *hex_bytes.WordArray = ?_hex_bytes
Define *st.AArray = *ctx
Define.i i
*st\a[*ctx\pos] ! *ctx\xor_byte
*st\a[*ctx\rsiz - 1] ! $80
SHA3_Process(*ctx)
While i < *ctx\hsiz
hash(i) = *hex_bytes\w[*st\a[i]]
i + 1
Wend
ProcedureReturn PeekS(@hash(), -1, #PB_Ascii)
EndProcedure
;-Exported functions
Procedure$ SHA3FileFingerprint(Filename$, Mode=224, Offset.q=0, Length.q=0, ProgressBarGadgetNr=-1)
#gnChunkSize = 1024 * 32; 32 KiB
Dim DataChunk.q(#gnChunkSize >> 3)
Define hash.s, file.i, error.i, ctx.SHA3Context
Define.q nBytes, nTotalBytes, nTotalBytesRead
file = ReadFile(#PB_Any, Filename$)
If file
nTotalBytes = Lof(file)
If Offset < nTotalBytes
FileSeek(file, Offset)
If Length = 0
Length = nTotalBytes
EndIf
If Offset + Length > nTotalBytes
Length = nTotalBytes - Offset
EndIf
nTotalBytes = Length
SHA3_init(@ctx, Mode)
While Length > 0
nBytes = ReadData(file, @DataChunk(), #gnChunkSize)
If nBytes
If nBytes > Length
nBytes = Length
EndIf
SHA3_update(@ctx, @DataChunk(), nBytes)
Length - nBytes
nTotalBytesRead + nBytes
If ProgressBarGadgetNr >= 0 And IsGadget(ProgressBarGadgetNr)
SetGadgetState(ProgressBarGadgetNr, 100 * nTotalBytesRead / nTotalBytes)
EndIf
Else
error = #True
Break
EndIf
Wend
If error <> #True
hash = SHA3_final(@ctx)
EndIf
EndIf
CloseFile(file)
EndIf
ProcedureReturn hash
EndProcedure
Procedure$ SHA3Fingerprint(*Buffer, Size, Mode=224)
Define ctx.SHA3Context
SHA3_init(@ctx, Mode)
SHA3_update(@ctx, *Buffer, Size)
ProcedureReturn SHA3_final(@ctx)
EndProcedure
EndModule
Last edited by wilbert on Sat Nov 21, 2015 8:30 pm, edited 6 times in total.
Windows (x64)
Raspberry Pi OS (Arm64)
Raspberry Pi OS (Arm64)
Re: SHA3 Module (Keccak Module)
Really nice, thank you!
Re: SHA3 Module (Keccak Module)
ThanksKukulkan wrote:Really nice, thank you!
I updated the post above with a small change.
Code: Select all
Debug SHA3::Fingerprint(@"", 0, 256)
Code: Select all
Debug SHA3::Fingerprint(@"", 0, 256 | SHA3::#ModeKeccak)
It was little effort to support both versions this way.
I'm thinking of adding Shake128 and Shake256 as well.
Those are also in the draft specification and are functions with a variable digest size.
You could get a 32 bit hash but also a 4096 bit hash using those functions.
The only thing is that I don't know how to add them without breaking backward compatibility since an extra parameter would be needed (digest size).
Windows (x64)
Raspberry Pi OS (Arm64)
Raspberry Pi OS (Arm64)
Re: SHA3 Module (Keccak Module)
As of august 5, SHA3 is finalized and finally a standard
http://keccak.noekeon.org/fips202final.html
Now the code in this thread can be verified against the test vectors of the finalized version.
Test vectors of finalized version :
https://github.com/gvanas/KeccakCodePac ... A3-224.txt
https://github.com/gvanas/KeccakCodePac ... A3-256.txt
https://github.com/gvanas/KeccakCodePac ... A3-384.txt
https://github.com/gvanas/KeccakCodePac ... A3-512.txt
https://github.com/gvanas/KeccakCodePac ... AKE128.txt
https://github.com/gvanas/KeccakCodePac ... AKE256.txt
http://keccak.noekeon.org/fips202final.html
Now the code in this thread can be verified against the test vectors of the finalized version.
Test vectors of finalized version :
https://github.com/gvanas/KeccakCodePac ... A3-224.txt
https://github.com/gvanas/KeccakCodePac ... A3-256.txt
https://github.com/gvanas/KeccakCodePac ... A3-384.txt
https://github.com/gvanas/KeccakCodePac ... A3-512.txt
https://github.com/gvanas/KeccakCodePac ... AKE128.txt
https://github.com/gvanas/KeccakCodePac ... AKE256.txt
Windows (x64)
Raspberry Pi OS (Arm64)
Raspberry Pi OS (Arm64)
Re: SHA3 Module (Keccak Module)
hi, when i use:wilbert wrote:ThanksKukulkan wrote:Really nice, thank you!
I updated the post above with a small change.will output the hash from the Draft FIPS202 specification.Code: Select all
Debug SHA3::Fingerprint(@"", 0, 256)
will output the original Keccak hash.Code: Select all
Debug SHA3::Fingerprint(@"", 0, 256 | SHA3::#ModeKeccak)
It was little effort to support both versions this way.
I'm thinking of adding Shake128 and Shake256 as well.
Those are also in the draft specification and are functions with a variable digest size.
You could get a 32 bit hash but also a 4096 bit hash using those functions.
The only thing is that I don't know how to add them without breaking backward compatibility since an extra parameter would be needed (digest size).
IncludeFile "SHA3_x64.pbi"
Debug SHA3::Fingerprint(@"testing", 0, 256 | SHA3::#ModeKeccak)
I got error: Module item'Fingerprint' not found
Why i get this error&
Re: SHA3 Module (Keccak Module)
That's right.
You have to rename the procedures and the declares from
SHA3Fingerprint
to
Fingerprint
Bernd
You have to rename the procedures and the declares from
SHA3Fingerprint
to
Fingerprint
Bernd
Re: SHA3 Module (Keccak Module)
Wilberts codes are "Arts of work" !
Wilbert is NOT a babbler, you have a problem and ask friendly, wilbert helps !
For the SHA3 Module,
rename simple inside the module with the PB rename function, as sample so :
SHA3::Fingerprint_(*Buffer, Size, 256)
Wilbert is NOT a babbler, you have a problem and ask friendly, wilbert helps !
For the SHA3 Module,
rename simple inside the module with the PB rename function, as sample so :
SHA3::Fingerprint_(*Buffer, Size, 256)
- RichAlgeni
- Addict
- Posts: 914
- Joined: Wed Sep 22, 2010 1:50 am
- Location: Bradenton, FL
Re: SHA3 Module (Keccak Module)
Here, here!!walbus wrote:Wilberts codes are "Arts of work" !
Wilbert is NOT a babbler, you have a problem and ask friendly, wilbert helps !
For the SHA3 Module,
rename simple inside the module with the PB rename function, as sample so :
SHA3::Fingerprint_(*Buffer, Size, 256)
Re: SHA3 Module (Keccak Module)
Hello all . I really need help with the module sha-3 kessak. I'm writing a program for mining Ethereum . But Sha-3 module provides the result is not as it should be.
Can someone suggest how to fix the module to the results were the same ?
Here are quotes from ethash doc (https://github.com/ethereum/wiki/wiki/Ethash)
A note regarding "SHA3" hashes described in this specification
Ethereum's development coincided with the development of the SHA3 standard, and the standards process made a late change in the padding of the finalized hash algorithm, so that Ethereum's "sha3_256" and "sha3_512" hashes are not standard sha3 hashes, but a variant often referred to as "Keccak-256" and "Keccak-512" in other contexts. See discussion, e.g. here, here, or here.
Please keep that in mind as "sha3" hashes are referred to in the description of the algorithm below.
And here is a link to sha-3 kessak used in Ethash: https://github.com/ethereum/ethash/blob ... ash/sha3.c
Can someone suggest how to fix the module to the results were the same ?
Here are quotes from ethash doc (https://github.com/ethereum/wiki/wiki/Ethash)
A note regarding "SHA3" hashes described in this specification
Ethereum's development coincided with the development of the SHA3 standard, and the standards process made a late change in the padding of the finalized hash algorithm, so that Ethereum's "sha3_256" and "sha3_512" hashes are not standard sha3 hashes, but a variant often referred to as "Keccak-256" and "Keccak-512" in other contexts. See discussion, e.g. here, here, or here.
Please keep that in mind as "sha3" hashes are referred to in the description of the algorithm below.
And here is a link to sha-3 kessak used in Ethash: https://github.com/ethereum/ethash/blob ... ash/sha3.c