SHA256/224 Fingerprint and FileFingerprint

[netmaestro]

Better still, try this one:

Code: Select all

      CompilerIf #PB_Compiler_OS = #PB_OS_Linux
        CallCFunctionFast(*callback, Int(progress) )
      CompilerElse
        CompilerIf #PB_Compiler_OS = #PB_OS_MacOS
          CallCFunctionFast(*callback, Int(progress) )
        CompilerElse 
          CallFunctionFast(*callback, Int(progress) )
        CompilerEndIf
      CompilerEndIf

[idle]

It's fine now I had changes the size of the buffer and I'm more than a bit dim at this time of the day.

[netmaestro]

Removed some unnecessary fat from the code.

Has anyone tried this on MacOS?

[Psychophanta]

Nice one, Netmaestro.
But by the way, can not understand the problem with not including asm in your works. Is it just for future portability?
EDIT: I say it because the 3 current versions of PB (win, mac and linux) are all i386 compatible.

[netmaestro]

Update today, I learned from wilbert a much better way to compute the sigmas and so the code runs a lot faster. It now processes a 50mb file in 5 seconds (down from about 23 when I first posted code)

[netmaestro]

Added commands SHA224Fingerprint() and SHA224FileFingerprint().

That's all the SHAs. No more SHAs to do. Fred did SHA1 natively, this thread and the other one cover 224, 256, 384 and 512. Now I have to work on a database for my wife's seeds.

Until SHA3 comes along, then it's back to hashing!

[netmaestro]

Here's another test, I like it because I like to watch dots go across the screen:

Code: Select all

; ********* Turn the debugger off *******

IncludeFile "sha256.pbi"

test$ = "15a1868c12cc53951e182344277447cd0979536badcc512ad24c67e9b2d4f3dd"
Procedure progress(value)
  Static lastvalue
  If value-lastvalue=2
    Print(".")
    lastvalue=value
  EndIf
EndProcedure

OpenConsole()
EnableGraphicalConsole(1)
ConsoleCursor(0)

*mem = AllocateMemory(536870912)
FillMemory(*mem, 536870912, $5a)

PrintN("")
Print("Working")
t = ElapsedMilliseconds()
result$ = SHA256Fingerprint(*mem, 536870912, @progress())
v = ElapsedMilliseconds()
ConsoleLocate(0,1)
PrintN("Done.                                                              ")
PrintN("")
PrintN(result$)
PrintN("")
If result$ = test$
  PrintN("Passed.")
  PrintN("")
  PrintN("512 megabytes in "+Str((v-t)/1000)+" seconds.")
  PrintN("")
Else
  PrintN("Failed.")
EndIf
ConsoleCursor(1)
Input()

[netmaestro]

Significant speed upgrade today, almost all the sha256_process() procedure is rewritten in asm to use registers instead of variables for the states. Someone better at asm could probably get more speed out of it in a few spots.

[wilbert]

Congratulations Netmaestro.
I read you beat them at speed

[netmaestro]

Congratulations Netmaestro.
I read you beat them at speed

We beat them. I couldn't have done it without your contributions.

[netmaestro]

btw this one looks weak to me but I haven't the skills to improve it:

Code: Select all

  !mov eax, esi
  !mov esi, [p.p_ctx]  
  !add esi, 8
  !movd edx, xmm0
  !add [esi +  0], edx
  !movd edx, xmm1
  !add [esi +  4], edx
  !movd edx, xmm2
  !add [esi +  8], edx
  !movd edx, xmm3
  !add [esi + 12], edx
  !movd edx, xmm4
  !add [esi + 16], edx
  !movd edx, xmm5
  !add [esi + 20], edx
  !movd edx, xmm6
  !add [esi + 24], edx
  !movd edx, xmm7
  !add [esi + 28], edx
  !mov esi, eax

It just adds the 8 SSE registers to their respective states in the *ctx.sha256_context\state variable static array. Surely this could be optimized?

[wilbert]

It looks pretty efficient to me but maybe someone else has a good idea.
Now the 512 version Netmaestro

[wilbert]

If you are still looking for speed improvement, you might want to consider using the mmx registers as well next to the sse registers so you can keep temporary values also in registers instead of memory.
Another thing that can be improved in the process function, is the w.UINT32_BUFFER. When it is declared by PureBasic like this, PureBasic adds code to clear the allocated memory. Usually this is a good thing but you initialize the buffer immediately after it. So filling 64 longs with the value 0 every time the process function is called is just a waste of time.

[netmaestro]

Another thing that can be improved in the process function, is the w.UINT32_BUFFER. When it is declared by PureBasic like this, PureBasic adds code to clear the allocated memory. Usually this is a good thing but you initialize the buffer immediately after it. So filling 64 longs with the value 0 every time the process function is called is just a waste of time

Dammit man you're killing me here. I can't for the life of me figure out where you're storing the 80 64bit words I'd like to emulate what you're doing for the 32bit version but I can't even see what it is.

[wilbert]

Just like PureBasic does, I'm using the stack only not setting everything to 0.
I decreased esp by 656, used the 656 bytes of space that creates and increased esp again by the same amount at the end of the routine.
There is a catch however. Your routine still has local values at the moment. PureBasic handles those as a certain offset from esp so by changing esp yourself, PureBasic may have a problem locating those variables. If you handle all your local variables inside the procedure yourself using ASM, you don't have this problem since you know where they are located. You can simply free the amount of total space (including the temporary variables) yourself from the stack and define the offsets yourself.