SHA256/224 Fingerprint and FileFingerprint

[netmaestro]

Ok, that's above my pay grade. After I've taken further steps in assembly programming I will likely revisit this approach but for now I don't understand it well enough to try it. Nice to see what a skilled asm coder can do though, it's impressive. Something to aspire to

[netmaestro]

Actually I think I could overcome the inefficiency of allocating and zeroing the memory at each call to the process procedure (and for a large file there will be many many) by storing the words in a global variable that gets initialized once when the program is called. But [p.v_var] inside a procedure doesn't know about the global. There must be a way to address it from inside a procedure, do you know how?

[wilbert]

I don't know how a global storage would affect thread safety; if there would be the possibility of problems when two threads would try to computer a hash.
You can use another workaround but I don't know if it makes a difference in speed. You would just have to try.
You can change the structure of the context like this

Code: Select all

Structure sha256_context
  total.q
  state.l[8]
  buffer.a[64]
  w.l[64]
EndStructure

To add the storage to the context.
Inside the process function, you only set a pointer

Code: Select all

*w.UINT32_BUFFER = @*ctx\w

and you do a search and replace to replace every w\w with *w\w .

[netmaestro]

On 7th read, your explanation of how your code works starts to filter through. I'm going to give it a try after all and see if I can't make it work. Failing that, of course you are right that the context is the place to store it. I'll let you know how I fare.

[netmaestro]

Today's update brings another significant increase in speed. I removed all temporary variables and now the values are stored/retrieved from registers. Also, the 256byte memory area for the temporary dword storage is changed from a local structured variable to a memory block in the context. Apart from the macro calls and one ProcedureReturn, no Purebasic code exists in the critical Process() function anymore. It is now 100% asm code with all the optimization muscle that 5 days' experience with asm coding can have, and it is now much faster than the hashslash program I've been testing it against. Last but not least, for processing a 50mb file it is only losing to the native Sha1FileFingerprint by a handful of milliseconds. Now to beat Fred...

[wilbert]

That's some serious ASM coding Netmaestro.
If you learned all of that in 5 days, that's great
Are you trying to be faster as the PB SHA1 with this code or do you have your own SHA-1 routine also ?

[netmaestro]

No this project and the sha512 version are my first foray into secure hashing. It has been a lot of fun learning from you and the others and watching the performance go up every day. I still don't understand a lot of what you've done with the 64bit side, but I'll keep trying and someday I hope to code like that. This assembler coding is a real blast!

[netmaestro]

oops the code I posted at 11 this morning had a bug. It was missing a crucial !ret at the bottom of the prototyped process function. Posted code is fixed now, please recopy if you're using this.

[Thorium]

I still don't understand a lot of what you've done with the 64bit side, but I'll keep trying and someday I hope to code like that. This assembler coding is a real blast!

Great to hear you have a blast with it. ASM isnt that hard to learn, just keep going and that day you understand will be next week. ^^

I dont know how you get infos about the instructions, but a common mistake by beginners is to read some stuff they found on google and dont take a look at the official reference manuals, which are freely available.
So here are the links, in case you dont have them allready, they are all PDF's:

Intel® 64 and IA-32 Architectures Software Developer’s Manual 1
This is the most important one, it covers all the basics about the CPU and ASM. Explaining addressing, the registers, status flags, SIMD (MMX, SEE etc.). Basicly all you want to know as a ASM coder.

Intel® 64 and IA-32 Architectures Software Developer’s Manual 2A
Intel® 64 and IA-32 Architectures Software Developer’s Manual 2B
This two are a complete instruction set reference.

There is actualy a 3rd manual. It contains information about system programing and some deeper stuff. So if you want to make your own OS, you will need that too. ^^

Intel® 64 and IA-32 Architectures Optimization Reference Manual
This one is a very interessting read. It contains explanations about how the CPU optimizes execution and most importantly how you can write your code that this optimizations work best. For example branch prediction and prefetching. It also contains a lot of advices for best performing code and some optimal reference implementations of algos like alpha blending 2 images with SIMD.

AMD also has reference manuals you will need if you want to use AMD features like 3DNow! Or want to optimize for AMD CPU's.

[electrochrisso]

This might come in handy, but you probably already been here NM.
http://www.agner.org/optimize/

[netmaestro]

No, I hadn't seen that, thanks for the link! It looks like there's lots of instructive material there. I appreciate it very much

[netmaestro]

The remarkably long string I was using in a datasection is replaced with a better approach as taught to me by wilbert. Code is updated.

[netmaestro]

One more update today, I finally sorted out how to make the increasing-the-stacksize approach for allocating memory work. The words block is no longer part of the context and all now-unused structures are removed. Latest code is posted.

[electrochrisso]

No worries NM, I thought I would refresh myself on ASM and I couldnt believe the amount of documentation out there.
And thanks for the code post too, very nice.

[netmaestro]

@thorium, thanks for the links! Those manuals have proven invaluable in the short time I've used them. I appreciate your help very much