sha-3 kessak

[Etayson]

Hello all . I really need help with the module sha-3 kessak.
I took it here: http://forums.purebasic.com/german/view ... =8&t=27209
I'm writing a program for mining Ethereum . But Sha-3 module provides the result is not as it should be.
Can someone suggest how to fix the module to the results were the same ?
Here are quotes from ethash doc (https://github.com/ethereum/wiki/wiki/Ethash)

A note regarding "SHA3" hashes described in this specification
Ethereum's development coincided with the development of the SHA3 standard, and the standards process made a late change in the padding of the finalized hash algorithm, so that Ethereum's "sha3_256" and "sha3_512" hashes are not standard sha3 hashes, but a variant often referred to as "Keccak-256" and "Keccak-512" in other contexts. See discussion, e.g. here, here, or here.

Please keep that in mind as "sha3" hashes are referred to in the description of the algorithm below.

And here is a link to sha-3 kessak used in Ethash: https://github.com/ethereum/ethash/blob ... ash/sha3.c

[wilbert]

If you are using the 64 bit version of PureBasic, try this module with the #ModeKeccak flag.
http://www.purebasic.fr/english/viewtop ... 88#p464988

If it's not producing the results you need, do you have any test hashes to compare ?

[Etayson]

If you are using the 64 bit version of PureBasic, try this module with the #ModeKeccak flag.
http://www.purebasic.fr/english/viewtop ... 88#p464988

If it's not producing the results you need, do you have any test hashes to compare ?

I am use mode kessak.Example, resser=SHA3::SHA3Fingerprint(@resser, Len(resser), 512 | SHA3::#ModeKeccak)
No, i have not a hashes to compare.

But I know that after performing the my procedure "hashimoto" should get the hash.
Here link to subroutines in Python. https://github.com/ethereum/wiki/wiki/Ethash
I did the same on Purebasic, performance results are identical
In general if you use the nonce "83cd23ad29fd1009" I have to get the "result" of "hashimoto" procedure = "0dfb352810119ec0fdf9a2a3c8840490d1ed7b0254b3f7421a557209e5720982"
But i have ="d0f4db1263f879cd6e727defba670d7d6347762c22b49bebc3bc29ad4c663b7f"

I found a link to python version of needed hash function: https://github.com/ethereum/pyethsaleto ... on_sha3.py Maybe this help?

[wilbert]

I am use mode kessak.Example, resser=SHA3::SHA3Fingerprint(@resser, Len(resser), 512 | SHA3::#ModeKeccak)
No, i have not a hashes to compare.

That should work.
I just compared again the results of my module with an online Keccak tool and the results match exactly.

Did you make sure there's no ascii / unicode problem ?
The SHA3Fingerprint procedure takes a buffer as input so it makes a difference if the internal string format is set to ascii or unicode.
If PureBasic is set to unicode mode, you probably need an additional buffer and PokeS to convert the string to ascii before you compute the hash.

[Etayson]

I am use mode kessak.Example, resser=SHA3::SHA3Fingerprint(@resser, Len(resser), 512 | SHA3::#ModeKeccak)
No, i have not a hashes to compare.

That should work.
I just compared again the results of my module with an online Keccak tool and the results match exactly.

Did you make sure there's no ascii / unicode problem ?
The SHA3Fingerprint procedure takes a buffer as input so it makes a difference if the internal string format is set to ascii or unicode.
If PureBasic is set to unicode mode, you probably need an additional buffer and PokeS to convert the string to ascii before you compute the hash.

I am use utf-8. In the All translations of symbols I do yourself , such as the character "a" =97 decimal. Each character takes 1 byte.
I know that your module corresponds to online kessak, but I do not understand what is the difference between your unit and the unit that is used in ethash. I gave the link above to Python version of the desired hash function (https://github.com/ethereum/pyethsaleto ... on_sha3.py).
You may be able to understand the difference. And help me ? Thank you very much for your responsiveness

Keep a link to the part of my program: https://yadi.sk/d/OiqBP3ccsohKG
This code altered with python.
Maybe this will help
There is an archive with the program and your module SHA- 3 and DAG file. It is needed for the program . DAG file is very large 1,5Gb
DAG file generated there is a real miner ethminer, so I'm sure in the correctness of the data Dag file. I hope you can help me

[Keya]

I am use utf-8.

I think wilbert was checking to make sure you've checked compiling with both Ascii and Unicode compile modes, as opposed to using #PB_UTF8 etc for strings! often code will work in one and not the other unless coded to accommodate both for example with CompilerIf statements

[wilbert]

@Etayson,
I never worked with Python so that source is not very helpful.
I did look at the sha3.c version of ethereum and it looks indeed like Keccak.
It should be compatible with my module as far as I can tell.

I also looked your code link with the .dag file inside it.
I have to say, I'm very confused by it. You are mixing 32 bit assembly code into a 64 bit application.
It surprises me it even compiles like this. Did you check your other assembly procedures to verify they work properly ?

[Etayson]

@Etayson,
I never worked with Python so that source is not very helpful.
I did look at the sha3.c version of ethereum and it looks indeed like Keccak.
It should be compatible with my module as far as I can tell.

I also looked your code link with the .dag file inside it.
I have to say, I'm very confused by it. You are mixing 32 bit assembly code into a 64 bit application.
It surprises me it even compiles like this. Did you check your other assembly procedures to verify they work properly ?

And that does not use 32 bit instruction in 64 bit application ?
I have subroutines in Python . From them I did on Purebasic. Identical results . So I can say that the code and paste in the assembly code working properly . It temporary code . Then I change it and optimized . I just need to understand the algorithm and make sure that everything is working correctly.

[wilbert]

And that does not use 32 bit instruction in 64 bit application ?

Instructions that operate on 32 bit values yes, but not for addressing memory.
A procedure like this usually causes problems when compiled on x64

Code: Select all

Procedure ValuePokeL(*a,b)  
  !mov esi,[p.p_a]  
  !mov eax,[p.v_b]
  !Rol ax,8
  !rol eax,16
  !Rol ax,8
  !mov [esi],eax
EndProcedure

You should use 64 bit registers for addressing memory and officially non volatile registers like esi/rsi need to be preserved.
It's possible PB itself uses the esi/rsi register for internal use.
For that reason I would use rdx instead of esi or preserve the register.

Code: Select all

Procedure ValuePokeL(*a, b)  
  !mov rdx, [p.p_a]  
  !mov eax, [p.v_b]
  !bswap eax
  !mov [rdx], eax
EndProcedure

That's why I was surprised the code didn't crash when I compiled it.

As for the whole algorithm, it's hard for me to understand what the Python commands exactly do.
That makes it difficult to compare your code with the Python source and see where the problem is located.
Maybe you could open a new topic or change the topic title to reflect the question is about the Ethash algorithm.
There might be users who have experience with Ethash or interest in it that aren't even reading this post because they have no interest in Keccak.

[Etayson]

And that does not use 32 bit instruction in 64 bit application ?

Instructions that operate on 32 bit values yes, but not for addressing memory.
A procedure like this usually causes problems when compiled on x64

Code: Select all

Procedure ValuePokeL(*a,b)  
  !mov esi,[p.p_a]  
  !mov eax,[p.v_b]
  !Rol ax,8
  !rol eax,16
  !Rol ax,8
  !mov [esi],eax
EndProcedure

You should use 64 bit registers for addressing memory and officially non volatile registers like esi/rsi need to be preserved.
It's possible PB itself uses the esi/rsi register for internal use.
For that reason I would use rdx instead of esi or preserve the register.

Code: Select all

Procedure ValuePokeL(*a, b)  
  !mov rdx, [p.p_a]  
  !mov eax, [p.v_b]
  !bswap eax
  !mov [rdx], eax
EndProcedure

That's why I was surprised the code didn't crash when I compiled it.

As for the whole algorithm, it's hard for me to understand what the Python commands exactly do.
That makes it difficult to compare your code with the Python source and see where the problem is located.
Maybe you could open a new topic or change the topic title to reflect the question is about the Ethash algorithm.
There might be users who have experience with Ethash or interest in it that aren't even reading this post because they have no interest in Keccak.

Thank you very much for taking me to attention and tried to help me .
I will look further . Because it's interesting. And it seems to me that the standard miner poorly optimized, because I get it now same hashrayt and this despite the fact that my code is not optimized

I see you are well rabiraetes in assembler . Can you answer one more question . Why do I not want to work the team push eax, for example. Push ax, Push al operating normally . But the 32 -bit registers of do not want to keep . The compiler generates an error "illegal instruction"

[wilbert]

I hope I understand your english correctly.
If you are wanting to use push and pop in 64 bit mode, you can only use 64 bit registers with push and pop like
push rax
push rsi
etc.

An additional problem is that PureBasic uses the stack offset when referencing procedure arguments.
For example

Code: Select all

!push rsi
!mov esi, [p.p_a]

won't work because after the push command, the stack pointer has changed and the reference to [p.p_a] isn't correct anymore.
So if you only need a few registers, it is recommended to use eax/rax, ecx/rcx or eax/rdx because you don't need to preserve those registers.

If you have an unoptimized version that is working correctly, I'd be happy to look at it to see if speed can be optimized.
That's usually what I like to do

[Etayson]

I hope I understand your english correctly.
If you are wanting to use push and pop in 64 bit mode, you can only use 64 bit registers with push and pop like
push rax
push rsi
etc.

An additional problem is that PureBasic uses the stack offset when referencing procedure arguments.
For example

Code: Select all

!push rsi
!mov esi, [p.p_a]

won't work because after the push command, the stack pointer has changed and the reference to [p.p_a] isn't correct anymore.
So if you only need a few registers, it is recommended to use eax/rax, ecx/rcx or eax/rdx because you don't need to preserve those registers.

If you have an unoptimized version that is working correctly, I'd be happy to look at it to see if speed can be optimized.
That's usually what I like to do

I understand that perfectly . But I do not understand why you can put and take the stack 16 and 8 bit register . But you can not take and put a 32 bit..

Code: Select all

!push ax
!pop ax
!push al
!pop al

work!

Code: Select all

!push eax
!pop eax

illegal instruction!
the difference is only in bit mode register.

[Keya]

On my Win7-64 i can push RAX and AX, but not EAX or AL/AH ... but push AX worked ok for you!?

[Etayson]

On my Win7-64 i can push RAX and AX, but not EAX or AL/AH ... but push AX worked ok for you!?

Yes, push ax works great! I also use win7 x64.
I just do not want to use a 64-bit registers , because writing a universal program for 32 bit and 64 bit machines

[Keya]

apologies, i meant "but push AL works ok for you!?"
is push al even valid on 32/64bit cpu? i cant get it to compile on x86 either, i think its maybe 8/16bit only like DOS apps?