Comparisons in mathematical simulations

[Piero]

Answering to Olli and Fvillanova:
I'm a Supernoob of Purebasic, but I see you are very, very helpful with one another in this forum; so I already feel like I owe you at least some interesting stuff (like https://en.wikipedia.org/wiki/Monty_Hall_problem)
Thank you!

[fvillanova]

The code that processes comparisons with 100 items in each sample is a little slower, it needs to do the comparison twice.
I'm using Wilbert's code twice:

Code: Select all

 FillMemory(@conte(), (mate(i0,14)+1)*8, 0, #PB_Integer)                   
 i6=mate(i0,10): kk2=mate(i0,11)-i6+1                         
 !mov r9, [v_i6]                ; r9 = @matf() + i6 * 8    
 !mov r8, r9                    ; r8 = @matf2() + i6 * 8 
 !shl r9, 3                       
 !add r9, [a_matf]                
 !shl r8, 3 
 !add r8, [a_matf2] 
 !mov r10, [a_conte]            ; r10 = @conte()     
 !mov rcx, [v_kk2]              ; rcx = kk2   
 !l_jump412:                      
 !mov rax, [v_i4q]              ; First block of (0 and 1) with 50 bytes in Quad variable - will be compared with matf()      
 !and rax, [r9]                   
 !popcnt rax, rax                 
 !add qword [r10 + rax * 8], 1  ; array conte() sum the totalcount
 !mov rax, [v_i5q]              ; Second block of (0 and 1) with 50 bytes in Quad variable - will be compared with matf2()
 !and rax, [r8] 
 !popcnt rax, rax 
 !add qword [r10 + rax * 8], 1  ; array conte() sum the totalcount
 !add r9, 8                       
 !add r8, 8 
 !sub rcx, 1                      
 !jnz l_jump412   

The question is: Did I correctly duplicate Wilbert's code?
It's working perfectly, but could it be faster?

[wilbert]

The question is: Did I correctly duplicate Wilbert's code?
It's working perfectly, but could it be faster?

I think it can be done faster but I'm confused by your code.
Initially when you asked for a popcount that could count more than 64 bits, you were adding the popcount results together but you aren't doing that anymore which is something totally different.

If the popcount value of (matf() & i4q) = 30 and the popcount value of (matf2() & i5q) = 48, in your original approach conte(78) would be increased by one, with your current approach conte(30) and conte (48) would both be increased by one.
Assume for the next item the popcount results would be 30 and 8.
In your original approach after the two items, conte(38) and conte(78) would have been increased by one, with your current approach conte(30) would have been increased by two and conte(8) and conte(48) increased by one.
That's something completely different.

[fvillanova]

I think it can be done faster but I'm confused by your code.
Initially when you asked for a popcount that could count more than 64 bits, you were adding the popcount results together but you aren't doing that anymore which is something totally different.

Hi Wilbert,
Everything is great!
Program processing fast and problems solved.
Routines with 64 and 100 byte strings are being used.
I wanted to show how the duplication of one of your routines was done by me.
I have no experience in ASM, I'm just now starting to take the first steps.
My English also makes it difficult to understand some expressions, sorry.
I just have to thank you for all the help you've given me over the years.
I'm doing a system check to speed it up with little details that when processed trillions of times any change to faster makes a big difference like this one:

Code: Select all

DisableDebugger
Dim cont.i(1)
Define.i i,j

t1.i = ElapsedMilliseconds()
!mov rax,1
!mov r10, [a_cont]
!mov rcx,100000000
!l_again:
!add [r10 + 8], rax
!sub rcx, 1                                        
!jnz l_again
Result$+"( !add [r10 + 8], rax ) in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

cont(1)=0
t1.i = ElapsedMilliseconds()
!mov r10, [a_cont]
!mov rcx,100000000
!l_again1:
!add qword [r10 + 8], 1
!sub rcx, 1                                        
!jnz l_again1
Result$+"( !add qword [r10 + 8], 1 ) in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

EnableDebugger
Debug Result$

smallest details ... big differences !

[wilbert]

I'm doing a system check to speed it up with little details that when processed trillions of times any change to faster makes a big difference like this one:

On my computer there's no difference between the two.
Which version is faster on your computer ?

[fvillanova]

On my computer there's no difference between the two.
Which version is faster on your computer ?

on my Dell i7 right now...
I abandoned version 6.02 and went back to 5.73 because I need the ExcelWriter routines working perfectly!!!

[wilbert]

on my Dell i7 right now...

That's a huge difference in speed on your computer !

[fvillanova]

Today I started to accelerate an old procedure.
It's my first steps in ASM, don't laugh at my source.
It's already 100% faster, with the help of others I think it will get even faster.
The objective is to receive from the procedure a numerical string in ascending order informing the number of equal occurrences of a vector ( vet1() or vet2() ) with an informed String containing numbers separated by spaces from 00 to 99
Does anyone have any ideas or tips?
I converted more parts of the original procedure from 2011 but they were slower than the original commands such as:

Code: Select all

If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf
aux="": For i=1 To j: aux+Str(dr(10+i)): Next  

so I left it as the original.
The old code from 2011 and today are here:

Code: Select all

DisableDebugger
Global Dim vet1.i(100): Global Dim vet2.i(100)
Define.i i,j
Define.s a1,a2
RandomSeed(0)
For i=0 To 100: vet1(i)=Random(10,1): vet2(i)=Random(10,1): Next    

Procedure.s Year2011(*String1,qt.i,flag.i)   ; String1 from 00 to 99 with spaces | qt = 2 to 50 = len(String1+1)/3  | flag= 1 in vet1() or 0 in vet2()
  Protected Dim dr.i(21)
  Protected.i i,j
  Protected.s aux
  For i=1 To qt
    aux=PeekS(*String1+(i-1)*6,2): If flag: dr(vet1(Val(aux)))+1: Else: dr(vet2(Val(aux)))+1: EndIf 
  Next  
  For i=1 To 10: If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf: Next
  SortArray(dr(),#PB_Sort_Ascending,11,10+j)
  aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux
EndProcedure  
Structure StringElement
  StructureUnion
    c.c[2]
    n.l 
  EndStructureUnion
  spacing.c 
EndStructure
Procedure.s Today(*cval.StringElement,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1

i=0
!mov r13, 10 
!l_loop2:
!add qword [p.v_i], 1  
    If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf
!sub r13, 1
!jnz l_loop2

SortArray(dr(),#PB_Sort_Ascending,11,10+j)

aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux      
EndProcedure

String1.s="15 12 18 87 95 10 13 21 14 88 80 77 51 53 43 26 81 00 11"

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again:

a1=Year2011(@String1,18,1)
a2=Year2011(@String1,18,0)

!sub r15, 1                                        
!jnz l_again
Result$+"( In 2011 ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again1:

a1=Today(@String1,18,1)
a2=Today(@String1,18,0)

!sub r15, 1                                        
!jnz l_again1
Result$+"( Today ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

EnableDebugger
Debug Result$

Any help will be welcome ...

[wilbert]

The objective is to receive from the procedure a numerical string in ascending order informing the number of equal occurrences of a vector ( vet1() or vet2() ) with an informed String containing numbers separated by spaces from 00 to 99
Does anyone have any ideas or tips?

At the moment you are calling your procedure twice, once with flag=0 and once with flag=1.
Do you always need both ?
If so, it is faster to process both at once instead of in two steps.

Your demo code outputs strings like 11112237.
Is it possible values above 9 are reached ?
Like 1112210 (1-1-1-2-2-10)

[fvillanova]

At the moment you are calling your procedure twice, once with flag=0 and once with flag=1.
Do you always need both ?
If so, it is faster to process both at once instead of in two steps.

Yes, if always twice, the fastest would be to do it once, but no, the procedure is usually called once, depending on the analysis the program is doing it uses only one of the arrays vet1() or vet2(),I put it twice just to show in the example.

Your demo code outputs strings like 11112237.
Is it possible values above 9 are reached ?
Like 1112210 (1-1-1-2-2-10)

Yes, it can happen at 10x , and the answer in my procedure would be: 1112210
The "10" will always come at the end of the response string, it is possible to even have the following: 112331010
two or more times the "10"
It doesn't matter if the answer is going to be 1 to 10 or 0 to 9, it could be AAABBCCCJJ where A=1, B=2, C=3,J=10 in reality the program uses the answer as an occurrence key.

[wilbert]

String building is a bit slow in PB.
If it really is okay for you to represent values above 9 with a single unicode character instead of two digits, it can improve the speed a lot.

[fvillanova]

String building is a bit slow in PB.
If it really is okay for you to represent values above 9 with a single unicode character instead of two digits, it can improve the speed a lot.

Yes, the return string can be from 0 to 9 without problem, the maximum number of occurrences is 10, the result below is valid with 0 to 9:
(new version) 0022300 = 113341010 (current version) both are unique and the same key.
I believe that the sort delays a lot, doesn't it?

[wilbert]

the maximum number of occurrences is 10

Can you explain why it can't be bigger than 10 ?
It looks to me if all 100 items of the vet1 or vet2 array would be equal, it would be > 10.
If you are absolutely sure of that limit of 10, you can count how many each value from 1 to 10 occurs and use those counts to output the result.
For example if dr(1 to 10) contains the values 1-4-3-1-2-0-1-1-3-2, counting will result in 4x1, 2x2, 2x3 and 1x4.
Those counts can then be used to output the result 111122334 (or 000011223 if you use 0-9 characters).
That will be faster compared to sorting.

Another question ...
Would it be okay for you to use fixed strings and call the procedure in a different way ?
Define.s{10} a1,a2 instead of Define.s a1,a2
and call by
Today(@String1,@a1,18,1) instead of a1=Today(@String1,18,1)
If that would be no problem to you, it can be made extremely fast.

[fvillanova]

Can you explain why it can't be bigger than 10 ?
It looks to me if all 100 items of the vet1 or vet2 array would be equal, it would be > 10.
If you are absolutely sure of that limit of 10, you can count how many each value from 1 to 10 occurs and use those counts to output the result.

Yes, you really understood my procedure.
I generated vet1() and vet2() randomly in the example allowing more than "'10" occurrences to exist.
But this was the way I showed the routine working only, the generation of vet1(1) and vet2() is controlled and does not allow more than 10 equal occurrences, they represent samplings of statistical tables.
This created key "111122334" or "000011223" will enter a "hashtable" like this:

Code: Select all

Global NewMap Example.i()
...
a1=Today(@String1,18,1): Example(a1)+1
...

A lot of information is consolidated using the created key.

For example if dr(1 to 10) contains the values 1-4-3-1-2-0-1-1-3-2, counting will result in 4x1, 2x2, 2x3 and 1x4.
Those counts can then be used to output the result 111122334 (or 000011223 if you use 0-9 characters).
That will be faster compared to sorting.

At this point you gave me the idea of not using the SortArray command and doing a different count, see:

Code: Select all

i=0
!mov r13, 10 
!l_loop2a:
!add qword [p.v_i], 1  
     dr(11+dr(i))+1: If dr(i)>j: j=dr(i): EndIf 
!sub r13, 1
!jnz l_loop2a

aux="": For i=1 To j : aux+Str(dr(11+i)): Next  
  ProcedureReturn LSet(aux,10,"z")   

The generated key remains unique and works for my system.
The "z" can be any character ... "0" or "x" .... but they need to exist to keep the answer key 10 characters long.
With this idea I doubled the speed again, see how the processing times are now:
( In 2011 ) result: 122222223 and 111122334 in 210 ms
( Today ) result: 122222223 and 111122334 in 102 ms
( New idea ) result: 171zzzzzzz and 4221zzzzzz in 48 ms

"171zzzzzzz" it is the same as "122222223" and "4221zzzzzz" it is the same as "111122334"

see the procedure test code:

Code: Select all

DisableDebugger
Global Dim vet1.i(100): Global Dim vet2.i(100)
Define.i i,j
Define.s a1,a2
RandomSeed(0)
For i=0 To 100: vet1(i)=Random(10,1): vet2(i)=Random(10,1): Next    

Procedure.s Year2011(*String1,qt.i,flag.i)   ; String1 from 00 to 99 with spaces | qt = 2 to 50 = len(String1+1)/3  | flag= 1 in vet1() or 0 in vet2()
  Protected Dim dr.i(21)
  Protected.i i,j
  Protected.s aux
  For i=1 To qt
    aux=PeekS(*String1+(i-1)*6,2): If flag: dr(vet1(Val(aux)))+1: Else: dr(vet2(Val(aux)))+1: EndIf 
  Next  
  For i=1 To 10: If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf: Next
  SortArray(dr(),#PB_Sort_Ascending,11,10+j)
  aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux
EndProcedure  
Structure StringElement
  StructureUnion
    c.c[2]
    n.l 
  EndStructureUnion
  spacing.c 
EndStructure
Procedure.s Today(*cval.StringElement,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1

i=0
!mov r13, 10 
!l_loop2:
!add qword [p.v_i], 1  
    If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf
!sub r13, 1
!jnz l_loop2

SortArray(dr(),#PB_Sort_Ascending,11,10+j)

aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux      
EndProcedure
Procedure.s NewIdea(*cval.StringElement,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1a:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1a

i=0
!mov r13, 10 
!l_loop2a:
!add qword [p.v_i], 1  
     dr(11+dr(i))+1: If dr(i)>j: j=dr(i): EndIf 
!sub r13, 1
!jnz l_loop2a

aux="": For i=1 To j : aux+Str(dr(11+i)): Next  
  ProcedureReturn LSet(aux,10,"z")      
EndProcedure

String1.s="15 12 18 87 95 10 13 21 14 88 80 77 51 53 43 26 81 00 11"

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again:

a1=Year2011(@String1,18,1)
a2=Year2011(@String1,18,0)

!sub r15, 1                                        
!jnz l_again
Result$+"( In 2011 ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again1:

a1=Today(@String1,18,1)
a2=Today(@String1,18,0)

!sub r15, 1                                        
!jnz l_again1
Result$+"( Today ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again2:

a1=NewIdea(@String1,18,1)
a2=NewIdea(@String1,18,0)

!sub r15, 1                                        
!jnz l_again2
Result$+"( New idea ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

EnableDebugger
Debug Result$

[fvillanova]

Another question ...
Would it be okay for you to use fixed strings and call the procedure in a different way ?
Define.s{10} a1,a2 instead of Define.s a1,a2
and call by
Today(@String1,@a1,18,1) instead of a1=Today(@String1,18,1)
If that would be no problem to you, it can be made extremely fast.

New idea based on your idea:

Code: Select all

DisableDebugger
Global Dim vet1.i(100): Global Dim vet2.i(100)
Define.i i,j
Define.s a1,a2
RandomSeed(0)
For i=0 To 100: vet1(i)=Random(10,1): vet2(i)=Random(10,1): Next    

Procedure.s Year2011(*String1,qt.i,flag.i)   ; String1 from 00 to 99 with spaces | qt = 2 to 50 = len(String1+1)/3  | flag= 1 in vet1() or 0 in vet2()
  Protected Dim dr.i(21)
  Protected.i i,j
  Protected.s aux
  For i=1 To qt
    aux=PeekS(*String1+(i-1)*6,2): If flag: dr(vet1(Val(aux)))+1: Else: dr(vet2(Val(aux)))+1: EndIf 
  Next  
  For i=1 To 10: If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf: Next
  SortArray(dr(),#PB_Sort_Ascending,11,10+j)
  aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux
EndProcedure  
Structure StringElement
  StructureUnion
    c.c[2]
    n.l 
  EndStructureUnion
  spacing.c 
EndStructure
Procedure.s Today(*cval.StringElement,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1

i=0
!mov r13, 10 
!l_loop2:
!add qword [p.v_i], 1  
    If dr(i)<>0: dr(11+j)=dr(i):j+1: EndIf
!sub r13, 1
!jnz l_loop2

SortArray(dr(),#PB_Sort_Ascending,11,10+j)

aux="": For i=1 To j: aux+Str(dr(10+i)): Next  
  ProcedureReturn aux      
EndProcedure
Procedure.s NewIdea(*cval.StringElement,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1a:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1a

i=0
!mov r13, 10 
!l_loop2a:
!add qword [p.v_i], 1  
     dr(11+dr(i))+1: If dr(i)>j: j=dr(i): EndIf 
!sub r13, 1
!jnz l_loop2a

aux="": For i=1 To j : aux+Str(dr(11+i)): Next  
  ProcedureReturn LSet(aux,10,"z")      
EndProcedure
Procedure.s Wilbert(*cval.StringElement,*where,qt.i,i2.i) 
  Protected Dim dr.i(21)
  Protected.i i,j,n
  Protected.s aux  
!mov r11, [p.a_dr] 
!mov r8, [a_vet1]   
!mov r9, [a_vet2] 
!mov r13, [p.v_qt]  
!l_loop1w:
   n = (*cval\c[0] * 10 + *cval\c[1] - 528)*8
!mov r10,[p.v_n]
    If i2
!mov r14, [r8 + r10]      
!add qword [r11 + r14 * 8], 1      
    Else
!mov r14, [r9 + r10]         
!add qword [r11 + r14 * 8], 1        
    EndIf
    *cval + 6    
!sub r13, 1
!jnz l_loop1w

i=0
!mov r13, 10 
!l_loop2w:
!add qword [p.v_i], 1  
     dr(11+dr(i))+1: If dr(i)>j: j=dr(i): EndIf 
!sub r13, 1
!jnz l_loop2w

  For i=1 To j: PokeS(@where+(i-1)*2,Str(dr(11+i)),1): Next  

  ProcedureReturn    
EndProcedure
String1.s="15 12 18 87 95 10 13 21 14 88 80 77 51 53 43 26 81 00 11"

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again:

a1=Year2011(@String1,18,1)
a2=Year2011(@String1,18,0)

!sub r15, 1                                        
!jnz l_again
Result$+"( In 2011 ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again1:

a1=Today(@String1,18,1)
a2=Today(@String1,18,0)

!sub r15, 1                                        
!jnz l_again1
Result$+"( Today ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again2:

a1=NewIdea(@String1,18,1)
a2=NewIdea(@String1,18,0)

!sub r15, 1                                        
!jnz l_again2
Result$+"( New idea ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

t1.i = ElapsedMilliseconds()
!mov r15, 50000
!l_again3:

Wilbert(@String1,@a1,18,1)
Wilbert(@String1,@a2,18,0)

!sub r15, 1                                        
!jnz l_again3
Result$+"( Wilbert ) result: "+a1+" and "+a2+" in "+StrF((ElapsedMilliseconds()-t1),0)+" ms"+Chr(13)

EnableDebugger
Debug Result$

( In 2011 ) result: 122222223 and 111122334 in 226 ms
( Today ) result: 122222223 and 111122334 in 107 ms
( New idea ) result: 171zzzzzzz and 4221zzzzzz in 50 ms
( Wilbert ) result: 171zzzzzzz and 4221zzzzzz in 12 ms
wow!