Page 3 of 3

Re: Sorting option(quick/merge) for arrays.

Posted: Tue Mar 06, 2012 2:43 am
by skywalk
Yeah, I generally don't use lists or maps except for small data sizes.
I really prefer arrays for speed.
But, as you see here, I have to morph the sorting procedures because of the instability of QuickSort, and that eats into my speed. For structured arrays, I really wish it was MergeSort or an option.
No matter what, this forum is awesome. Thanks Trond and Demivec.

Re: Sorting option(quick/merge) for arrays.

Posted: Thu Mar 08, 2012 6:01 am
by skywalk
Hi Trond,
While checking Demivec's code, I found that your SubSortStructuredArray() Macro, while elegantly implemented, does not support subsequent calls to sort additional Fields and still maintain the proper hierarchical order. Demivec's Multi-Field-SubSort is more thorough in this regard.
So, I sort once using SortStructuredArray(), then use your SubSortStructuredArray() Macro on a secondary field. If I need to do a 3rd subSort then I have to expand the approach to Demivec's.

The hassle of structure specific Macros and Procedures to accomplish a stable sort is super frustrating given the stability is already coded for SortStructuredList(). Why hinder structured arrays in this regard :?:

I agree QuickSort makes sense for non-structured arrays, but seriously reduces the efficiency at least in my application. If I have 5 or more fields, then range sorting arrays to preserve duplicate locations winds up being slower than going to Lists. :(

Re: Sorting option(quick/merge) for arrays.

Posted: Thu Mar 08, 2012 5:06 pm
by Trond
skywalk wrote:Hi Trond,
While checking Demivec's code, I found that your SubSortStructuredArray() Macro, while elegantly implemented, does not support subsequent calls to sort additional Fields and still maintain the proper hierarchical order.
Can you post a code that shows this problem? It seems to work with my test code: http://pastebin.ca/2126177

Re: Sorting option(quick/merge) for arrays.

Posted: Thu Mar 08, 2012 5:17 pm
by skywalk

Code: Select all

Macro SubSortStructuredArray(Arr, MainField, SubOptions, SubOffset, SubType)
  Define SubSort_Start, SubSort_I
  SubSort_Start = 0
  For SubSort_I = 1 To ArraySize(Arr())
    If Arr(SubSort_Start)\MainField <> Arr(SubSort_I)\MainField
      If SubSort_Start - (SubSort_I - 1) <> 0 ; Performance optimization
        ; Debug Str(SubSort_Start) + ", " + Str(subsort_i-1)
        SortStructuredArray(Arr(), SubOptions, SubOffset, SubType, SubSort_Start, SubSort_I-1)
      EndIf
      SubSort_Start = SubSort_I
    EndIf
  Next
  ; Debug Str(SubSort_Start) + ", " + Str(subsort_i-1)
  SortStructuredArray(Arr(), SubOptions, SubOffset, SubType, SubSort_Start, SubSort_I-1)
EndMacro
;-{ TEST
Macro DebugIt(arStruc, nPts, hdr="", tw=4)
  Debug hdr
  Debug LSet("--", tw) + LSet("--", tw) + LSet("--", tw)
  For i = 0 To nPts-1
    Debug LSet(Str(arstruc(i)\a), tw) + LSet(arstruc(i)\b$, tw) + LSet(arstruc(i)\c$, tw)
  Next
EndMacro
Structure myABC
  a.i
  b$
  c$
EndStructure
Define.i i, nPts, tw = 4
Define.s r$
Restore SortThis
Read.i nPts
Dim myL.myABC(nPts-1)
For i = 0 To nPts-1
  Read.s r$: myL(i)\a = Val(r$)
  Read.s myL(i)\b$
  Read.s myL(i)\c$
Next
Debug "-- Before Sort --"
DebugIt(myL, nPts, LSet("a", tw) + LSet("b$", tw) + LSet("c$", tw))
; Primary sort on a
SortStructuredArray (myL(),    #PB_Sort_Ascending,                   OffsetOf(myABC\a),  #PB_Sort_Integer)
; 2nd sort on b$
SubSortStructuredArray(myL, a,   #PB_Sort_Ascending | #PB_Sort_NoCase, OffsetOf(myABC\b$), #PB_Sort_String)
Debug "-- OK After SortBy a+,b$+ --"
DebugIt(myL, nPts, LSet("a", tw) + LSet("b$", tw) + LSet("c$", tw))

Debug "-- FAIL After SortBy a+,b$+,c$+ --"
; Primary sort on a
SortStructuredArray (myL(),    #PB_Sort_Ascending,                   OffsetOf(myABC\a),  #PB_Sort_Integer)
; 2nd sort on b$
SubSortStructuredArray(myL, a,   #PB_Sort_Ascending | #PB_Sort_NoCase, OffsetOf(myABC\b$), #PB_Sort_String)
; 3rd sort on c$ FAILS since it does not reference Field a
SubSortStructuredArray(myL, b$,  #PB_Sort_Ascending | #PB_Sort_NoCase, OffsetOf(myABC\c$), #PB_Sort_String)
DebugIt(myL, nPts, LSet("a", tw) + LSet("b$", tw) + LSet("c$", tw))

DataSection
  SortThis:
  Data.i 12
  ;       a,  b$,  c$
  Data.s "1", "9", "1"
  Data.s "2", "2", "5"
  Data.s "3", "1", "b"
  Data.s "4", "1", "a"
  Data.s "5", "3", "z"
  Data.s "7", "3", "y"
  Data.s "6", "3", "3"
  Data.s "8", "3", "2"
  Data.s "9", "3", "x"
  Data.s "7", "5", "a"  ;<- The problem of a 3rd sort appears when both Field1 and Field2 are duplicates
  Data.s "7", "5", "w"  ;<- Field1 = primary, Field2 = secondary, etc.
  Data.s "7", "4", "x"
  IWantThis:
  Data.i 12
  ;       a,  b$,  c$
  Data.s "1", "9", "1"
  Data.s "2", "2", "5"
  Data.s "3", "1", "b"
  Data.s "4", "1", "a"
  Data.s "5", "3", "z"
  Data.s "6", "3", "3"
  Data.s "7", "3", "y"
  Data.s "7", "4", "x"
  Data.s "7", "5", "a"   ;<- c$ = a then w
  Data.s "7", "5", "w"
  Data.s "8", "3", "2"
  Data.s "9", "3", "x"
  IGetThis:
  ; -- FAIL After SortBy a+,b$+,c$+ --
  ; a   b$  c$
  ; --  --  --
  ; 1   9   1
  ; 2   2   5
  ; 4   1   a
  ; 3   1   b
  ; 6   3   3
  ; 7   3   y   ;<- Primary Field is disrupted
  ; 5   3   z   
  ; 7   4   x   ;<- Primary Field is disrupted
  ; 7   5   a   ;<- c$ = a then w
  ; 7   5   w
  ; 8   3   2
  ; 9   3   x
EndDataSection
;-}

Re: Sorting option(quick/merge) for arrays.

Posted: Thu Mar 08, 2012 8:22 pm
by Trond
I see.