Maybe you can use this as a basis ...
Before that, clean up the beginning and the end with Trim(...).
Link: SplitString to list and array with option double-quotes
StringField - Alterable functions - Feasibility demos
Re: StringField_BF - A alterable function - Feasibility demo
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
Re: StringField_BF - A alterable function - Feasibility demo
My StringField tool is ready, these test codes here are no longer needed by me.
They just serve as code example.
They just serve as code example.
地球上の平和
Re: StringField - Alterable functions - Feasibility demos
Well, despite all the prophecies of doom,
in the first real-world application of creating data lines, I achieve an enormous increase in speed when using my tool StringFields_BF compared to the StringField of PB.
This is due to the intended different way of working of my tool.
Instead of creating about 800 data rows in one second, I can more as 100,000 data rows in one second with my tool.
This achieves exactly what should be achieved.
https://www.purebasic.fr/english/viewto ... 12&t=77248
https://www.purebasic.fr/english/viewto ... 12&t=77219
in the first real-world application of creating data lines, I achieve an enormous increase in speed when using my tool StringFields_BF compared to the StringField of PB.
This is due to the intended different way of working of my tool.
Instead of creating about 800 data rows in one second, I can more as 100,000 data rows in one second with my tool.
This achieves exactly what should be achieved.
https://www.purebasic.fr/english/viewto ... 12&t=77248
https://www.purebasic.fr/english/viewto ... 12&t=77219
地球上の平和
Re: StringField - Alterable functions - Feasibility demos
There seems to be something wrong with your code as I'm getting around 9000ms for stringField_BF vs 3ms for OK_StringField_BS. It does the same thing in a different way.
Code: Select all
DeclareModule StringField_BF
EnableExplicit
Declare.s StringField_BF(string$, index, separator$)
EndDeclareModule
Module StringField_BF
Global NewList index() : AddElement(index())
Global result$, length
Procedure.s StringField_BF(string$, index, separator$)
; StringField - By Saki - Unicode - This code is free for using and enhancing
Protected i, ii, pos_1, pos_2, length_result, comp, amount_indexes, count_index
Protected len_separator=StringByteLength(separator$), *pointer.word, jump_in
Protected *string=@string$, *separator=@separator$
If Not PeekW(*string) : ProcedureReturn "" : EndIf
If index<1 : ProcedureReturn "" : EndIf
ClearList(index()) : AddElement(index())
If comp=CompareMemory(*string, *separator, len_separator)
jump_in=1 : index+1
EndIf
i=-2
Repeat
i+2
comp=CompareMemory(*string+i, *separator, len_separator)
If comp+jump_in
count_index+1
jump_in=0 : ii+1 : i+len_separator-2 : amount_indexes+1
AddElement(index()) : index()=i+2
EndIf
*pointer=*string+i+len_separator
Until count_index>index Or Not *pointer\w
AddElement(index())
If index>count_index : index=count_index : EndIf
amount_indexes=ii : i=0
If amount_indexes
Repeat
SelectElement(index(), i) : pos_1=index()
SelectElement(index(), i+1) : pos_2=index()
length_result=pos_2-pos_1-len_separator
If length_result>0 And pos_2-pos_1>0
If length<length_result
length=length_result
result$=Space(length_result>>1)
EndIf
FillMemory(@result$, length, 0, #PB_Word)
CopyMemory(*string+pos_1, @result$, length_result)
If i=index : ProcedureReturn result$ : EndIf
EndIf
i+1
Until i>index Or i>amount_indexes-1
EndIf
EndProcedure
EndModule
UseModule StringField_BF
Procedure OK_StringField_BS(*source,List StringFields.s(),separator=' ')
Protected *inp.Character,key.s
If *source
*inp = *source
While *inp\c <> 0
While (*inp\c <> separator)
*inp+2
Wend
key = PeekS(*source,(*inp-*source)>>1)
AddElement(StringFields())
StringFields() = key
If *inp\c <> 0
*inp+2
*source = *inp
Else
Break
EndIf
Wend
EndIf
ProcedureReturn ListSize(StringFields())
EndProcedure
Define separator$=" "
Define index, result$, string$
string$=" Hello I am a splitted string "
CompilerIf #PB_Compiler_Debugger
Define multiplier=2
CompilerElse
Define multiplier=9
CompilerEndIf
Define i
For i=1 To multiplier
string$+string$
Next i
Debug Len(string$)
Debug "================"
Define len_string=Len(string$)
Define time=ElapsedMilliseconds()
For index=0 To len_string
result$ = stringField_BF(string$, index, separator$)
If result$<>"" : Debug result$ : EndIf
Next
Define et = ElapsedMilliseconds()
MessageRequester("", Str(et-time))
Debug "++++++OK_StringField_BS ++++++++"
Define time1=ElapsedMilliseconds()
Define NewList MySplitStringBS.s()
OK_StringField_BS(@string$,MySplitStringBS())
ForEach MySplitStringBS()
result$ = MySplitStringBS()
If result$ <> "" : Debug result$ : EndIf
Next
Define et1 = ElapsedMilliseconds()
MessageRequester("", Str(et1-time1))
Re: StringField - Alterable functions - Feasibility demos
Hi, I'm looking,
I don't have more than 5ms here on a first test with debugger and purifier.
Try converting a 3mb image into data lines with the Data Encoder.
For me it takes about half a second and create about 50,000 lines.
I am only tested on Windows 573
Should there be a strange problem.
Did you provide the solution right away, that's cool.
But your variant is definitely faster as it seems, then it comes in anyway.
I have to take a closer look.
I don't have more than 5ms here on a first test with debugger and purifier.
Try converting a 3mb image into data lines with the Data Encoder.
For me it takes about half a second and create about 50,000 lines.
I am only tested on Windows 573
Should there be a strange problem.
Did you provide the solution right away, that's cool.
But your variant is definitely faster as it seems, then it comes in anyway.
I have to take a closer look.
地球上の平和
Re: StringField - Alterable functions - Feasibility demos
I'm talking about the 2nd code example in the 1st post of this topic that's supposed to be twice as fast, I copied it and compared in the previous post. What's the intent of your code? As it's not doing a good job of StringFields, change the multiplier=10 and it takes 50000ms vs 5ms for the OK_StringField_BS.
Re: StringField - Alterable functions - Feasibility demos
Ah, I see.
It's been a while now, but the point was, as I recall, that constantly recreating a string consumed a lot of time.
A larger empty string was then created once before, zeroed and the lines copied into it.
Then the string was zeroed again to include the next line.
This method is dirty, but it has never caused any problems.
The code is purely experimental and has been quickly discarded.
It is only left at the top of the thread in fairness to show people how to do things and then let them decide if it is OK or wrong.
This code has also only been tested on Windows.
Usually I make everything at least runnable on Windows first, then the test on Linux and then on Mac.
Sometimes I think this has to run on all OS because it doesn't contain anything special and then one test doesn't work after all.
The code above has been discarded very early on, it hasn't seen Linux or Mac yet.
The codes above are purely experimental, just to see what works well and what doesn't.
In the final tool only the well working code parts were combined, which was then very quickly finished.
The problem with the StringField function in PB is that the entire string must be re-read for each new substring to be selected.
This problem can be very serious with large files.
The original idea was to use CopyMemory and some tricks to speed up this process without fundamentally changing the function.
However, it became clear very quickly that this does not work.
Besides, it is not easy to program such a function without errors if you also want to try code tricks.
the-weavster split the code into two parts, which I wanted to avoid.
The way he did it, I always did it with my CSV crypters, that's the usual way.
So I combined the old knowledge with the new code parts to create a universally applicable new tool.
In the new data lines tool I then replaced the there used
PB StringField function with my StringFields_BF function to see what it does.
This was extremely faster.
As a test, more as 700,000 lines created in about 1 second with the BF version, the PB Version about 800 lines in one second.
It's been a while now, but the point was, as I recall, that constantly recreating a string consumed a lot of time.
A larger empty string was then created once before, zeroed and the lines copied into it.
Then the string was zeroed again to include the next line.
This method is dirty, but it has never caused any problems.
The code is purely experimental and has been quickly discarded.
It is only left at the top of the thread in fairness to show people how to do things and then let them decide if it is OK or wrong.
This code has also only been tested on Windows.
Usually I make everything at least runnable on Windows first, then the test on Linux and then on Mac.
Sometimes I think this has to run on all OS because it doesn't contain anything special and then one test doesn't work after all.
The code above has been discarded very early on, it hasn't seen Linux or Mac yet.
The codes above are purely experimental, just to see what works well and what doesn't.
In the final tool only the well working code parts were combined, which was then very quickly finished.
The problem with the StringField function in PB is that the entire string must be re-read for each new substring to be selected.
This problem can be very serious with large files.
The original idea was to use CopyMemory and some tricks to speed up this process without fundamentally changing the function.
However, it became clear very quickly that this does not work.
Besides, it is not easy to program such a function without errors if you also want to try code tricks.
the-weavster split the code into two parts, which I wanted to avoid.
The way he did it, I always did it with my CSV crypters, that's the usual way.
So I combined the old knowledge with the new code parts to create a universally applicable new tool.
In the new data lines tool I then replaced the there used
PB StringField function with my StringFields_BF function to see what it does.
This was extremely faster.
As a test, more as 700,000 lines created in about 1 second with the BF version, the PB Version about 800 lines in one second.
地球上の平和
Re: StringField - Alterable functions - Feasibility demos
still think you should fix up the code in this thread as your example doesn't work as expected
I've posted examples of StringField methods here
https://www.purebasic.fr/english/viewto ... 43#p570243
I've posted examples of StringField methods here
https://www.purebasic.fr/english/viewto ... 43#p570243
List=1759ms Array=2094ms PB=7633ms Len = 928 reps 10000
Re: StringField - Alterable functions - Feasibility demos
I removed it, not worth bothering with it further as it was purely experimental.
地球上の平和