Page 5 of 5
Re: Fast string
Posted: Sun Feb 09, 2020 3:31 am
by pdwyer
In the past I often worked around this with loops inside loops with a mod(loop,100) or something to build smaller strings first which was reasonably effective but recently, after a little thought I tried something else.
In many cases, these sorts of long string concatenation loops are for preparing data for disk (not always of course). Many times I've hit this is a loop to prepare lines of text or csv for a data dump, export or log dump. In fact I had this come up just the other day and was seeing significant slowdown as the string length went over 100's of kb and was just about to add the mod() trick and thought, "why not just stream straight to disk?" and then had that auto-thought (because disk is slower than memory!).
So I tried it, Openfile(), loop the strings being built to hold a single line of CSV, WriteString(), end loop, CloseFile().
All delays went away and it scales a lot larger.
I would guess that if your target was a network destination and streaming data the situation would be similar.
Perhaps if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it. If strings of data get really huge then streaming will be a better plan in most cases anyway.
I'm sure there will always be cases where fast string cat of large strings is needed and I'm sure people here are hitting them, but perhaps there are less than I first imagined.
Re: Fast string
Posted: Tue Feb 11, 2020 10:06 pm
by HeX0R
No one thought about CopyMemoryString()?
Code: Select all
DisableDebugger
Str.s = ""
Text.s = "1234567890"
Time = ElapsedMilliseconds()
*Buffer = AllocateMemory(2000002)
*Pointer = *Buffer
CopyMemoryString(Text, @*Pointer)
For i=1 To 99999
CopyMemoryString(Text)
Next i
MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
EnableDebugger
Debug PeekS(*Buffer)
Re: Fast string
Posted: Tue Feb 11, 2020 10:14 pm
by BarryG
HeX0R wrote:No one thought about CopyMemoryString()?
Nice, but all the tricks like this shouldn't be needed.
pdwyer wrote:if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it
Nah, PureBasic should just build them faster, as HeX0R just showed.
Re: Fast string
Posted: Mon Nov 06, 2023 3:31 pm
by Quin
+1
Re: Fast string
Posted: Fri Nov 17, 2023 6:02 pm
by ShadowStorm
Hi, not bad at all, I propose this humble code, it can certainly be improved
Code: Select all
; -----------------------------------------------------------------------------------------------
; By HeX0R the 11/02/2020
; DisableDebugger
;
; Str.s = ""
; Text.s = "1234567890"
; Time = ElapsedMilliseconds()
; *Buffer = AllocateMemory(25000000)
; *Pointer = *Buffer
; CopyMemoryString(Text, @*Pointer)
; For i=1 To 999999
; CopyMemoryString(Text)
; Next i
;
; MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
; EnableDebugger
; Debug PeekS(*Buffer)
; -----------------------------------------------------------------------------------------------
; By ShadowStorm the 17/11/2023, from HeX0R Code.
; This is just a single buffer.
; To have several independent buffers, see my StringBuilder 2 And 3 that I created on the PureBasic forum.
; Ceci est juste une mémoire tampon unique.
; Pour avoir plusieur mémoire tampon indépendente, voir mon StringBuilder 2 et 3 que j'ai créér sur le forum Français de PureBasic.
Global *SimplyStringBuilder_Buffer
Global *Pointer
; Create and initialize the StringBuilder buffer.
; Créer et initialiser le tampon StringBuilder.
Procedure.b SimplyStringBuilder_Initialize()
; If a buffer already exists, it will be deleted. Only one buffer is allowed, To avoid memory leaks when successive calls are made To this function.
; Si un tampon existe déjà, il sera supprimé. Un seul tampon est autorisé, afin d'éviter les fuites de mémoire lors d'appels successifs à cette fonction.
If *SimplyStringBuilder_Buffer
FreeMemory(*SimplyStringBuilder_Buffer)
EndIf
; This is meant To be simple, so super large memory buffer For speed, no checking Or reallocation.
; Ceci est censé être simple, donc un tampon de mémoire super grand pour la vitesse, pas de vérification ou de réallocation.
*SimplyStringBuilder_Buffer = AllocateMemory(999999999)
*Pointer = *SimplyStringBuilder_Buffer
CopyMemoryString("", @*Pointer)
If *SimplyStringBuilder_Buffer
ProcedureReturn 1
Else
ProcedureReturn 0
EndIf
EndProcedure
; Clears the StringBuilder buffer.
; Efface la mémoire tampon du StringBuilder.
Procedure.b SimplyStringBuilder_Clear()
; Clears the buffer but does not delete it.
; Efface la mémoire tampon mais ne la supprime pas.
If *SimplyStringBuilder_Buffer
FillMemory(*SimplyStringBuilder_Buffer, MemorySize(*SimplyStringBuilder_Buffer))
ProcedureReturn 1
Else
ProcedureReturn 0
EndIf
EndProcedure
; Add a string to the end of the StringBuilder buffer.
; Ajoute une chaîne a la fin du tampon StringBuilder.
Procedure.b SimplyStringBuilder_AddString(String.s)
If *SimplyStringBuilder_Buffer And String.s > ""
CopyMemoryString(String.s, @*Pointer)
ProcedureReturn 1
Else
ProcedureReturn 0
EndIf
EndProcedure
; Retrieves Data from the StringBuilder buffer.
; Récupère les données de la mémoire tampon du StringBuilder.
Procedure.s SimplyStringBuilder_GetString()
If *SimplyStringBuilder_Buffer
ProcedureReturn PeekS(*SimplyStringBuilder_Buffer)
Else
ProcedureReturn ""
EndIf
EndProcedure
; ------------------------------------- EXEMPLE ------------------------------------------------------------------------------------
; Create and initialize buffer.
; Créer et Initialise la mémoire tampon.
SimplyStringBuilder_Initialize()
; Character string to be added to the buffer.
; Chaîne de caractère à ajouter à la mémoire tampon.
Text.s = "1234567890"
Time = ElapsedMilliseconds()
For i = 1 To 999999
; Adds string to buffer 9 999 999 times.
; Ajoute 9 999 999 fois la chaîne dans la mémoire tampon.
SimplyStringBuilder_AddString(Text)
Next i
MessageRequester("", Str((ElapsedMilliseconds()-Time)) + " Millisecondes.")
; Retrieves Data from the buffer.
; Récupère les données de la mémoire tampon.
Debug SimplyStringBuilder_GetString()
Debug Chr(13) + "Longueur: " + Len(SimplyStringBuilder_GetString()) + " Octets."
Debug Chr(13) + "Taille approximative en mémoire en Mega Octets: " + StrF(StringByteLength(SimplyStringBuilder_GetString()) / 1024 / 1000, 3) + " Mo."
Re: Fast string
Posted: Fri Nov 17, 2023 6:40 pm
by Demivec
HeX0R wrote: Tue Feb 11, 2020 10:06 pm
No one thought about CopyMemoryString()?
It was in the first pages in the thread, even from 2014.
Re: Fast string
Posted: Fri Nov 17, 2023 6:53 pm
by ShadowStorm
My new simplistic version here is the fastest of all my StringBuilders.
Re: Fast string
Posted: Fri Nov 17, 2023 8:59 pm
by HeX0R
There are plenty of such threads now, here e.g.:
viewtopic.php?t=80831