It is currently Thu Nov 26, 2020 7:12 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5
Author Message
 Post subject: Re: Fast string
PostPosted: Sun Feb 09, 2020 3:31 am 
Offline
Addict
Addict
User avatar

Joined: Tue May 08, 2007 1:27 pm
Posts: 2756
Location: Chiba, Japan
In the past I often worked around this with loops inside loops with a mod(loop,100) or something to build smaller strings first which was reasonably effective but recently, after a little thought I tried something else.

In many cases, these sorts of long string concatenation loops are for preparing data for disk (not always of course). Many times I've hit this is a loop to prepare lines of text or csv for a data dump, export or log dump. In fact I had this come up just the other day and was seeing significant slowdown as the string length went over 100's of kb and was just about to add the mod() trick and thought, "why not just stream straight to disk?" and then had that auto-thought (because disk is slower than memory!).

So I tried it, Openfile(), loop the strings being built to hold a single line of CSV, WriteString(), end loop, CloseFile().

All delays went away and it scales a lot larger.

I would guess that if your target was a network destination and streaming data the situation would be similar.

Perhaps if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it. If strings of data get really huge then streaming will be a better plan in most cases anyway.

I'm sure there will always be cases where fast string cat of large strings is needed and I'm sure people here are hitting them, but perhaps there are less than I first imagined.

_________________
Paul Dwyer

“In nature, it’s not the strongest nor the most intelligent who survives. It’s the most adaptable to change” - Charles Darwin
“If you can't explain it to a six-year old you really don't understand it yourself.” - Albert Einstein


Top
 Profile  
Reply with quote  
 Post subject: Re: Fast string
PostPosted: Tue Feb 11, 2020 10:06 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Mon Sep 20, 2004 7:12 am
Posts: 614
Location: Hell
No one thought about CopyMemoryString()?
Code:
DisableDebugger


Str.s   = ""
Text.s  = "1234567890"
Time    = ElapsedMilliseconds()
*Buffer  = AllocateMemory(2000002)
*Pointer = *Buffer
CopyMemoryString(Text, @*Pointer)
For i=1 To 99999
  CopyMemoryString(Text)
Next i

MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
EnableDebugger
Debug PeekS(*Buffer)

_________________
Link dead?
Change h3x0r.ath.cx into hex0rs.coderbu.de and all will be fine.


Top
 Profile  
Reply with quote  
 Post subject: Re: Fast string
PostPosted: Tue Feb 11, 2020 10:14 pm 
Offline
Addict
Addict

Joined: Thu Apr 18, 2019 8:17 am
Posts: 1085
HeX0R wrote:
No one thought about CopyMemoryString()?

Nice, but all the tricks like this shouldn't be needed.

pdwyer wrote:
if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it

Nah, PureBasic should just build them faster, as HeX0R just showed.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5

All times are UTC + 1 hour


Who is online

Users browsing this forum: User_Russian and 19 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye