Please speed up this string operation

[Dude]

The following code is meant to put an ASCII box around a block of text, and it works great for small text, but for large blocks of text (even just 100 KB in size), it's horrendously slow. The goal is to use it on the clipboard text, and since the clipboard text can be any size at any given time... you know what I mean? It has to be as blazingly fast as possible.

So, can someone please optimize it for me? I understand it can be sped up considerably with *i.Character operations (like the GetLongestLine() procedure) but I'm not sure how to apply it.

I'd prefer an easy, generic, non-specific solution because I want to be able to learn from that to create future similar text mods. Thank you.

Side-question: Is the long delay caused by StringField() or LSet() being too slow, or the loop, or what?

Desired output:

Code: Select all

+-------+
| One   |
| Two   |
| Three |
+-------+

Code: Select all

#SOC=SizeOf(Character)

Procedure GetLongestLine(text$)
  *i.Character=@text$
  While *i\c
    If *i\c=#CR Or *i\c=#LF
      If s>l : l=s : EndIf
      s=0
    Else
      s+1
    EndIf
    *i+#SOC
  Wend
  If s>l : l=s : EndIf ; For single lines without #CRLF$ at end.
  ProcedureReturn l
EndProcedure

Procedure.s Boxed(text$)
  If text$
    maxlen=GetLongestLine(text$)
    t$="+"+LSet("",maxlen+2,"-")+"+"+#CRLF$
    For i=1 To CountString(text$,#CRLF$)
      t$+"| "+LSet(StringField(text$,i,#CRLF$),maxlen)+" |"+#CRLF$
    Next
    t$+"+"+LSet("",maxlen+2,"-")+"+"+#CRLF$
  EndIf
  ProcedureReturn t$
EndProcedure

text$="One"+#CRLF$
text$+"Two"+#CRLF$
text$+"Three"+#CRLF$

MessageRequester("Boxed",Boxed(text$))

[Josh]

• First problem is the t$+... command in the loop of your procedure Boxed().
  With your procedure GetLongestLine() you know the number of lines and the length of your lines. So you know how long your result must be. Allocate memory with this size and write in this.
• Next problem is the CountString() at the end of your loop. Never do that, put the result of CountString() into a variable before.
• Next problem is the StringField-command. Better is to run through text$ and look for start and end of each sequenz.

[wilbert]

Josh is absolutely right.
I would also recommend to pass a pointer to GetLongestLine instead of a string.

Code: Select all

#SOC=SizeOf(Character)

Procedure.i GetLongestLine(*c.Character, *numlines.Integer=0)
  Protected.i l, n, s
  While *c\c
    If *c\c=#LF
      If s>l : l=s : EndIf
      s=0 : n+1
    ElseIf *c\c<>#CR
      s+1
    EndIf
    *c+#SOC
  Wend
  If s ; For last line without #LF at end.
    If s>l : l=s : EndIf
    n+1
  EndIf
  If *numlines : *numlines\i=n : EndIf
  ProcedureReturn l
EndProcedure

text$="One"+#CRLF$
text$+"Two"+#CRLF$
text$+"Three"+#CRLF$

Debug GetLongestLine(@text$, @numlines)
Debug numlines

[Dude]

First problem is the t$+... command in the loop of your procedure Boxed()

I knew this would be slow, and am looking for a faster way to append to it. Hence my request for help.

Next problem is the StringField-command

So, StringField() is slow, then? Thanks, I won't use it in future. What's a faster workaround?

[Josh]

So, StringField() is slow then, and adds to the bottleneck? Thanks, I won't use it in future. What's a faster workaround?

No, I didn't say that StringField() is slow. The Problem is, you use it in the loop an Pb has to run through the string at each loop. Think about what that means with 10k lines.

Even worse, the CountString () in the loop call. Pb must go through the whole string with each pass.

I have given you the better way.

[Dude]

I didn't know that StringField() loops through the ENTIRE string with each call. That's bad.

[infratec]

Maybe this is fast enough (not speed tested):

Code: Select all

#SOC = SizeOf(Character)

#CharactersInFrontOfText = 1

Structure LineInfoStructure
  Addr.i
  Length.i
EndStructure


Procedure.i GetLongestLine(*c.Character, List LineInfoList.LineInfoStructure())
  
  Protected.i l, s
  
  
  ClearList(LineInfoList())
  AddElement(LineInfoList())
  LineInfoList()\Addr = *c
  While *c\c
    If *c\c=#LF
      If s>l : l=s : EndIf
      LineInfoList()\Length = s
      AddElement(LineInfoList())
      LineInfoList()\Addr = *c + #SOC
      s=0
    ElseIf *c\c<>#CR
      s+1
    EndIf
    *c+#SOC
  Wend
  If s ; For last line without #LF at end.
    If s>l : l=s : EndIf
    LineInfoList()\Length = s
  Else
    DeleteElement(LineInfoList())
  EndIf
  
  ProcedureReturn l
  
EndProcedure



Procedure.s Boxed(*text.Character)
  
  Protected.i NumLines, MaxLineLength, i, j
  Protected Result$
  Protected *BoxText, *Help.Character
  Protected NewList LineInfoList.LineInfoStructure()
  
  
  MaxLineLength = GetLongestLine(*text, LineInfoList())
  NumLines = ListSize(LineInfoList())
  
  ;Debug MaxLineLength
  ;Debug NumLines
  
  ; allocate the needed memory for the result text
  ;'+' + MaxLineLength + '+' + 'CRLF' ->
  ; 1  + MaxLineLength +  1  +   2
  ; multiplied by number of lines + 1 startline + 1 endline
  ; + additional character for the string termination
  *BoxText = AllocateMemory((1 + MaxLineLength + 1 + 2) * (1 + NumLines + 1) * #SOC + #SOC, #PB_Memory_NoClear)
  If *BoxText
    
    ; fill all with space character
    FillMemory(*BoxText, MemorySize(*BoxText), ' ', #PB_Character)
    
    ; terminate the string
    ; write a terminating 0 at last address of the memory - 1 character
    PokeC(*BoxText + MemorySize(*BoxText) - #SOC, #Null)
    
    ; write first box line
    PokeC(*BoxText + Ptr , '+')
    FillMemory(*BoxText + Ptr + #SOC, MaxLineLength * #SOC, '-', #PB_Character)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength) * #SOC, '+')
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 2) * #SOC, #LF)
    
    ; calc the pointer for the first text line    
    *Help = *BoxText + (1 + MaxLineLength + 1 + 2) * #SOC
    
    ; process each line of text
    ForEach LineInfoList()
      *Help\c = '|' : *Help + #SOC ; write '+' and increase the pointer
      
      ; copy the original text to the place of the pointer
      CopyMemory(LineInfoList()\Addr, *Help, LineInfoList()\Length * #SOC)
      
      ; add the MaxLineLength to the pointer
      *Help + (MaxLineLength * #SOC)
      
      *Help\c = '|' : *Help + #SOC ; write '+' and increase the pointer
      *Help\c = #CR : *Help + #SOC ; write CR and increase the pointer
      *Help\c = #LF : *Help + #SOC ; write LF and increase the pointer
    Next
    
    ; calc Ptr
    ; size of one line multiplied with number of lines + the additional start line
    Ptr = (1 + MaxLineLength + 1 + 2) * (NumLines + 1) * #SOC
    
    ; write last box line
    PokeC(*BoxText + Ptr , '+')
    FillMemory(*BoxText + Ptr + #SOC, MaxLineLength * #SOC, '-', #PB_Character)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength) * #SOC, '+')
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 2) * #SOC, #LF)
    
    ;ShowMemoryViewer(*BoxText, MemorySize(*BoxText))
    
    Result$ = PeekS(*BoxText)
    
    FreeMemory(*BoxText)
  EndIf
  
  ProcedureReturn Result$
  
EndProcedure


text$="One"+#CRLF$
text$+"Two"+#CRLF$
text$+"Three"+#CRLF$

Debug Boxed(@text$)

Bernd

[Dude]

infratec, that is indeed fast, but unfortunately I don't understand it and won't be able to adapt it to other text styles. You see, I don't just want to wrap text inside a box, but other styles too. So I was hoping to get a fast and easy to understand framework to build off. Your solution seems to be very specific in its implementation. I've edited my first post to better explain what I mean.

Thanks very much for helping in this specific text mod situation, though.

[infratec]

Hi Dude,

it is not specific ...

What do you want to 'adapt'

The missing space at start and end?
It's easy to modify it for this addition.

Maybe I'll name the 'constants' to make it clearer.

[infratec]

Added more comments to make it clearer.
Fixed a 'bug' + in front and end instead of |

[Dude]

I mean I want to wrap the text in other styles, not just simple boxes. But your code is above my knowledge.

Can you make it more generic? Let's say I want to also put text inside this ASCII border:

Code: Select all

  .-----------------------------------------------------------------.
 /  .-.                                                         .-.  \
|  /   \            Lines                                     /   \  |
| |\_.  |           of                                       |    /| |
|\|  | /|           text                                     |\  | |/|
| `---' |           here                                     | `---' |
|       |                                                    |       |
|       |----------------------------------------------------|       |
\       |                                                    |       /
 \     /                                                      \     /
  `---'                                                        `---'

See, it's not so simple.

[infratec]

Your first desired ouput looks a bit different

But also your new 'layout' is possible with a 'rewrite'.
So if more text lines are available they are only included between

| | bla | |

Is this right?

[wilbert]

I mean I want to wrap the text in other styles, not just simple boxes. But your code is above my knowledge.

Can you make it more generic? Let's say I want to also put text inside this ASCII border:

Code: Select all

  .-----------------------------------------------------------------.
 /  .-.                                                         .-.  \
|  /   \            Lines                                     /   \  |
| |\_.  |           of                                       |    /| |
|\|  | /|           text                                     |\  | |/|
| `---' |           here                                     | `---' |
|       |                                                    |       |
|       |----------------------------------------------------|       |
\       |                                                    |       /
 \     /                                                      \     /
  `---'                                                        `---'

See, it's not so simple.

How do you want to define such an ascii border ?
In the data section ?

[Dude]

Basically, I just want to add new text before and after each line of text, but I need to know the longest line in that text first, so formatting can be done correctly.

[infratec]

Hi,

try this:

Code: Select all

#SOC = SizeOf(Character)

#CharactersInFrontOfText = 1

Structure LineInfoStructure
  Addr.i
  Length.i
EndStructure


Procedure.i GetLongestLine(*c.Character, List LineInfoList.LineInfoStructure())
  
  Protected.i l, s
  
  
  ClearList(LineInfoList())
  AddElement(LineInfoList())
  LineInfoList()\Addr = *c
  While *c\c
    If *c\c=#LF
      If s>l : l=s : EndIf
      LineInfoList()\Length = s
      AddElement(LineInfoList())
      LineInfoList()\Addr = *c + #SOC
      s=0
    ElseIf *c\c<>#CR
      s+1
    EndIf
    *c+#SOC
  Wend
  If s ; For last line without #LF at end.
    If s>l : l=s : EndIf
    LineInfoList()\Length = s
  Else
    DeleteElement(LineInfoList())
  EndIf
  
  ProcedureReturn l
  
EndProcedure



Procedure.s Boxed(*text.Character)
  
  Protected.i NumLines, MaxLineLength, i, j
  Protected Result$
  Protected *BoxText, *Help.Character
  Protected NewList LineInfoList.LineInfoStructure()
  
  
  MaxLineLength = GetLongestLine(*text, LineInfoList())
  NumLines = ListSize(LineInfoList())
  
  ;Debug MaxLineLength
  ;Debug NumLines
  
  ; allocate the needed memory for the result text
  ;'+' + MaxLineLength + '+' + 'CRLF' ->
  ; 1  + MaxLineLength +  1  +   2
  ; multiplied by number of lines + 1 startline + 1 endline
  ; + additional character for the string termination
  *BoxText = AllocateMemory((1 + MaxLineLength + 1 + 2) * (1 + NumLines + 1) * #SOC + #SOC, #PB_Memory_NoClear)
  If *BoxText
    
    ; fill all with space character
    FillMemory(*BoxText, MemorySize(*BoxText), ' ', #PB_Character)
    
    ; terminate the string
    ; write a terminating 0 at last address of the memory - 1 character
    PokeC(*BoxText + MemorySize(*BoxText) - #SOC, #Null)
    
    ; write first box line
    PokeC(*BoxText + Ptr , '+')
    FillMemory(*BoxText + Ptr + #SOC, MaxLineLength * #SOC, '-', #PB_Character)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength) * #SOC, '+')
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 2) * #SOC, #LF)
    
    ; calc the pointer for the first text line    
    *Help = *BoxText + (1 + MaxLineLength + 1 + 2) * #SOC
    
    ; process each line of text
    ForEach LineInfoList()
      *Help\c = '|' : *Help + #SOC ; write '+' and increase the pointer
      
      ; copy the original text to the place of the pointer
      CopyMemory(LineInfoList()\Addr, *Help, LineInfoList()\Length * #SOC)
      
      ; add the MaxLineLength to the pointer
      *Help + (MaxLineLength * #SOC)
      
      *Help\c = '|' : *Help + #SOC ; write '+' and increase the pointer
      *Help\c = #CR : *Help + #SOC ; write CR and increase the pointer
      *Help\c = #LF : *Help + #SOC ; write LF and increase the pointer
    Next
    
    ; calc Ptr
    ; size of one line multiplied with number of lines + the additional start line
    Ptr = (1 + MaxLineLength + 1 + 2) * (NumLines + 1) * #SOC
    
    ; write last box line
    PokeC(*BoxText + Ptr , '+')
    FillMemory(*BoxText + Ptr + #SOC, MaxLineLength * #SOC, '-', #PB_Character)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength) * #SOC, '+')
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (1 + MaxLineLength + 2) * #SOC, #LF)
    
    ;ShowMemoryViewer(*BoxText, MemorySize(*BoxText))
    
    Result$ = PeekS(*BoxText)
    
    FreeMemory(*BoxText)
  EndIf
  
  ProcedureReturn Result$
  
EndProcedure



Structure LineListStructure
  Left$
  Right$
EndStructure




Procedure.s DudeBoxed(*text.Character, List StartLineList.LineListStructure(), HorFillChar.c, TextLineDistance.i, *NormalLine.LineListStructure, List EndLineList.LineListStructure())
  
  Protected.i NumTextLines, MaxLineLength, i, j, LeftLength, RightLength, StartLineListOk, LineInfoListOk, TotalLines, Lines1
  Protected Result$
  Protected *BoxText, *Help.Character
  Protected NewList LineInfoList.LineInfoStructure()
  
  
  MaxLineLength = GetLongestLine(*text, LineInfoList())
  NumTextLines = ListSize(LineInfoList())
  
  FirstElement(StartLineList())
  LeftLength = Len(StartLineList()\Left$)
  RightLength = Len(StartLineList()\Right$)
  
  If ListSize(StartLineList()) > NumTextLines
    Lines1 = ListSize(StartLineList())
  Else
    Lines1 = 1 + NumTextLines
  EndIf
  
  
  If TextLineDistance = 0
    TotalLines = Lines1 + 1 + ListSize(EndLineList())
  Else
    If (TextLineDistance * 2) > ListSize(StartLineList())
      TotalLines = (TextLineDistance * 2) + Lines1 + ListSize(EndLineList())
    Else
      TotalLines = Lines1 + (TextLineDistance * 2) + 1 + ListSize(EndLineList())
    EndIf
  EndIf
  
  ;Debug MaxLineLength
  ;Debug NumLines
  
  ; allocate the needed memory for the result text
  ;'+' + MaxLineLength + '+' + 'CRLF' ->
  ; 1  + MaxLineLength +  1  +   2
  ; multiplied by number of lines + 1 startline + 1 endline
  ; + additional character for the string termination
  *BoxText = AllocateMemory((LeftLength + MaxLineLength + RightLength + 2) * TotalLines * #SOC + #SOC, #PB_Memory_NoClear)
  If *BoxText
    
    ; fill all with space character
    FillMemory(*BoxText, MemorySize(*BoxText), ' ', #PB_Character)
    
    ; terminate the string
    ; write a terminating 0 at last address of the memory - 1 character
    PokeC(*BoxText + MemorySize(*BoxText) - #SOC, #Null)
    
    ; write first box line
    PokeS(*BoxText + Ptr , StartLineList()\Left$, -1, #PB_String_NoZero)
    FillMemory(*BoxText + Ptr + LeftLength * #SOC, MaxLineLength * #SOC, HorFillChar, #PB_Character)
    PokeS(*BoxText + Ptr + (LeftLength + MaxLineLength) * #SOC, StartLineList()\Right$, -1, #PB_String_NoZero)
    PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC, #LF)
    
    StartLineListOk = NextElement(StartLineList())
    
    ; calc the pointer for the first text line    
    *Help = *BoxText + (LeftLength + MaxLineLength + RightLength + 2) * #SOC
    
    ; process each line of text
    LineInfoListOk = FirstElement(LineInfoList())
    j = 0
    Repeat
      If StartLineListOk
        PokeS(*Help, StartLineList()\Left$, -1, #PB_String_NoZero)
      Else
        PokeS(*Help, *NormalLine\Left$, -1, #PB_String_NoZero)
      EndIf
      
      If j >= TextLineDistance And LineInfoListOk
        ; copy the original text to the place of the pointer
        CopyMemory(LineInfoList()\Addr, *Help + LeftLength * #SOC, LineInfoList()\Length * #SOC)
        LineInfoListOk = NextElement(LineInfoList())
      EndIf
      
      ; add the MaxLineLength to the pointer
      *Help + (LeftLength + MaxLineLength) * #SOC
      If StartLineListOk
        PokeS(*Help, StartLineList()\Right$, -1, #PB_String_NoZero)
        StartLineListOk = NextElement(StartLineList())
      Else
        PokeS(*Help, *NormalLine\Right$, -1, #PB_String_NoZero)
      EndIf
      *Help + RightLength * #SOC
      
      *Help\c = #CR : *Help + #SOC ; write CR and increase the pointer
      *Help\c = #LF : *Help + #SOC ; write LF and increase the pointer
      
      j + 1
      
    Until TotalLines >= j And LineInfoListOk = #False And StartLineListOk = #False
    
    ; calc Ptr
    ; size of one line multiplied with number of lines + the additional start line
    Ptr = (LeftLength + MaxLineLength + RightLength + 2) * (j + 1) * #SOC
    
    For i = 1 To TextLineDistance
      PokeS(*BoxText + Ptr , *NormalLine\Left$, -1, #PB_String_NoZero)
      PokeS(*BoxText + Ptr + (LeftLength + MaxLineLength) * #SOC, *NormalLine\Right$, -1, #PB_String_NoZero)
      PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 1) * #SOC, #CR)
      PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC, #LF)
      Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC
    Next i
    
    
    ; write last box line
    PokeS(*BoxText + Ptr , *NormalLine\Left$, -1, #PB_String_NoZero)
    FillMemory(*BoxText + Ptr + LeftLength * #SOC, MaxLineLength * #SOC, HorFillChar, #PB_Character)
    PokeS(*BoxText + Ptr + (LeftLength + MaxLineLength) * #SOC, *NormalLine\Right$, -1, #PB_String_NoZero)
    PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 1) * #SOC, #CR)
    PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC, #LF)
    
    Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC
    
    ForEach EndLineList()
      PokeS(*BoxText + Ptr , EndLineList()\Left$, -1, #PB_String_NoZero)
      PokeS(*BoxText + Ptr + (LeftLength + MaxLineLength) * #SOC, EndLineList()\Right$, -1, #PB_String_NoZero)
      PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 1) * #SOC, #CR)
      PokeC(*BoxText + Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC, #LF)
      Ptr + (LeftLength + MaxLineLength + RightLength + 2) * #SOC
    Next
    
    ;ShowMemoryViewer(*BoxText, MemorySize(*BoxText))
    
    Result$ = PeekS(*BoxText)
    
    FreeMemory(*BoxText)
  EndIf
  
  ProcedureReturn Result$
  
EndProcedure




text$="One"+#CRLF$
text$+"Two"+#CRLF$
text$+"Three"+#CRLF$
; text$+"Four"+#CRLF$
; text$+"Five"+#CRLF$
; text$+"Six"+#CRLF$
; text$+"Seven"+#CRLF$

Debug Boxed(@text$)


NewList StartLineList.LineListStructure()
Define NormalLine.LineListStructure
NewList EndLineList.LineListStructure()

AddElement(StartLineList())
StartLineList()\Left$  = "  .------"
StartLineList()\Right$ = "------.  "

AddElement(StartLineList())
StartLineList()\Left$  = " /  .-.  "
StartLineList()\Right$ = "  .-.  \ "

AddElement(StartLineList())
StartLineList()\Left$  = "|  /   \ "
StartLineList()\Right$ = " /   \  |"

AddElement(StartLineList())
StartLineList()\Left$  = "| |\_.  |"
StartLineList()\Right$ = "|    /| |"

AddElement(StartLineList())
StartLineList()\Left$  = "|\|  | /|"
StartLineList()\Right$ = "|\  | |/|"

AddElement(StartLineList())
StartLineList()\Left$  = "| `---' |"
StartLineList()\Right$ = "| '---´ |"


NormalLine\Left$       = "|       |"
NormalLine\Right$      = "|       |"


AddElement(EndLineList())
EndLineList()\Left$    = "\       |"
EndLineList()\Right$   = "|       /"

AddElement(EndLineList())
EndLineList()\Left$    = " \     / "
EndLineList()\Right$   = " \     / "

AddElement(EndLineList())
EndLineList()\Left$    = "  `---'  "
EndLineList()\Right$   = "  '---´  "


Debug DudeBoxed(@text$, StartLineList(), '-', 0, @NormalLine, EndLineList())

Bernd