[Done] PB 6.40 Alpha 1 - Issue with referenced strings which are modified

[Matheos]

Small loop should be very fast, can you post your code ?

Yes, one example that concerns me especially, is where we have a console session open to provide an editor. The data is in a very proprietary form, which often contains control codes. We have to convert the control codes to a displayable symbol (not in the actual data but only when it's displayed).

Therefore, as the user moves around the data with PgUp/PgDn/Home/End and so on, we have to convert the control characters into a displayable character, normally 149 which is a dot, so the user can identify the characters that must not be changed or overwritten :

Code: Select all

ReplaceString(Params.s, Chr(60251), Chr(149), #PB_String_InPlace)

[Fred]

How big is param ? To me, using a PB function for this should be very close in term of speed. Something like (use it with caution, I didn't test it much):

Code: Select all

Procedure ReplaceChar(*String.Character, CharToSearch.c, CharToReplace.c)
  If *String = 0
    ProcedureReturn
  EndIf
  
  While *String\c
    If *String\c = CharToSearch
      *String\c = CharToReplace
    EndIf
    
    *String+SizeOf(Character)
  Wend
EndProcedure

Test$ = "Hello World !"

ReplaceChar(@Test$, 'o', 'O')

Debug Test$

With this proc, you can also replace several char in one pass.

[User_Russian]

Code: Select all

    *String+1

PB uses Unicode strings, and each character requires 2 bytes. I think it's better to do it this way.

Code: Select all

*String+SizeOf(Character)

[ChrisR]

If used with ReplaceChar(@Test$, 'o', ''), the length needs to be recalculated with PeekS
By changing the ReplaceChar procedure like this, you don't have to call PeekS
but Timo won't agree with this hack

Code: Select all

Procedure ReplaceChar(*String.Character, CharToSearch.c, CharToReplace.c)
  If *String = 0 : ProcedureReturn : EndIf
  
  Protected *Len.Integer = *String -SizeOf(Integer)
  *Len\i = 0
  While *String\c
    If *String\c = CharToSearch
      *String\c = CharToReplace
      If CharToReplace = 0
        Break   ; Comment it for testing the lenght 
      EndIf
    EndIf
    *Len\i + 1
    *String + SizeOf(Character)
  Wend
EndProcedure

Test$ = "Hello World !"
ReplaceChar(@Test$, 'o', 'O')
Debug Test$

ReplaceChar(@Test$, ' ', '')
Debug Test$
Debug Len(Test$)
;Test$ = PeekS(@Test$)
;Debug Len(Test$)

[STARGÅTE]

Interestingly, a self-written loop is faster than ReplaceString(, #PB_String_InPlace).
However, this self-written function only works for single character replacement.

Code: Select all

CompilerIf #PB_Compiler_Debugger
	CompilerError "No Debugger!!!!!!!!!!"
CompilerEndIf

Procedure ReplaceChar(*String.Character, CharToSearch.c, CharToReplace.c)
	If *String = 0
		ProcedureReturn
	EndIf
	While *String\c
		If *String\c = CharToSearch
			*String\c = CharToReplace
		EndIf
		*String + SizeOf(Character)
	Wend
EndProcedure

OpenConsole()

Define Time.q, Time1.q, Time2.q
Define String.s = ReplaceString(Space(10000), " ", "Abc")

Time = ElapsedMilliseconds()
For I = 1 To 1000
	ReplaceString(String, "A", "a", #PB_String_InPlace)
	ReplaceString(String, "a", "A", #PB_String_InPlace)
Next
Time1 = ElapsedMilliseconds() - Time

Time = ElapsedMilliseconds()
For I = 1 To 1000
	ReplaceChar(@String, Asc("A"), Asc("a"))
	ReplaceChar(@String, Asc("a"), Asc("A"))
Next
Debug String
Time2 = ElapsedMilliseconds() - Time


PrintN("ReplaceString: " + Time1 + " ms")
PrintN("ReplaceChar: " + Time2 + " ms")
Input()

Therefore, as the user moves around the data with PgUp/PgDn/Home/End and so on, we have to convert the control characters into a displayable character, normally 149 which is a dot, so the user can identify the characters that must not be changed or overwritten :

Code: Select all

ReplaceString(Params.s, Chr(60251), Chr(149), #PB_String_InPlace)

You can replace this function and you will gain speed, instead of loosing time.

Code: Select all

ReplaceChar(@Params, 60251, 149)

If used with ReplaceChar(@Test$, 'o', ''), the length needs to be recalculated with PeekS

[Matheos]

Interestingly, a self-written loop is faster than ReplaceString(, #PB_String_InPlace).
However, this self-written function only works for single character replacement.

Thanks for clarifying, so far I haven't had time to install PB 6.40. The performance gain that you mention wasn't the case in the past when we tried self-written loops, but that could well have been because our work was compiled under ASM, which we've since moved away from.

Happy to try it therefore and write our own code. Personally it still leaves me nervous of committing heavily to PureBasic, as great as it is, the reason being that there have been knee-jerk changes, which have broken our code 3 times now, one of which was later retracted. Sometimes it seems that a forum member's query leads to a quick change to the language and that's a worry for us. Still, there's perhaps a price to pay for the good points in the language. I wish that a pre-validation could have been put into PB_String_InPlace to prevent the overflow mentioned by Fred. It seems a pity but we'll overlook it and continue for now.

[Fred]

It's not unusual for a programming language to change things here and here, we try to keep it to a minimum and it's often fast to go from one version to another. Try to migrate from .net 4.8 to .net 9 to see how big can be the changes for a mainstream language.

[ChrisR]

I added handling for the null char to work like ReplaceString(String$, " ", ""), to remove the searched character, here remove all space
If prefered instead of ReplaceString!

Code: Select all

; ReplaceChar() with the null char behaves differently than ReplaceString(), it truncates the string rather than deleting the searched character and we need PeeKS() just after to reset the lenght
Procedure ReplaceChar(*String.Character, CharToSearch.c, CharToReplace.c)
  If *String = 0 Or CharToSearch = 0
    ProcedureReturn
  EndIf
  
  While *String\c
    If *String\c = CharToSearch
      *String\c = CharToReplace
    EndIf
    *String + SizeOf(Character)
  Wend
EndProcedure

; ReplaceCharNew() with Cthe null char need PeeKS() just after to reset the lenght
Procedure ReplaceCharNew(*String.Character, CharToSearch.c, CharToReplace.c)
  If *String = 0 Or CharToSearch = 0
    ProcedureReturn
  EndIf
  
  ;ShowMemoryViewer(*String, (MemoryStringLength(*String) +1) * SizeOf(Character))
  While *String\c
    If *String\c = CharToSearch
      If CharToReplace
        *String\c = CharToReplace
      Else
        MoveMemory(*String + SizeOf(Character), *String, MemoryStringLength(*String) * SizeOf(Character))
        Continue
      EndIf
    EndIf
    *String + SizeOf(Character)
  Wend
EndProcedure

; Warning: It is not recommended to use this hack to calculate the correct size, without an official guarantee that there will not be future changes to this prefix ♥;)
Procedure ReplaceCharNewLen(*String.Character, CharToSearch.c, CharToReplace.c)
  If *String = 0 Or CharToSearch = 0
    ProcedureReturn
  EndIf
  
  Protected *Len.integer = *String - SizeOf(Integer)
  *Len\i = 0
  ;ShowMemoryViewer(*String, (MemoryStringLength(*String) +1) * SizeOf(Character))
  While *String\c
    If *String\c = CharToSearch
      If CharToReplace
        *String\c = CharToReplace
      Else
        MoveMemory(*String + SizeOf(Character), *String, MemoryStringLength(*String) * SizeOf(Character))
        Continue
      EndIf
    EndIf
    *Len\i + 1
    *String + SizeOf(Character)
  Wend
EndProcedure

Define String$, Start, I
OpenConsole()

String$ = "Hello  World !"
String$ = ReplaceString(String$, " ", Chr(0))
PrintN("ReplaceString(): " +#TAB$+ String$ + " - Len(" + Len(String$) + ")")

String$ = "Hello  World !"
ReplaceChar(@String$, ' ', '')
;PrintN("ReplaceChar(): " +#TAB$+ String$ + " - Len(" + Len(String$) + ")")   ; UnComment to see the mess if done before PeekS()! 
String$ = PeekS(@String$)
PrintN("ReplaceChar(): " +#TAB$+#TAB$+ String$ + " - Len(" + Len(String$) + ")")  

String$ = "Hello  World !"
ReplaceCharNew(@String$, ' ', '')
;PrintN("ReplaceCharNew(): " +#TAB$+ String$ + " - Len(" + Len(String$) + ")")   ; UnComment to see the mess if done before PeekS()! 
String$ = PeekS(@String$)
PrintN("ReplaceCharNew(): " +#TAB$+ String$ + " - Len(" + Len(String$) + ")")

String$ = "Hello  World !"
ReplaceCharNewLen(@String$, ' ', '')
PrintN("ReplaceCharNewLen(): " +#TAB$+ String$ + " - Len(" + Len(String$) + ")")

CompilerIf Not #PB_Compiler_Debugger
  
  PrintN("")
  Start = ElapsedMilliseconds()
  For I = 1 To 200000
    String$ = "Hello World !"
    String$ = ReplaceString(String$, " ", "")
  Next
  PrintN("ReplaceString(): " +#TAB$+ Str(ElapsedMilliseconds() - Start) + " ms")
  
  Start = ElapsedMilliseconds()
  For I = 1 To 200000
    String$ = "Hello World !"
    ReplaceChar(@String$, ' ', '')
    String$ = PeekS(@String$)
  Next
  PrintN("ReplaceChar(): " +#TAB$+#TAB$+ Str(ElapsedMilliseconds() - Start) + " ms")
  
  Start = ElapsedMilliseconds()
  For I = 1 To 200000
    String$ = "Hello World !"
    ReplaceCharNew(@String$, ' ', '')
    String$ = PeekS(@String$)
  Next
  PrintN("ReplaceCharNew(): " +#TAB$+ Str(ElapsedMilliseconds() - Start) + " ms")  
  
  Start = ElapsedMilliseconds()
  For I = 1 To 200000
    String$ = "Hello World !"
    ReplaceCharNewLen(@String$, ' ', '')
  Next
  PrintN("ReplaceCharNewLen(): " +#TAB$+ Str(ElapsedMilliseconds() - Start) + " ms")
  
CompilerEndIf

PrintN("")
PrintN("Presses the Return key to Close")
Input()