Workaround FreeString Resources (Set to Nothing)

Share your advanced PureBasic knowledge/code with the community.
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Workaround FreeString Resources (Set to Nothing)

Post by mk-soft »

Linux x86 not tested...

Code: Select all

;-TOP

; FreeString Helper, by mk-soft, v1.03, 18.08.2019

; *********************************************************

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  CompilerSelect #PB_Compiler_OS
    CompilerCase #PB_OS_MacOS
      Macro FreeGlobalString(_string_)
        !XOR rsi,rsi
        !LEA rdi,[v_#_string_]
        !CALL _SYS_FastAllocateStringFree4
      EndMacro
    CompilerCase #PB_OS_Windows
      Macro FreeGlobalString(_string_)
        !XOR rdx,rdx
        !LEA rcx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree4
      EndMacro
    CompilerCase #PB_OS_Linux
      Macro FreeGlobalString(_string_)
        !XOR rsi,rsi
        !LEA rdi,[v_#_string_]
        !CALL SYS_FastAllocateStringFree4
      EndMacro
  CompilerEndSelect
CompilerElse
  CompilerSelect #PB_Compiler_OS
    CompilerCase #PB_OS_MacOS
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
    CompilerCase #PB_OS_Windows
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
    CompilerCase #PB_OS_Linux
      ;TODO
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
  CompilerEndSelect
CompilerEndIf


Macro FreeStructureString(_struct_, _offset_)
  ClearStructure(_struct_ + _offset_, string)
EndMacro

; *********************************************************
;
Debug "Free global string"
a.s = "Hello World!"
p.i = 0
CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  !lea rax, [v_a]
  !mov qword [v_p], rax
CompilerElse  
  !lea eax, [v_a]
  !mov dword [v_p], eax
CompilerEndIf

Debug "Adress of gobale var (a) = " + p
Debug "Pointer to string = " + PeekI(p)
Debug "String from pointer = " + PeekS(PeekI(p))
Debug "----"
;a = #Null$ ; not work with PB v5.7x
FreeGlobalString(a)
Debug "Pointer to string after free global string = " + PeekI(p)

Debug "********"
Debug "Free single string inside structure"

Structure People
  Name$
  LastName$
  City.s
  Age.l
EndStructure

Student.People\Name$ = "Paul"
Student\LastName$ = "Morito"
Student\City = "New York"
Student\Age = 10
Debug "Data 1"
Debug Student\Name$
Debug Student\LastName$
Debug Student\City
Debug Student\Age
Debug "Adress to string = " + PeekI(Student+OffsetOf(People\LastName$))
Debug "String from adress = " + PeekS(PeekI(Student+OffsetOf(People\LastName$)))
;Student\LastName$ = #Null$ ; Not work with PB v5.7x
FreeStructureString(Student, OffsetOf(People\LastName$))
Debug ""
Debug "Data 2"
Debug Student\Name$
Debug Student\LastName$
Debug Student\City
Debug Student\Age
Debug "Adress to string = " + PeekI(Student+OffsetOf(People\LastName$))
Last edited by mk-soft on Sat Aug 17, 2019 7:17 pm, edited 1 time in total.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Workaround FreeString Resources

Post by skywalk »

Not sure I understand why I have to worry about freeing structured strings?
My use model is to either set them and read them, or set them, read them, then maybe change them.
Why does it matter if PB maintains a string buffer when I set my structured string to \""?
If I later change the \"" to another value like \"something else", does the PB string buffer adapt or stay at some larger allocation? Why do I care? When my app closes, PB frees all its internal memory.

I never understood why setting a \str = #Null$ should be different than \str = #Empty$.

Code: Select all

Macro FreeStructureString(_struct_, _offset_)
  ClearStructure(_struct_ + _offset_, string)
EndMacro
Structure udtFoo
  iVal.i
  sVal$
EndStructure
*mem.udtFoo = AllocateMemory(SizeOf(udtFoo))
*mem\iVal = 100
*mem\sVal$ = "Hello Structured String$"
Debug "-- Before Changes --"
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- Space(0) --"
*mem\sVal$ = Space(0)
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- #Null$ --"
*mem\sVal$ = #Null$
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- #Empty$ --"
*mem\sVal$ = #Empty$
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- FreeStructureString() --"
FreeStructureString(*mem, OffsetOf(udtFoo\sVal$))
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- ClearStructure() --"
ClearStructure(*mem, udtFoo)
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- Use Structure again --"
*mem\sVal$ = "Hello New Structured String$"
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Workaround FreeString Resources

Post by mk-soft »

The problem is, if you work with very large strings, that with "a.s = #Empty$" the requested memory will not be reduced, and therefore too much memory will be lost at runtime.

P.S.
With version PB v5.62 the memory was released with "a.s = #Null$".
Since version PB v5.7x it doesn't work anymore.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Workaround FreeString Resources

Post by Mijikai »

Its really important to be able to handle strings properly - this cant be stressed enough.
It was working until PB 5.70 now it is broken.
Im just hoping this is all - for now i will stay with Version 5.62.
Having unpredicted behaviour and nasty memory leaks is a risk im not willing to take.
I really hope this gets fixed in the next release.
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Workaround FreeString Resources

Post by mk-soft »

You can use the workaround wherever you use "#Null$".
For whole structures "ClearStructure" works flawlessly.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Workaround FreeString Resources

Post by skywalk »

I am still confused.
Why is setting \str$ = #Null$ different from \str$ = #Empty$?
They are either 0 bytes or 0 bytes + SizeOf(Character).
You are saying that PB would shrink its memory buffer to zero whenever #Null$ was assigned.
But the memory buffer stays at some maximum value if I changed \str$ = "1"? :shock:
Where is the memory leak if PB's string buffer is managed.
I thought Fred said this was done to make strings faster?
Otherwise, every string assignment will require memory allocations.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Workaround FreeString Resources

Post by Mijikai »

mk-soft wrote:You can use the workaround wherever you use "#Null$".
For whole structures "ClearStructure" works flawlessly.
Thx but using a workaround for some core functionality is not for me.
And who knows what else is broken there.
Last edited by Mijikai on Sat Aug 17, 2019 4:17 pm, edited 1 time in total.
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Workaround FreeString Resources

Post by mk-soft »

@skywalk

"#Null$" is a pointer to Nothing (0)
"#Empty$" is a pointer to a string with the length zero.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
chi
Addict
Addict
Posts: 1028
Joined: Sat May 05, 2007 5:31 pm
Location: Linz, Austria

Re: Workaround FreeString Resources

Post by chi »

Thanks, mk-soft!

For strings inside a procedure I added a "p." in front of "v_#_string_"

Code: Select all

Macro FreeProcString(_string_)
  !XOR rdx,rdx
  !LEA rcx,[p.v_#_string_]
  !CALL SYS_FastAllocateStringFree4
EndMacro
[/size]
Et cetera is my worst enemy
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Workaround FreeString Resources

Post by skywalk »

I understand the difference of the pointers. I do not understand the memory leak?
Are you saying PB never reduces the size of the memory holding *str$?
Why do I care?

Code: Select all

*str$ = "1"
[address of str$][1][0][0][0]
*str$ = "12"
[address of str$][1][0][2][0][0][0] <-- PB increases string buffer
*str$ = "1"
[address of str$][1][0][0][0][0][0] <-- PB maintains string buffer
                                    <-- Why is this not a BUG?
*str$ = #Empty$
[address of str$][0][0][0][0][0][0] <-- PB maintains string buffer
*str$ = #Null$
[address of str$][0][0][0][0][0][0] <-- PB BUG v571 maintains string buffer
[address of str$][0]                <-- PB v562 reduces string buffer
Why does PB not resize the string buffer for all assignments?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Workaround FreeString Resources

Post by Mijikai »

skywalk wrote:I understand the difference of the pointers. I do not understand the memory leak?
...
Allocated memory not just magically dissapears beacause you fill it with zeros...
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Workaround FreeString Resources

Post by skywalk »

I did not say it disappears. The pointer stays the same.
A memory leak occurs when the pointer changes but the prior memory is not released.
I do not see PB v571 doing this?
It is just reserving more space than the underlying string requires.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
#NULL
Addict
Addict
Posts: 1440
Joined: Thu Aug 30, 2007 11:54 pm
Location: right here

Re: Workaround FreeString Resources

Post by #NULL »

I guess the problem might be, as an example, if you have many elements, all using an temporary large string, you can't get rid of them.
The memory consumption will spike after 3 (at 4), and will only drop again after 6 (at 7).

Code: Select all

MessageRequester("","1")

NewList l.s()
n = 10000
For i=0 To n
  AddElement(l())
Next

MessageRequester("","2")

ForEach l()
  l() = "a"
Next

MessageRequester("","3")

ForEach l()
  l() = Space(100000)
Next

MessageRequester("","4")

ForEach l()
  l() = ""
Next

MessageRequester("","5")

ForEach l()
  l() = #Null$
Next

MessageRequester("","6")

ClearList(l())

MessageRequester("","7")


User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Workaround FreeString Resources

Post by mk-soft »

skywalk wrote:I did not say it disappears. The pointer stays the same.
A memory leak occurs when the pointer changes but the prior memory is not released.
I do not see PB v571 doing this?
It is just reserving more space than the underlying string requires.
If you do everything right, there will be no memory leakage

- AllocateStructure -> FreeStructure
- AllocateMemory -> ClearMemory -> FreeMemory.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Workaround FreeString Resources (Set to Nothing)

Post by skywalk »

Yes, so I am asking why Mijikai says PB strings are broken?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Post Reply