Workaround FreeString Resources (Set to Nothing)

mk-soft · **Posted:** Sat Aug 17, 2019 1:50 pm

Linux x86 not tested...

Code:

;-TOP

; FreeString Helper, by mk-soft, v1.03, 18.08.2019

; *********************************************************

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  CompilerSelect #PB_Compiler_OS
    CompilerCase #PB_OS_MacOS
      Macro FreeGlobalString(_string_)
        !XOR rsi,rsi
        !LEA rdi,[v_#_string_]
        !CALL _SYS_FastAllocateStringFree4
      EndMacro
    CompilerCase #PB_OS_Windows
      Macro FreeGlobalString(_string_)
        !XOR rdx,rdx
        !LEA rcx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree4
      EndMacro
    CompilerCase #PB_OS_Linux
      Macro FreeGlobalString(_string_)
        !XOR rsi,rsi
        !LEA rdi,[v_#_string_]
        !CALL SYS_FastAllocateStringFree4
      EndMacro
  CompilerEndSelect
CompilerElse
  CompilerSelect #PB_Compiler_OS
    CompilerCase #PB_OS_MacOS
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
    CompilerCase #PB_OS_Windows
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
    CompilerCase #PB_OS_Linux
      ;TODO
      Macro FreeGlobalString(_string_)
        !XOr edx,edx
        !LEA ecx,[v_#_string_]
        !CALL SYS_FastAllocateStringFree
      EndMacro
  CompilerEndSelect
CompilerEndIf


Macro FreeStructureString(_struct_, _offset_)
  ClearStructure(_struct_ + _offset_, string)
EndMacro

; *********************************************************
;
Debug "Free global string"
a.s = "Hello World!"
p.i = 0
CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  !lea rax, [v_a]
  !mov qword [v_p], rax
CompilerElse  
  !lea eax, [v_a]
  !mov dword [v_p], eax
CompilerEndIf

Debug "Adress of gobale var (a) = " + p
Debug "Pointer to string = " + PeekI(p)
Debug "String from pointer = " + PeekS(PeekI(p))
Debug "----"
;a = #Null$ ; not work with PB v5.7x
FreeGlobalString(a)
Debug "Pointer to string after free global string = " + PeekI(p)

Debug "********"
Debug "Free single string inside structure"

Structure People
  Name$
  LastName$
  City.s
  Age.l
EndStructure

Student.People\Name$ = "Paul"
Student\LastName$ = "Morito"
Student\City = "New York"
Student\Age = 10
Debug "Data 1"
Debug Student\Name$
Debug Student\LastName$
Debug Student\City
Debug Student\Age
Debug "Adress to string = " + PeekI(Student+OffsetOf(People\LastName$))
Debug "String from adress = " + PeekS(PeekI(Student+OffsetOf(People\LastName$)))
;Student\LastName$ = #Null$ ; Not work with PB v5.7x
FreeStructureString(Student, OffsetOf(People\LastName$))
Debug ""
Debug "Data 2"
Debug Student\Name$
Debug Student\LastName$
Debug Student\City
Debug Student\Age
Debug "Adress to string = " + PeekI(Student+OffsetOf(People\LastName$))

skywalk · **Posted:** Sat Aug 17, 2019 2:44 pm

Not sure I understand why I have to worry about freeing structured strings?
My use model is to either set them and read them, or set them, read them, then maybe change them.
Why does it matter if PB maintains a string buffer when I set my structured string to \""?
If I later change the \"" to another value like \"something else", does the PB string buffer adapt or stay at some larger allocation? Why do I care? When my app closes, PB frees all its internal memory.

I never understood why setting a \str = #Null$ should be different than \str = #Empty$.

Code:

Macro FreeStructureString(_struct_, _offset_)
  ClearStructure(_struct_ + _offset_, string)
EndMacro
Structure udtFoo
  iVal.i
  sVal$
EndStructure
*mem.udtFoo = AllocateMemory(SizeOf(udtFoo))
*mem\iVal = 100
*mem\sVal$ = "Hello Structured String$"
Debug "-- Before Changes --"
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- Space(0) --"
*mem\sVal$ = Space(0)
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- #Null$ --"
*mem\sVal$ = #Null$
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- #Empty$ --"
*mem\sVal$ = #Empty$
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- FreeStructureString() --"
FreeStructureString(*mem, OffsetOf(udtFoo\sVal$))
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- ClearStructure() --"
ClearStructure(*mem, udtFoo)
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"
Debug "-- Use Structure again --"
*mem\sVal$ = "Hello New Structured String$"
Debug "iVal   = " + Str(PeekI(*mem + OffsetOf(udtFoo\iVal)))
Debug "@sVal$ = " + Str(PeekI(*mem + OffsetOf(udtFoo\sVal$)))
Debug "sVal$  = " + *mem\sVal$; + "#EOS"

mk-soft · **Posted:** Sat Aug 17, 2019 3:13 pm

The problem is, if you work with very large strings, that with "a.s = #Empty$" the requested memory will not be reduced, and therefore too much memory will be lost at runtime.

P.S.
With version PB v5.62 the memory was released with "a.s = #Null$".
Since version PB v5.7x it doesn't work anymore.

Mijikai · **Joined:** Sun Sep 11, 2016 2:17 pm **Posts:** 648

Its really important to be able to handle strings properly - this cant be stressed enough.
It was working until PB 5.70 now it is broken.
Im just hoping this is all - for now i will stay with Version 5.62.
Having unpredicted behaviour and nasty memory leaks is a risk im not willing to take.
I really hope this gets fixed in the next release.

mk-soft · **Posted:** Sat Aug 17, 2019 3:38 pm

You can use the workaround wherever you use "#Null$".
For whole structures "ClearStructure" works flawlessly.

skywalk · **Posted:** Sat Aug 17, 2019 3:44 pm

I am still confused.
Why is setting \str$ = #Null$ different from \str$ = #Empty$?
They are either 0 bytes or 0 bytes + SizeOf(Character).
You are saying that PB would shrink its memory buffer to zero whenever #Null$ was assigned.
But the memory buffer stays at some maximum value if I changed \str$ = "1"? :shock:

Where is the memory leak if PB's string buffer is managed.
I thought Fred said this was done to make strings faster?
Otherwise, every string assignment will require memory allocations.

Mijikai · **Joined:** Sun Sep 11, 2016 2:17 pm **Posts:** 648

mk-soft wrote:

You can use the workaround wherever you use "#Null$".
For whole structures "ClearStructure" works flawlessly.

Thx but using a workaround for some core functionality is not for me.
And who knows what else is broken there.

mk-soft · **Posted:** Sat Aug 17, 2019 3:54 pm

@skywalk

"#Null$" is a pointer to Nothing (0)
"#Empty$" is a pointer to a string with the length zero.

chi · **Posted:** Sat Aug 17, 2019 4:04 pm

Thanks, mk-soft!

For strings inside a procedure I added a "p." in front of "v_#_string_"

Code:

Macro FreeProcString(_string_)
  !XOR rdx,rdx
  !LEA rcx,[p.v_#_string_]
  !CALL SYS_FastAllocateStringFree4
EndMacro

skywalk · **Posted:** Sat Aug 17, 2019 4:13 pm

I understand the difference of the pointers. I do not understand the memory leak?
Are you saying PB never reduces the size of the memory holding *str$?
Why do I care?

Code:

*str$ = "1"
[address of str$][1][0][0][0]
*str$ = "12"
[address of str$][1][0][2][0][0][0] <-- PB increases string buffer
*str$ = "1"
[address of str$][1][0][0][0][0][0] <-- PB maintains string buffer
                                    <-- Why is this not a BUG?
*str$ = #Empty$
[address of str$][0][0][0][0][0][0] <-- PB maintains string buffer
*str$ = #Null$
[address of str$][0][0][0][0][0][0] <-- PB BUG v571 maintains string buffer
[address of str$][0]                <-- PB v562 reduces string buffer
Why does PB not resize the string buffer for all assignments?

Mijikai · **Joined:** Sun Sep 11, 2016 2:17 pm **Posts:** 648

skywalk wrote:

I understand the difference of the pointers. I do not understand the memory leak?
...

Allocated memory not just magically dissapears beacause you fill it with zeros...

skywalk · **Posted:** Sat Aug 17, 2019 4:46 pm

I did not say it disappears. The pointer stays the same.
A memory leak occurs when the pointer changes but the prior memory is not released.
I do not see PB v571 doing this?
It is just reserving more space than the underlying string requires.

#NULL · **Posted:** Sat Aug 17, 2019 5:41 pm

I guess the problem might be, as an example, if you have many elements, all using an temporary large string, you can't get rid of them.
The memory consumption will spike after 3 (at 4), and will only drop again after 6 (at 7).

Code:

MessageRequester("","1")

NewList l.s()
n = 10000
For i=0 To n
  AddElement(l())
Next

MessageRequester("","2")

ForEach l()
  l() = "a"
Next

MessageRequester("","3")

ForEach l()
  l() = Space(100000)
Next

MessageRequester("","4")

ForEach l()
  l() = ""
Next

MessageRequester("","5")

ForEach l()
  l() = #Null$
Next

MessageRequester("","6")

ClearList(l())

MessageRequester("","7")

mk-soft · **Posted:** Sat Aug 17, 2019 6:27 pm

skywalk wrote:

I did not say it disappears. The pointer stays the same.
A memory leak occurs when the pointer changes but the prior memory is not released.
I do not see PB v571 doing this?
It is just reserving more space than the underlying string requires.

If you do everything right, there will be no memory leakage

- AllocateStructure -> FreeStructure
- AllocateMemory -> ClearMemory -> FreeMemory.

skywalk · **Posted:** Sat Aug 17, 2019 7:40 pm

Yes, so I am asking why Mijikai says PB strings are broken?

PureBasic Forum

Workaround FreeString Resources (Set to Nothing)

Who is online