Memory reallocation within a sub function...

[SnowyDog]

Using 6.20 x64

Passing a pointer to an allocated memory block to a sub function which reallocates that memory block to a larger size fails when the memory pointer is passed directly as a parameter. The pointer becomes invalid with error "the specified 'MemoryID' is not valid".

This however succeeds when: -

• The sub function reallocates the block to a smaller size than originally allocated

• The pointer is passed as a member of a structure, regardless of whether the reallocation size is larger or smaller

I can understand the first case, because the reallocation requires a larger memory block, the pointer must typically be changed, so the pointer then goes out of scope as far as the calling code is concerned. The pointer remains the same when the block is reallocated to a smaller size, therefore remains in scope to the calling code.

But, I do not understand why the reallocation to a larger size in the sub function works OK when the pointer is passed as the member of a structure.

Example code below, change constant #Test between 1 and 2 for 1=passing directly, 2=passing pointer within a structure.

I'd like to know the reason for this, please. Thanks.

Code: Select all

EnableExplicit

#Test=1

#StartSize=1024
#NewSize=1048576

Structure MEMBLK
		*MemPtr
EndStructure

Declare Main()
Declare SubFunc1(*MemPtr)
Declare SubFunc2(*MemBlk.MEMBLK)

End Main()

Procedure Main()
	CompilerSelect #Test
		CompilerCase 1
			; Pointer not within structure
			Debug("Pointer not within structure")
			Protected *MemPtr
			*MemPtr=AllocateMemory(#StartSize)
			Debug("Before:"+#CRLF$+"   *MemPtr=0x"+Hex(*MemPtr,#PB_Quad)+#CRLF$+
			      "   MemorySize(*MemPtr)="+Str(MemorySize(*MemPtr)))
			SubFunc1(*MemPtr)
			Debug(" After:"+#CRLF$+"   *MemPtr=0x"+Hex(*MemPtr,#PB_Quad)+#CRLF$+
			      "   MemorySize(*MemPtr)="+Str(MemorySize(*MemPtr)))
				
		CompilerCase 2
			; Pointer within structure
			Debug("Pointer within structure")
			Protected *MemBlk.MEMBLK
			*MemBlk=AllocateStructure(MEMBLK)	
			*MemBlk\MemPtr=AllocateMemory(#StartSize)
			Debug("Before:"+#CRLF$+"   *MemBlk\MemPtr=0x"+Hex(*MemBlk\MemPtr,#PB_Quad)+#CRLF$+
			      "   MemorySize(*MemBlk\MemPtr)="+Str(MemorySize(*MemBlk\MemPtr)))
			SubFunc2(*MemBlk)
			Debug(" After:"+#CRLF$+"   *MemBlk\MemPtr=0x"+Hex(*MemBlk\MemPtr,#PB_Quad)+#CRLF$+
			      "   MemorySize(*MemBlk\MemPtr)="+Str(MemorySize(*MemBlk\MemPtr)))
			
		CompilerDefault
			Debug "Unsupported test value"
	CompilerEndSelect
	
EndProcedure

CompilerSelect #Test
	CompilerCase 1
		; Pointer not within structure
		Procedure SubFunc1(*MemPtr)
			*MemPtr=ReAllocateMemory(*MemPtr,#NewSize)
		EndProcedure
		
	CompilerCase 2
		; Pointer within structure
		Procedure SubFunc2(*MemBlk.MEMBLK)
			*MemBlk\MemPtr=ReAllocateMemory(*MemBlk\MemPtr,#NewSize)
		EndProcedure
		
CompilerEndSelect

[mk-soft]

With a structure variable you transfer the pointer to the variable content...

Simplified with a structure variable integer

Code: Select all

Procedure foo(*ByRef_Memory.integer)
  *ByRef_Memory\i = AllocateMemory(2028)
EndProcedure

Define *mem

foo(@*Mem); <- With @*mem ByRef to adress to *mem variable content
Debug MemorySize(*mem)
FreeMemory(*mem)

[idle]

also probably better to return like *MemPtr = SubFunc1(*MemPtr)
so your functions are

Code: Select all

*MemPtr = SubFunc1(*MemPtr) 
Procedure SubFunc1(*MemPtr)
	ProcedureReturn ReAllocateMemory(*MemPtr,#NewSize)
EndProcedure

[jacdelad]

Your memory pointer is given to the function, but the variable becomes a local one (*MemPtr in your function is only valid within the function, because you defined it in the procedure-line as being created and filled by the parameter -> don't use the same name for local and global variables). You can either give it a different name in the function or, like idle said, use the return value.

[SnowyDog]

Thanks for your replies.

In your example, mk-soft, you are assigning the .integer structure to the pointer - something that I wasn't even aware was possible.

I thought it was only possible to assign the built-in data types to a variable using the single-letter suffix (e.g. myvar.i) but of course assigning a type to a pointer is meaningless. Are these built-in structure types documented somewhere and if so where? I mean I can now see that there are built-in structures for all the main data types, but don't see any mention of this in the documentation.

So am I right to think that a pointer can only ever be passed by reference to a procedure either as a member of a structure, or by assigning a structure to the pointer?

Is there any way to pass a pointer by reference to a procedure without assigning a structure to it or not having it as a member in a structure that itself is passed by reference?

I have always believed that PB recognises any variable name prefixed with '*' as being a pointer, but it seems that this is wrong?

Thanks.

[jacdelad]

What exactly do you want to achieve? To me it looks like you overcomplicate the situation.

[SnowyDog]

I just want to understand how I have managed to misunderstand this fundamental concept! I have been using PB for some time and this is the first time I've run into this.

It seems to me that there is nothing in the PB help or online docs that indicates that a pointer can't simply be passed by reference to a procedure without a structure. Also, I can find no explanation that the built-in .integer struct that mk-soft demonstrated, exists.

Where should I be looking (in available documentation) to find this information, please?

[Quin]

You can't have a pointer to a stack-allocated type, because for some reason even though C can do it PB refuses. So, in order to have a pointer to an integer, long, etc., you have to use those structures you found. Don't feel bad, this is woefully confusing and rather annoying, I wish PB would just let us have pointers to ints, but one can dream. And this definitely needs documented better, I had the same question months ago after programming in PB for 6+ years.

[SnowyDog]

You can't have a pointer to a stack-allocated type, because for some reason even though C can do it PB refuses. So, in order to have a pointer to an integer, long, etc., you have to use those structures you found. Don't feel bad, this is woefully confusing and rather annoying, I wish PB would just let us have pointers to ints, but one can dream. And this definitely needs documented better, I had the same question months ago after programming in PB for 6+ years.

Thanks Quin, I'm glad I'm not the only one! It is confusing and in the absence of documentation to the contrary, it's easy to fall into the trap of assuming it works like C (or PowerBASIC as I used to use). Am now going to go back through all my code (modules shared by different apps) to update and to check for any potential problems that could arise in released apps from this revelation.

PureBasic is excellent and I'd be happy to support it by contributing to improved documentation, but given this discovery I'm not sure I have a sufficient grasp of the basics. Am now left wondering what else I've not understood correctly!

[skywalk]

Where should I be looking (in available documentation) to find this information, please?

Have you tried opening Tools-Structure Viewer[Alt+s]?

[Quin]

Where should I be looking (in available documentation) to find this information, please?

Have you tried opening Tools-Structure Viewer[Alt+s]?

Why is this always the response? Sure, that lets you see what types are available, but nowhere in the help file is it mentioned that this is how to pass stack-allocated values by reference to functions.

[SnowyDog]

Where should I be looking (in available documentation) to find this information, please?

Have you tried opening Tools-Structure Viewer[Alt+s]?

Thanks, yes - I see the built-in structures there which is great - now I know I need to use one to pass a pointer by reference to a procedure.

But agree with Quin - the documentation is missing information about this fundamental requirement in PB.

[mk-soft]

Even though I have been using PureBasic for over 20 years I have to admit that the use of pointers in the help is not sufficient.
But there are many here in the forum to fill this gap.

All basic variable type also gives a predefined structure for pointers. Integer, Quad, Long, Word, Byte, etc ...

[skywalk]

My simple answer is the forum knows how to answer what the manual is lacking. That and learn by doing. I put breadcrumb comments all throughout my code, especially after long breaks on an app.

[Demivec]

I just want to understand how I have managed to misunderstand this fundamental concept! I have been using PB for some time and this is the first time I've run into this.

It seems to me that there is nothing in the PB help or online docs that indicates that a pointer can't simply be passed by reference to a procedure without a structure. Also, I can find no explanation that the built-in .integer struct that mk-soft demonstrated, exists.

Where should I be looking (in available documentation) to find this information, please?

You do appear to have some very basic misunderstandings for the advance topic of pointers.
I've listed a few concepts below (that's all there really is). The Help file could use some more examples but most of the information is in the help file. The additional things could use some mentioning in the help file. They are known by advanced users and have been discussed in the forum but aren't really present in the help file.

The information regarding pointers is:

• Pointers' names include the '*' prefix at all times except within structures. With structures the prefix is only present in the definition of the structure. See Pointers and Memory Addresses. See Pointers and Memory Addresses; Structures

• Pointers point to memory addresses. Pointers contain Integer values, the same as Integers. This means whenever the value of a pointer is needed, this value could be held by an Integer variable. Neither Integers nor unstructured Pointers can be dereferenced directly (they can be for simple types with the Peek and Poke memory functions).
  See Pointers and Memory Addresses; Memory Library.

• Pointers can be structured (declared with a structured type) and dereferenced
  See Pointers and Memory Addresses; Structures

• Additional topic -- pointer math (increment or decrement of pointer) with structured pointers is done by memory bytes and not multiples of structure size being referenced, no Help file reference?.

• Additional topic -- comparisons between pointer values (i.e. *pointer_1>*pointer_2) should be made in light of fact that pointers are signed values even though they reference unsigned memory addresses, no Help file reference?.

• Additional topic -- Predefined structures exist for convenience in dereferencing basic variable types, no Help file reference. They could also be defined as needed, just like any other structure.

• Additional topic -- Dereferencing a pointer to a static array of length 0, no Help file reference.

To show some additional methods for accomplishing the issues of your original post I've extended your code example to include three more methods. A pointer can be passed by reference to a procedure without a structure. The problems were with how you did it. I cover a few different ways in the code below.

Code: Select all

EnableExplicit

#StartSize=1024
#NewSize=1048576

Structure MEMBLK
  *MemPtr
EndStructure

Declare Main()
Declare.i SubFunc1a(*MemPtr, newSize)
Declare.i SubFunc1b(MemPtr, newSize)
Declare.i SubFunc1c(*PtrMemPtr, newSize)
Declare SubFunc2a(*MemBlk.MEMBLK, newSize)
Declare SubFunc2b(*MemPtr.Integer, newSize)

Main()
End

Macro EvaluateReAllocate(_IntResultVar_, _IntUpdateMemVar_)
  If _IntResultVar_ = 0
    Debug "Memory reallocation failed"
  Else
    _IntUpdateMemVar_ = _IntResultVar_
    _IntResultVar_ = 0 ;value no longer needed
  EndIf
EndMacro

Macro DebugBefore(_varMemoryRefStr_, _varMemoryRefVar_)
  Debug "Before:" + #CRLF$ + "   " + _varMemoryRefStr_ + " = 0x" + Hex(_varMemoryRefVar_, #PB_Quad) + #CRLF$ +
        "   MemorySize(" + _varMemoryRefStr_ + ") =" + Str(MemorySize(_varMemoryRefVar_))
EndMacro

Macro DebugAfter(_varMemoryRefStr_, _varMemoryRefVar_)
  Debug " After:" + #CRLF$ + "   " + _varMemoryRefStr_ + " = 0x" + Hex(_varMemoryRefVar_, #PB_Quad) + #CRLF$ +
        "   MemorySize(" + _varMemoryRefStr_ + ") =" + Str(MemorySize(_varMemoryRefVar_)) + #CRLF$ 
EndMacro

Procedure Main()
  Protected *result
  
  ; Pointer not within structure
  Debug "Pointer not within structure"
  Protected *MemPtr
  *MemPtr = AllocateMemory(#StartSize)
  DebugBefore("*MemPtr",*MemPtr)
  *result = SubFunc1a(*MemPtr, #NewSize)
  EvaluateReAllocate(*result, *MemPtr)
  DebugAfter("*MemPtr",*MemPtr)
  FreeMemory(*MemPtr): *MemPtr = 0
  
  ; Integer variable containing address not within structure
  Debug "Integer variable containing address not within structure"
  *MemPtr = AllocateMemory(#StartSize)
  DebugBefore("*MemPtr",*MemPtr)
  *result = SubFunc1b(*MemPtr, #NewSize)
  EvaluateReAllocate(*result, *MemPtr)
  DebugAfter("*MemPtr",*MemPtr)
  FreeMemory(*MemPtr): *MemPtr = 0
  
  ; Pointer to a Pointer or Integer not within a structure
  Debug "Pointer to a Pointer or Integer not within a structure"
  *MemPtr = AllocateMemory(#StartSize)
  DebugBefore("*MemPtr",*MemPtr)
  *result = SubFunc1c(@*MemPtr, #NewSize)
  EvaluateReAllocate(*result, *MemPtr)
  DebugAfter("*MemPtr",*MemPtr)
  FreeMemory(*MemPtr): *MemPtr = 0
  
  ; Pointer within custom structure
  Debug "Pointer within structure"
  Protected *MemBlk.MEMBLK
  *MemBlk = AllocateStructure(MEMBLK)  
  *MemBlk\MemPtr = AllocateMemory(#StartSize)
  DebugBefore("*MemBlk\MemPtr",*MemBlk\MemPtr)
  SubFunc2a(*MemBlk, #NewSize)
  DebugAfter("*MemBlk\MemPtr",*MemBlk\MemPtr) 
  FreeMemory(*MemBlk\MemPtr): *MemBlk\MemPtr = 0
  FreeStructure(*MemBlk): *MemBlk = 0
  
  ; Pointer to Integer dereferenced with predefined structure
  Debug "Pointer to Integer dereferenced with predefined structure"
  *MemBlk = AllocateStructure(MEMBLK)  
  *MemBlk\MemPtr = AllocateMemory(#StartSize)
  DebugBefore("*MemBlk\MemPtr",*MemBlk\MemPtr)
  SubFunc2b(*MemBlk\MemPtr, #NewSize)
  DebugAfter("*MemBlk\MemPtr",*MemBlk\MemPtr)
  FreeMemory(*MemBlk\MemPtr): *MemBlk\MemPtr = 0
  
EndProcedure


; Pointer not within structure, returns value of ReAllocateMemory() attempt
Procedure.i SubFunc1a(*MemPtr, newSize)
  ProcedureReturn ReAllocateMemory(*MemPtr, newSize)
EndProcedure

; Integer variable containing address not within structure, returns value of ReAllocateMemory() attempt
Procedure.i SubFunc1b(MemPtr, newSize)
  ProcedureReturn ReAllocateMemory(MemPtr, newSize)
EndProcedure

; Pointer to a Pointer or Integer not within a structure, returns value of ReAllocateMemory() attempt
Procedure.i SubFunc1c(*PtrMemPtr, newSize)
  ProcedureReturn ReAllocateMemory(PeekI(*PtrMemPtr), newSize)
EndProcedure

; Pointer within custom structure
Procedure SubFunc2a(*MemBlk.MEMBLK, newSize)
  Protected *result
  
  *result = ReAllocateMemory(*MemBlk\MemPtr, newSize)
  EvaluateReAllocate(*result, *MemBlk\MemPtr)
EndProcedure

; Pointer to Integer dereferenced with predefined structure
Procedure SubFunc2b(*MemPtr.Integer, newSize)
  Protected *result
  
  *result = ReAllocateMemory(*MemPtr\i, newSize)
  EvaluateReAllocate(*result, *MemPtr\i)
EndProcedure[/code]