Dim and out of memory

[luis]

What happen when a Dim fails ?

Using memory allocation you can test the returned value for zero, is there a clean way to trap a Dim failure ?


BTW: don't know what happen if a Dim fails, probably a program crash... I didn't look at the generated code but I suppose dim-med vars are allocated on the heap, so probably could be doable something like this: if a Dim test.i(10000) fails, the array could be resized to 0 and one could test with ArraySize to see if the Dim was successful.

[Demivec]

BTW: don't know what happen if a Dim fails, probably a program crash... I didn't look at the generated code but I suppose dim-med vars are allocated on the heap, so probably could be doable something like this: if a Dim test.i(10000) fails, the array could be resized to 0 and one could test with ArraySize to see if the Dim was successful.

I have a few comments on your question, though no solutions.

An array of size 0 still takes up room for the array structure and 1 element and so would not be a great solution. Not having enough memory for an array when it is dimmed is similar to not having enough memory for any other variable, if it doesn't exist the program shouldn't run. I believe it also shares something with memory allocation in that the memory isn't used when the array is dimensioned until it is accessed for the first time.

For instance, without uncommenting the last line, this code works without error for me with Win XP and 2GB ram:

Code: Select all

Structure test
  q.q[999999999]
EndStructure


Dim a.test(999999999) ;this array would take up 7275957+ TB according to my rough calculations.

Debug a(0)\q[1] ;first element is fine
;Debug a(999999999)\q[999999998]  ;uncomment this and it will error trying to access the last element

I think it is a matter of determining how much memory is available and then dimensioning an array accordingly. Perhaps you could attempt to allocate the memory first and if it succeeds, unallocate the memory and dimension the array to a similar memory size.

[luis]

Hi Demivec!

An array of size 0 still takes up room for the array structure and 1 element and so would not be a great solution. Not having enough memory for an array when it is dimmed is similar to not having enough memory for any other variable, if it doesn't exist the program shouldn't run.

I still believe it could be a good solution. To me seem the dimensioning of an array should be similar to a memory allocation with malloc(). Maybe I don't have enough memory for a big array but I can have plenty of memory for many simple variables.

If a big malloc() can fail, would be nice for a big dim to fail the same way, and the simplest way to communicate that back to the calling program would probably be to change my (10000) to (0).

A failed dim, as a failed malloc, doesn't necessarily mean the program must quit. Maybe it can work with other smaller data (for example a smaller image) or it can choose to work with many smaller chunks of the same data, if possible.

I think it is a matter of determining how much memory is available and then dimensioning an array accordingly. Perhaps you could attempt to allocate the memory first and if it succeeds, unallocate the memory and dimension the array to a similar memory size.

I thought about that too but I dislike it, allocating memory with malloc(), if ok deallocate and allocate an array of the same size... it's quite ugly. I'm also not sure it's equivalent. And you can add the fact another program can request the same amount of data between the two memory allocations. And all of this is complicated by the use of the virtual memory by the OS.

BUT the point is: if I can get a valid result (ok/fail) from a malloc() it should be possible to use the same principle with a Dim(), depending on how PB treat the memory allocation needed for a Dim.

Anyway, yes, nothing of this seem to offer a solution to the original problem.

Thank you for the reply


EDIT: oh, just got your remark about the size of 1 item in an array. You are talking of a BIG structured single item. Yes, you are right.
Anyway I was talking about simple arrays, primitive data types. Usually I use AllocateMemory() instead of BIG arrays, but sometimes an array is more handy, especially since some PB library functions works only with arrays and not with memory areas, even if they are structured in some way.

[luis]

A simpler code snippets

Code: Select all

Dim q.q(999999999)

q(1) = 123 ; invalid memory access

it fails right away


relative code :

Code: Select all

; Dim q.q(999999999)
  PUSH   dword 13
  PUSH   dword a_q
  PUSH   dword 0
  MOV    edx,dword [a_q]
  CALL   SYS_FreeArray
  MOV    eax,1000000000
  PUSH   dword 8
  CALL   SYS_AllocateArray
; 
; q(1) = 123 ; invalid memory
  MOV    ebp,dword [a_q]
  LEA    eax,[ebp+8]
  MOV    dword [eax+4],0
  MOV    dword [eax],123
; 

Maybe CALL SYS_AllocateArray simply fails but there is no check so

Code: Select all

 
  MOV    ebp,dword [a_q]
  LEA    eax,[ebp+8]
  MOV    dword [eax+4],0
  MOV    dword [eax],123

results in the illegal memory access

[Fred]

I modified ArraySize(), so it will return -1 if the array isn't declared.

[luis]

NO! This is great!

So we could do something like

Code: Select all

Dim q.q(999999999)

If ArraySize(q()) <> -1 
    q(12345) = 123 ; all is well
Else
   ; out of mem
EndIf

?

Thank you Fred!

[Fred]

yep.

[Demivec]

Awesome!

[Little John]

Awesome!

Yes, indeed. Thanks, Fred!
And thanks to luis for raising this issue!

Regards, Little John

[Andre]

I modified ArraySize(), so it will return -1 if the array isn't declared.

Added a note + example to the Docs.

[Kaeru Gaman]

An array of size 0 still takes up room for the array structure and 1 element

does it?

To delete the content of an array and release its used memory during program flow, call Dim with array name and 0 elements.

to me this is clear:
Dim called with 0 does not resize the Array to one Element, what would make no sense since nobody would use an Array instead of a single variable...
but it is exceptionally handled to erase the Array.


@topic
really cool completion of the ArraySize function! thnx!

[luis]

Dim called with 0 does not resize the Array to one Element [...] but it is exceptionally handled to erase the Array.

Good catch, I always thought the item (0) would be present but you are probably right, make sense considering the sentence from the manual.

[lazy]
Would be nice to know for sure (without trying to understand it by myself looking into pb code / libs !)
[/lazy]