It is currently Thu May 23, 2013 8:56 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Dim and out of memory
PostPosted: Mon Oct 19, 2009 3:30 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Aug 31, 2005 11:09 pm
Posts: 2240
Location: Italy
What happen when a Dim fails ?

Using memory allocation you can test the returned value for zero, is there a clean way to trap a Dim failure ?


BTW: don't know what happen if a Dim fails, probably a program crash... I didn't look at the generated code but I suppose dim-med vars are allocated on the heap, so probably could be doable something like this: if a Dim test.i(10000) fails, the array could be resized to 0 and one could test with ArraySize to see if the Dim was successful.

_________________
[ Home ] [ My PC ] [ New to PB ? ]


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 5:51 pm 
Offline
Addict
Addict
User avatar

Joined: Mon Jul 25, 2005 3:51 pm
Posts: 2401
Location: Utah, USA
luis wrote:
BTW: don't know what happen if a Dim fails, probably a program crash... I didn't look at the generated code but I suppose dim-med vars are allocated on the heap, so probably could be doable something like this: if a Dim test.i(10000) fails, the array could be resized to 0 and one could test with ArraySize to see if the Dim was successful.


I have a few comments on your question, though no solutions.

An array of size 0 still takes up room for the array structure and 1 element and so would not be a great solution. Not having enough memory for an array when it is dimmed is similar to not having enough memory for any other variable, if it doesn't exist the program shouldn't run. I believe it also shares something with memory allocation in that the memory isn't used when the array is dimensioned until it is accessed for the first time.

For instance, without uncommenting the last line, this code works without error for me with Win XP and 2GB ram:

Code:
Structure test
  q.q[999999999]
EndStructure


Dim a.test(999999999) ;this array would take up 7275957+ TB according to my rough calculations.

Debug a(0)\q[1] ;first element is fine
;Debug a(999999999)\q[999999998]  ;uncomment this and it will error trying to access the last element


I think it is a matter of determining how much memory is available and then dimensioning an array accordingly. Perhaps you could attempt to allocate the memory first and if it succeeds, unallocate the memory and dimension the array to a similar memory size.

_________________
Image


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 6:57 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Aug 31, 2005 11:09 pm
Posts: 2240
Location: Italy
Hi Demivec!

Demivec wrote:
An array of size 0 still takes up room for the array structure and 1 element and so would not be a great solution. Not having enough memory for an array when it is dimmed is similar to not having enough memory for any other variable, if it doesn't exist the program shouldn't run.


I still believe it could be a good solution. To me seem the dimensioning of an array should be similar to a memory allocation with malloc(). Maybe I don't have enough memory for a big array but I can have plenty of memory for many simple variables.

If a big malloc() can fail, would be nice for a big dim to fail the same way, and the simplest way to communicate that back to the calling program would probably be to change my (10000) to (0).

A failed dim, as a failed malloc, doesn't necessarily mean the program must quit. Maybe it can work with other smaller data (for example a smaller image) or it can choose to work with many smaller chunks of the same data, if possible.

Demivec wrote:
I think it is a matter of determining how much memory is available and then dimensioning an array accordingly. Perhaps you could attempt to allocate the memory first and if it succeeds, unallocate the memory and dimension the array to a similar memory size.


I thought about that too but I dislike it, allocating memory with malloc(), if ok deallocate and allocate an array of the same size... it's quite ugly. I'm also not sure it's equivalent. And you can add the fact another program can request the same amount of data between the two memory allocations. And all of this is complicated by the use of the virtual memory by the OS.

BUT the point is: if I can get a valid result (ok/fail) from a malloc() it should be possible to use the same principle with a Dim(), depending on how PB treat the memory allocation needed for a Dim.

Anyway, yes, nothing of this seem to offer a solution to the original problem.

Thank you for the reply :)


EDIT: oh, just got your remark about the size of 1 item in an array. You are talking of a BIG structured single item. Yes, you are right.
Anyway I was talking about simple arrays, primitive data types. Usually I use AllocateMemory() instead of BIG arrays, but sometimes an array is more handy, especially since some PB library functions works only with arrays and not with memory areas, even if they are structured in some way.

_________________
[ Home ] [ My PC ] [ New to PB ? ]


Last edited by luis on Mon Oct 19, 2009 9:34 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 7:09 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Aug 31, 2005 11:09 pm
Posts: 2240
Location: Italy
A simpler code snippets

Code:
Dim q.q(999999999)

q(1) = 123 ; invalid memory access


it fails right away


relative code :

Code:
; Dim q.q(999999999)
  PUSH   dword 13
  PUSH   dword a_q
  PUSH   dword 0
  MOV    edx,dword [a_q]
  CALL   SYS_FreeArray
  MOV    eax,1000000000
  PUSH   dword 8
  CALL   SYS_AllocateArray
;
; q(1) = 123 ; invalid memory
  MOV    ebp,dword [a_q]
  LEA    eax,[ebp+8]
  MOV    dword [eax+4],0
  MOV    dword [eax],123
;


Maybe CALL SYS_AllocateArray simply fails but there is no check so

Code:
 
  MOV    ebp,dword [a_q]
  LEA    eax,[ebp+8]
  MOV    dword [eax+4],0
  MOV    dword [eax],123


results in the illegal memory access :?:

_________________
[ Home ] [ My PC ] [ New to PB ? ]


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 8:19 pm 
Offline
Administrator
Administrator

Joined: Fri May 17, 2002 4:39 pm
Posts: 8876
Location: France
I modified ArraySize(), so it will return -1 if the array isn't declared.


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 8:33 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Aug 31, 2005 11:09 pm
Posts: 2240
Location: Italy
NO! This is great!

So we could do something like

Code:
Dim q.q(999999999)

If ArraySize(q()) <> -1
    q(12345) = 123 ; all is well
Else
   ; out of mem
EndIf


?

Thank you Fred!

_________________
[ Home ] [ My PC ] [ New to PB ? ]


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 9:18 pm 
Offline
Administrator
Administrator

Joined: Fri May 17, 2002 4:39 pm
Posts: 8876
Location: France
yep.


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Mon Oct 19, 2009 9:48 pm 
Offline
Addict
Addict
User avatar

Joined: Mon Jul 25, 2005 3:51 pm
Posts: 2401
Location: Utah, USA
Awesome!

_________________
Image


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Tue Oct 20, 2009 8:57 am 
Offline
Addict
Addict
User avatar

Joined: Thu Jun 07, 2007 3:25 pm
Posts: 1574
Location: Berlin, Germany
Demivec wrote:
Awesome!

Yes, indeed. Thanks, Fred!
And thanks to luis for raising this issue!

Regards, Little John

_________________
Math problems?
Call 1-800-[(10x)(13i)^2]-[sin(xy)/2.362x].


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Sat Nov 28, 2009 10:29 pm 
Offline
PureBasic Team
PureBasic Team
User avatar

Joined: Fri Apr 25, 2003 6:14 pm
Posts: 909
Location: Germany (Saxony, Deutscheinsiedel)
Fred wrote:
I modified ArraySize(), so it will return -1 if the array isn't declared.


Added a note + example to the Docs. :)

_________________
Bye,
...André
(PureBasicTeam::Docs & Support - PureArea.net | Order:: PureBasic | PureVisionXP)


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Sat Nov 28, 2009 10:40 pm 
Offline
Addict
Addict
User avatar

Joined: Sun Mar 19, 2006 1:57 pm
Posts: 4835
Location: Germany
Demivec wrote:
An array of size 0 still takes up room for the array structure and 1 element

does it?

Dim Help wrote:
To delete the content of an array and release its used memory during program flow, call Dim with array name and 0 elements.


to me this is clear:
Dim called with 0 does not resize the Array to one Element, what would make no sense since nobody would use an Array instead of a single variable...
but it is exceptionally handled to erase the Array.


@topic
really cool completion of the ArraySize function! thnx!

_________________
oh... and have a nice day.


Top
 Profile  
 
 Post subject: Re: Dim and out of memory
PostPosted: Sat Nov 28, 2009 10:50 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Aug 31, 2005 11:09 pm
Posts: 2240
Location: Italy
Kaeru Gaman wrote:
Dim called with 0 does not resize the Array to one Element [...] but it is exceptionally handled to erase the Array.


Good catch, I always thought the item (0) would be present but you are probably right, make sense considering the sentence from the manual.

[lazy]
Would be nice to know for sure (without trying to understand it by myself looking into pb code / libs !)
[/lazy]

_________________
[ Home ] [ My PC ] [ New to PB ? ]


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye