Problem with Dim

[ktamp]

Hello there. Using PureBasic 4.10 Beta 2 under Windows XP, I decided to test the limits of Dim as in "Dim a.b(#MaxDim)".

* Although the maximum value of #MaxDim accepted by the compiler is $fffffffe, no memory is being allocated. In fact Dim fails if #MaxDim is significantly bigger than 1,830,000,000; under normal circumstances, a value around 1,833,000,000 is possible.

* If Dim fails, the value of @a() is normally zero. The weird thing is that, if #MaxDim is anything between $7fffffff and $fffffffe, the value of @a() is non-zero even though no memory is being allocated!

[Kaeru Gaman]

... test the limits of Dim as in "Dim a.b(#MaxDim)".

* Although the maximum value of #MaxDim accepted by the compiler is $fffffffe, no memory is being allocated. In fact Dim fails if #MaxDim is significantly bigger than 1,830,000,000; under normal circumstances, a value around 1,833,000,000 is possible.

wich is absolutely correct.

with a 32bit windows XP the maximum Memory accessable by programs is 2GB.
the maximum adressable by 32bit hardware is 4GB.
since other programs need some mem, too, 1.7GB seems a realistic value to be allocated as a single memory-block.

if you allocated $fffffffe bytes, all memory would have been used for your array,
none left for OS, Ports, DMA of GraficMem etc...

so you see, it is just physically impossible, not a bug of PB.

to access more than 2GB by a program, you'll need a 64bit OS on 64bit hardware.
(and better a 64bit compiler, too)

[netmaestro]

Also, PureBasic is very conservative about how much memory it uses. Pull up the CPU montior from the ide and run this test, watching the used memory at each stage. Memory usage doesn't rise until the array is actually filled:

Code: Select all

; pull up CPU monitor from debugger menu in IDE and watch used memory at each step
#max = 300000000
Debug "dimming the array with 300 million spots"
Dim a.b(#MAX) ; insignificant usage
Delay(5000)
Debug "putting 99 into last position"
a(#max) = 99 ; insignificant usage
Delay(5000)
Debug "putting a value into all positions"
For i=1 To #max
  a(i) = 99 ; usage climbs 
Next
Debug "all positions filled"
Debug a(#MAX)
Delay(5000)

[ABBKlaus]

totally correct, but this is weird :

Code: Select all

#MAXDIM=$7FFFFFFF
Debug "" ; when debug is removed this error is shown : [Error] Array size can´t be negative.
Dim a.b(#MAXDIM)
Debug GetLastError_() ; Return 8 = ERROR_NOT_ENOUGH_MEMORY

Tested with PB4.02 and PB4.10B2

[ktamp]

#MAXDIM=$7FFFFFFF

Apparently a signed long is used for array size, so $7FFFFFFF is the maximum value. But the first element of an array has index 0, so maximum index must be $7FFFFFFF - 1 = $7FFFFFFE.

Any notes about getting a non-zero pointer value using an index between 2G and 4G?

[ABBKlaus]

#MAXDIM=$7FFFFFFF

Apparently a signed long is used for array size, so $7FFFFFFF is the maximum value. But the first element of an array has index 0, so maximum index must be $7FFFFFFF - 1 = $7FFFFFFE.

Any notes about getting a non-zero pointer value using an index between 2G and 4G?

But that was not the point

There is definitely a bug somewhere in the compiler, see this example :

Code: Select all

#MAXDIM=$7FFFFFFF
Debug ""
Dim a.b(#MAXDIM) 
Dim b.b(#MAXDIM) ; only the second dim raised an error : [Error] Array size can´t be negative. 
Debug GetLastError_() ; Return 8 = ERROR_NOT_ENOUGH_MEMORY

Edit : now i got it, PB is allocating memory for the array + 1
$7FFFFFFE + 1 is $7FFFFFFF
$7FFFFFFF +1 is $80000000 = negative !

Code: Select all

; Dim a.b($7FFFFFFE) 
  PUSH   dword 1
  PUSH   dword a_a
  PUSH   dword 0
  MOV    edx,dword [a_a]
  CALL   SYS_FreeArray
  MOV    eax,2147483647
  PUSH   dword 1
  CALL   SYS_AllocateArray
; Dim b.b($FFFF) 
  PUSH   dword 1
  PUSH   dword a_b
  PUSH   dword 0
  MOV    edx,dword [a_b]
  CALL   SYS_FreeArray
  MOV    eax,65536
  PUSH   dword 1
  CALL   SYS_AllocateArray

[ktamp]

About 32bit systems allocating at most 2GB of memory, I've tested #MaxDim for all numeric types of PB. The allocated memory is always about 1.7GB. Anybody else noticing something weird here?

Code: Select all

Type  #MaxDim
.b    1850000000
.c    1850000000
.w     920000000
.l    1530000000
.f    1530000000
.q    1840000000
.d    1840000000

[Kaeru Gaman]

interesting... the return-value is wrong.
the error-return must occur far earlier.
what I explained is a hardware and OS issue.
it's impossible to allocate 1'530'000'000 Longs or 1'840'000'000 quads.

[ktamp]

In my previous test Dim would always allocate about 1.7GB (the free part of the 32bit-addressable 2GB memory). The inconsistent #MaxDim results made me rethink of my method.

This time, instead of a binary search model, I've tested memory allocation in incremental steps of 10^7 bytes. #MaxDim results are consistent.

The problematic results have been caused by Dim reporting success (@a() <> 0) from some minimum value of #MaxDim and up. These values are reported in the following table:

Code: Select all

Type  #MaxDim     Min false #MaxDim
.b    1850000000  $FFFFFFE9
.c    1850000000  $FFFFFFE9
.w     920000000  $7FFFFFF4
.l     460000000  $3FFFFFF9
.f     460000000  $3FFFFFF9
.q     230000000  $1FFFFFFC
.d     230000000  $1FFFFFFC

[ktamp]

I know I'm getting annoying, but...

* While testing quad arrays, I found that Dim(Size) uses only the last 29 bits of Size. That is $01111111, $21111111, $41111111, $61111111, $81111111, $A1111111, $C1111111 and $E1111111 are all taken as $01111111... Cool bug!

* (edit) As expected, the last 30 bits are being used for long arrays and the last 31 bits for word arrays...

* In areas of (real) "false allocation" there can be sub-areas where the compiler thinks the size of the array is negative. Byte arrays are an example of this behaviour: Size values between $FFFFFFF3 and $FFFFFFFE cause false allocation, while values below $FFFFFFF3 are considered to be negative.