Page 1 of 1
Problem with Dim
Posted: Tue Jul 10, 2007 6:47 pm
by ktamp
Hello there. Using PureBasic 4.10 Beta 2 under Windows XP, I decided to test the limits of Dim as in "Dim a.b(#MaxDim)".
* Although the maximum value of #MaxDim accepted by the compiler is $fffffffe, no memory is being allocated. In fact Dim fails if #MaxDim is significantly bigger than 1,830,000,000; under normal circumstances, a value around 1,833,000,000 is possible.
* If Dim fails, the value of @a() is normally zero. The weird thing is that, if #MaxDim is anything between $7fffffff and $fffffffe, the value of @a() is non-zero even though no memory is being allocated!
Re: [4.10b2 - WinXP] Bugs in Dim
Posted: Tue Jul 10, 2007 7:04 pm
by Kaeru Gaman
ktamp wrote:... test the limits of Dim as in "Dim a.b(#MaxDim)".
* Although the maximum value of #MaxDim accepted by the compiler is $fffffffe, no memory is being allocated. In fact Dim fails if #MaxDim is significantly bigger than 1,830,000,000; under normal circumstances, a value around 1,833,000,000 is possible.
wich is absolutely correct.
with a 32bit windows XP the maximum Memory
accessable by
programs is 2GB.
the maximum
adressable by 32bit
hardware is 4GB.
since other programs need some mem, too, 1.7GB seems a realistic value to be allocated as a single memory-block.
if you allocated $fffffffe bytes, all memory would have been used for your array,
none left for OS, Ports, DMA of GraficMem etc...
so you see, it is just physically impossible, not a bug of PB.
to access more than 2GB by a program, you'll need a 64bit OS on 64bit hardware.
(and better a 64bit compiler, too)
Posted: Tue Jul 10, 2007 7:35 pm
by netmaestro
Also, PureBasic is very conservative about how much memory it uses. Pull up the CPU montior from the ide and run this test, watching the used memory at each stage. Memory usage doesn't rise until the array is actually filled:
Code: Select all
; pull up CPU monitor from debugger menu in IDE and watch used memory at each step
#max = 300000000
Debug "dimming the array with 300 million spots"
Dim a.b(#MAX) ; insignificant usage
Delay(5000)
Debug "putting 99 into last position"
a(#max) = 99 ; insignificant usage
Delay(5000)
Debug "putting a value into all positions"
For i=1 To #max
a(i) = 99 ; usage climbs
Next
Debug "all positions filled"
Debug a(#MAX)
Delay(5000)
Posted: Tue Jul 10, 2007 7:45 pm
by ABBKlaus
totally correct, but this is weird :
Code: Select all
#MAXDIM=$7FFFFFFF
Debug "" ; when debug is removed this error is shown : [Error] Array size can´t be negative.
Dim a.b(#MAXDIM)
Debug GetLastError_() ; Return 8 = ERROR_NOT_ENOUGH_MEMORY
Tested with PB4.02 and PB4.10B2
Posted: Tue Jul 10, 2007 8:28 pm
by ktamp
#MAXDIM=$7FFFFFFF
Apparently a signed long is used for array size, so $7FFFFFFF is the maximum value. But the first element of an array has index 0, so maximum index must be $7FFFFFFF - 1 = $7FFFFFFE.
Any notes about getting a non-zero pointer value using an index between 2G and 4G?
Posted: Tue Jul 10, 2007 8:48 pm
by ABBKlaus
ktamp wrote:#MAXDIM=$7FFFFFFF
Apparently a signed long is used for array size, so $7FFFFFFF is the maximum value. But the first element of an array has index 0, so maximum index must be $7FFFFFFF - 1 = $7FFFFFFE.
Any notes about getting a non-zero pointer value using an index between 2G and 4G?
But that was not the point
There is definitely a bug somewhere in the compiler, see this example :
Code: Select all
#MAXDIM=$7FFFFFFF
Debug ""
Dim a.b(#MAXDIM)
Dim b.b(#MAXDIM) ; only the second dim raised an error : [Error] Array size can´t be negative.
Debug GetLastError_() ; Return 8 = ERROR_NOT_ENOUGH_MEMORY
Edit : now i got it, PB is allocating memory for the array + 1
$7FFFFFFE + 1 is $7FFFFFFF
$7FFFFFFF +1 is $80000000 = negative !
Code: Select all
; Dim a.b($7FFFFFFE)
PUSH dword 1
PUSH dword a_a
PUSH dword 0
MOV edx,dword [a_a]
CALL SYS_FreeArray
MOV eax,2147483647
PUSH dword 1
CALL SYS_AllocateArray
; Dim b.b($FFFF)
PUSH dword 1
PUSH dword a_b
PUSH dword 0
MOV edx,dword [a_b]
CALL SYS_FreeArray
MOV eax,65536
PUSH dword 1
CALL SYS_AllocateArray
Posted: Wed Jul 11, 2007 5:40 pm
by ktamp
About 32bit systems allocating at most 2GB of memory, I've tested #MaxDim for all numeric types of PB. The allocated memory is always about 1.7GB. Anybody else noticing something weird here?
Code: Select all
Type #MaxDim
.b 1850000000
.c 1850000000
.w 920000000
.l 1530000000
.f 1530000000
.q 1840000000
.d 1840000000
Posted: Wed Jul 11, 2007 10:16 pm
by Kaeru Gaman
interesting... the return-value is wrong.
the error-return must occur far earlier.
what I explained is a hardware and OS issue.
it's impossible to allocate 1'530'000'000 Longs or 1'840'000'000 quads.
Posted: Thu Jul 12, 2007 11:37 pm
by ktamp
In my previous test Dim would always allocate about 1.7GB (the free part of the 32bit-addressable 2GB memory). The inconsistent #MaxDim results made me rethink of my method.
This time, instead of a binary search model, I've tested memory allocation in incremental steps of 10^7 bytes. #MaxDim results are consistent.
The problematic results have been caused by Dim reporting success (@a() <> 0) from some minimum value of #MaxDim and up. These values are reported in the following table:
Code: Select all
Type #MaxDim Min false #MaxDim
.b 1850000000 $FFFFFFE9
.c 1850000000 $FFFFFFE9
.w 920000000 $7FFFFFF4
.l 460000000 $3FFFFFF9
.f 460000000 $3FFFFFF9
.q 230000000 $1FFFFFFC
.d 230000000 $1FFFFFFC
Posted: Sat Jul 14, 2007 3:53 pm
by ktamp
I know I'm getting annoying, but...
* While testing quad arrays, I found that Dim(Size) uses only the last 29 bits of Size. That is $01111111, $21111111, $41111111, $61111111, $81111111, $A1111111, $C1111111 and $E1111111 are all taken as $01111111... Cool bug!
* (edit) As expected, the last 30 bits are being used for long arrays and the last 31 bits for word arrays...
* In areas of (real) "false allocation" there can be sub-areas where the compiler thinks the size of the array is negative. Byte arrays are an example of this behaviour: Size values between $FFFFFFF3 and $FFFFFFFE cause false allocation, while values below $FFFFFFF3 are considered to be negative.