Page 1 of 1
Large Arrays and memory question
Posted: Sat Jan 20, 2024 11:45 am
by charvista
I am wondering if creating a large integer or string Array will occupy a large amount of memory, when it is empty ? (or almost empty)
Say I am doing Dim A.s(1000000) and I put very few elements in it, will this large array be memory consuming ?
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 12:06 pm
by idle
It's going to be the num dimensions x size of Integer plus the strings you set
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 12:54 pm
by Caronte3D
Maybe some ReDim use, would be good here
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 12:59 pm
by jacdelad
charvista wrote: Sat Jan 20, 2024 11:45 am
I am wondering if creating a large integer or string Array will occupy a large amount of memory, when it is empty ? (or almost empty)
Say I am doing Dim A.s(1000000) and I put very few elements in it, will this large array be memory consuming ?
The array always consumes the same amount of memory, no matter how many elements you actually use. Plus, like idle said, more memory if you use strings or structures.
You should maybe use a list or an array?
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 2:15 pm
by charvista
Thank you for your replies.
jacdelad wrote: Sat Jan 20, 2024 12:59 pm
The array always consumes the same amount of memory, no matter how many elements you actually use. Plus, like idle said, more memory if you use strings or structures.
You should maybe use a list or an array?
So I understand that creating an array with a huge dimension will not affect the memory or the processor, only a huge amount of data might have a bad impact.
That's good to know, because my intention is that when something comes at element number 468'501 for example, then I'm doing A.s(468501)="something". But only a few of them. So it is safe doing so. I was only afraid for the huge array size!!
Live long and prosper!
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 2:31 pm
by infratec
I don't know if you understand the answer correctly.
In your eample, it is an array of strings.
A string is in this case a pointer to the memory where the data is stored.
So at time of creation your array needs the size of the elements multipied by the size of a pointer (x86 = 4, x64 = 8 bytes)
If you fill a string, then the size needed is increased only by the byte size of this string.
This happens for each string you fill with something.
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 2:36 pm
by mk-soft
Are there really all 1000000 elements or are there many gaps?
Otherwise maps also work ...
Code: Select all
Global NewMap myMap.s()
Element = 146501
AddMapElement(myMap(), Str(Element))
myMap() = "Hello"
If FindMapElement(myMap(), Str(Element))
Debug myMap()
EndIf
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 2:37 pm
by AZJIO
Compile, open the task manager and see the amount of memory used. It is not difficult to test and understand the behavior of the array.
Code: Select all
; Define Dim Arr(10000000) ; 76.7 Mb
Define Dim Arr(1000000) ; 8 Mb (your example. 8 * 1000000 = 8000000 bytes)
; Define Dim Arr(500000) ; 4.2 Mb
; Define Dim Arr(50000) ; 0.8 Mb
; Define Dim Arr(5000) ; 0.4 Mb
; Define Dim Arr(500) ; 0.4 Mb
; Define Dim Arr(50) ; 0.4 Mb
; Define Dim Arr(5) ; 0.4 Mb
Delay(10000)
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 4:32 pm
by charvista
Now I understand better, thank you.
I'm running on x64 machine, so if I create an array with 1 million elements, it creates 1 million indexes, 8 bytes each.
I tested the example from AZJIO, I have the same memory results, so it takes 8 Mb upon creation of the array.
AZJIO, I was wondering why you use "Define" in front of a "Dim" ? The Define is not necessary to declare arrays, only variables...
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 5:23 pm
by mk-soft
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 6:18 pm
by AZJIO
charvista wrote: Sat Jan 20, 2024 4:32 pm
why you use "Define" in front of a "Dim" ?
Dim, NewList, NewMap define what the variable is, but do not define the scope. Global, Define, Protected - define the scope of visibility.
Define can also be used with arrays, the lists and the maps.
New maps are always locals by default, so Global or Shared commands have to be used if a map declared in the main source need to be used in procedures.
The
new list are always locals, which means Global or Shared commands have to be used if a list declared in the main source need to be used in procedures.
The
new arrays are always locals, which means Global or Shared commands have to be used if an array declared in the main source need to be used in procedures.
mk-soft wrote: Sat Jan 20, 2024 5:23 pm
Use always ...
I used EnableExplicit, and the compiler does not complain about the lack of "Global/Define". PB 6.04
Re: Large Arrays and memory question
Posted: Sat Jan 20, 2024 9:18 pm
by DarkDragon
AZJIO wrote: Sat Jan 20, 2024 6:18 pmI used EnableExplicit, and the compiler does not complain about the lack of "Global/Define". PB 6.04
EnableExplicit enforces a declaration. For normal variables you need Define/Global/Protected/Shared/Static, but for lists, arrays and maps the NewXXX keywords are enough hint for being a declaration in which case the default scope as with Define is being used. It doesn't enforce the scope being declared explicitly, it enforces the declaration of the variable at all.
Re: Large Arrays and memory question
Posted: Sun Jan 21, 2024 11:39 am
by charvista
Interesting and good to know.
Code: Select all
EnableExplicit
Define.s a,b,c,Dim d(3),e,f
d(0)=8 ; will raise an error, since d() was declared as string array