PureBasic Forums - English

Posted: **Wed Mar 27, 2024 4:50 pm**

PureBasic allows many type variables which can be standard integers, float, double, quad and char numbers or even string characters. Here is the list of the native supported types and a brief description :
...
Name.......Extension...Memory consumption......Range
String........s............ string length + 1...........unlimited
...

In the Help file, under 'Variables', we can read that the length of a String is unlimited.
Then, why does then the following code return an [ERROR] Invalid memory access. (write error at address 39788544) ?

Code: Select all

p.s=Space(18306000000)
Debug Len(p.s)

Yes, it is more than 18 billion bytes (billion in American English, it is 18 milliards in French, Dutch, German, Italian,...)...

Posted: **Wed Mar 27, 2024 5:28 pm**

charvista wrote: Wed Mar 27, 2024 4:50 pm Yes, it is more than 18 billion bytes (billion in American English, it is 18 milliards in French, Dutch, German, Italian,...)...

No, it is unicode, so 18 billion characters need 36 GB RAM, do you have more than 36 GB RAM?

Posted: **Wed Mar 27, 2024 5:32 pm**

charvista wrote: Wed Mar 27, 2024 4:50 pm
Code: Select all
p.s=Space(18306000000)
Debug Len(p.s)
Yes, it is more than 18 billion bytes (billion in American English, it is 18 milliards in French, Dutch, German, Italian,...)...

18306000000 = 18306000000 Unicode chars at 2 bytes a piece = 36612000000 bytes = 36612000 KB = 36120 MB = 36.12 GB.

Do you have 34.12 GB of free memory to create the string?

The string length is unlimited but your RAM is limited.

Posted: **Wed Mar 27, 2024 5:46 pm**

I have 64 Gb of RAM....

Posted: **Wed Mar 27, 2024 5:54 pm**

As you can see in his signature he has 64 GiB of RAM. This does not mean it is free but at least it is there.

However this only works if the program gets compiled with 64 bit compiler.

I tested it on my system where I've got 47 GiB of free RAM but the program crashed before it had allocated any memory.
It worked with half the characters (9153000000).

But then I tried this code:

Code: Select all

;       GGGMMMkkkBBB 
#size =   9500000000
p.s=Space(#size)
MessageRequester("Size", Str(Len(p.s)))

And for some reason the output was: 910065408

That means only 9,6% of the requested characters were there.

So what's the problem? Space() or Len()? I think it is Space(). Since Space() should allocate enough memory and also fill it with spaces the memory really should be consumed by the process. But I never saw the process acquiring so much memory. There was just a little bump in my memory graph.

When I try 10.000.000.000 characters the process just crashes.

Posted: **Wed Mar 27, 2024 6:42 pm**

Nevertheless, even if not enough space is possible to allocate for the string, Space() should not to generate an IMA, but just return an empty string.

Posted: **Wed Mar 27, 2024 7:02 pm**

@NicTheQuick
I tried your code, and I get the exact same result: 910065408
Demivec's and Stargåte's answer did me think about closing all running programs to free any using memory, but the problem is still there.

I tested more and found out that the maximum lies around 1'000'000'000, seen as Unicode, that's taking up only 2 GB of memory. That's very far away from the "unlimited" for a 64GB RAM under x64 processor.

Code: Select all

p.s=Space(1000000000)
Debug Len(p.s)
p.s=""
p.s=Space(1100000000)
Debug Len(p.s)
p.s=""

The first Debug will work, the second one crashes.

Posted: **Wed Mar 27, 2024 7:38 pm**

I think unlimit means a meaningful range.
So more than 64 kb and less than 1 mb with which the string management can also process well.

Anything in the gigabyte range should be processed directly with memory areas as ArrayOfChar.

Code: Select all


#GByte = 1024 * 1024 * 1024

Structure ArrayOfChar
  c.c[0]
EndStructure

size = 20 * #GByte

*mem.ArrayOfChar = AllocateMemory(size, #PB_Memory_NoClear)

If *mem
  Debug "Size = " + Str(MemorySize(*mem) / #GByte)
  FreeMemory(*mem)
Else
  Debug "Out of memory"
EndIf

Posted: **Wed Mar 27, 2024 8:18 pm**

mk-soft wrote: Wed Mar 27, 2024 7:38 pm I think unlimit means a meaningful range.
So more than 64 kb and less than 1 mb with which the string management can also process well.

Anything in the gigabyte range should be processed directly with memory areas as ArrayOfChar.
Code: Select all
<...>

Actually your code does nothing meaningful. I mean, yes, the memory is going to be allocated but if you don't write anything to it it will not be consumed and you won't see a spike in your memory usage.
Instead use FillMemory() and a MessageRequester() to pause the process before freeing the allocated memory again. Even without '#PB_Memory_NoClear' the memory usage will not raise because the kernel can mark memory as nulled even if it is not the case.
This works fine for me to see an actual increase in memory consumption:

Code: Select all

#GByte = 1024 * 1024 * 1024

Structure ArrayOfChar
	c.c[0]
EndStructure

size = 20 * #GByte

*mem.ArrayOfChar = AllocateMemory(size, #PB_Memory_NoClear)

If *mem
	FillMemory(*mem, size, $deadbeef, #PB_Long)
	MessageRequester("RAMeater", "Size = " + StrD(MemorySize(*mem) / #GByte, 3) + " GiB")
	FreeMemory(*mem)
Else
	MessageRequester("RAMeater", "Not enough food.")
EndIf

Posted: **Wed Mar 27, 2024 8:56 pm**

Yes, with *Memory, it is not a problem at all - below code works perfectly:

Code: Select all

#GByte = 1024 * 1024 * 1024
*mem=AllocateMemory(20 * #GByte)
Debug MemorySize(*mem)
PokeS(*mem+19999999950,"The FOOD is HERE",16,#PB_Unicode)
Delay(1000)
Debug PeekS(*mem+19999999950,16,#PB_Unicode)
FreeMemory(*mem)

But my problem was about the String variable. I believe it still has a 2GB barrier.
Because this works:

Code: Select all

p.s=Space(1000000000)
Debug Len(p.s)
q.s=Space(1000000000)
Debug Len(q.s)
r.s=Space(1000000000)
Debug Len(r.s)
s.s=Space(1000000000)
Debug Len(s.s)
t.s=Space(1000000000)
Debug Len(t.s)

And I do *not* clear the variables between them, so it is piled up in different variables.

Posted: **Wed Mar 27, 2024 10:13 pm**

And finally, even having a large *Memory is not of big help, because with WriteData(), the maximum buffer size is 2GB.
I got the PB message [ERROR] WriteData(): The maximum buffer size is 2GB.
PureBasic has obviously still barriers to 2GB...

Posted: **Wed Mar 27, 2024 11:14 pm**

The 2gb limit of WriteData() is documented - you can use multiple writes to overcome it.

Posted: **Thu Mar 28, 2024 12:19 am**

NicTheQuick wrote: Wed Mar 27, 2024 5:54 pmSo what's the problem? Space() or Len()?

Maybe try with LSet() and see? @Charvista, try this. If it works, then I guess Space() is the problem?

Code: Select all

p.s=LSet("",18306000000," ")
Debug Len(p.s)

Posted: **Thu Mar 28, 2024 12:27 am**

@pjay
Yes, that's true, it is documented. But I forgot it. Thank you for recalling me.

@BarryG
The problem is Space(). LSet() gives the same problem: [ERROR] Invalid memory access. (write error at address 39854080).

Posted: **Thu Mar 28, 2024 7:44 am**

mk-soft was right when he said "I think unlimited means a meaningful range". Unlimited should not be taken literally & is perhaps not the best terminology to use, there are always limits.

Personally, I think anyone who's using a gigabytes worth of singular string data should be taking a good long look at themselves in the mirror

PureBasic Forums - English

What is the maximum length a string variable can hold?

What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?

Re: What is the maximum length a string variable can hold?