[Implemented] Unrestricted strings

[geoff]

Following a discussion in the beginners section.

It would be useful to have strings and string arrays that could contain any character and be any length.

So a$=b$+c$ would work for 200k strings containing nulls.

This would be especially useful when you cannot control the character values in the strings, for example when they are encrypted or compressed.

In some Basics you have a string descriptor table that contains pointers to the strings in memory and a length variable so you don't need a null terminator. Nulls can be appended when you pass variables to an API routine.

[Edwin Knoppert]

String descriptors are NEVER passed to winapi.

Each allocation via Windows memory management will add extra byte(s).
But does not need to be empty unless you specified an option to clear mem.
(See HeapAlloc())

If you are not sure add extra bytes(2) having chr(0)
You can pass the pointer to the winapi call safely this way.
much api functions search for the terminator chr(0) but several require a length.
Both are dead easy, strings (BSTR) are only handled by very specific API's which you can forget.

So if you have a compressiong unit like zlib, use HeapAlloc() or sort of for memory.
When compression is done, use a native string var or so to obtain in parts.

[Edwin Knoppert]

You can recognize functions searching for chr(0) when they declared a lpsz... or similar (sz)
But it's best to read entire part.

[aszid]

i would also like to see this feature, however i believe it would be best to make it a new type of variable. otherwise the way strings are handled would have to be totally re-written. i believe that these unrestricted strings would likely not be as fast and/or memory friendly as normal strings, thus i would hate to see "standard" strings suffer for that sake.

I can think of many uses for unrestricted strings, many things that would currently be a real pain to do with purebasic. but as i said, i believe it would be better to keep them seperate from the normal strings.

the main benefits of having them as a new type rather than "upgrading" the current strings are backwards compatability and memory requirments.

backwards compatability - this way i wouldn't need to rewrite anything.

memory - with how the "new way" was described, it sounds like it would take more memory than normal strings, not a big deal with just a few strings, but i use arrays quite often, and they can get rather big.

i vote that the new string type (if ever made) should be called "data"

anyhow, that's just my 1½ cents

[Flype]

did you know that strings are limited to 63999 bytes (pb 3.72)
I had a silly bug cause of this limitation.
In the doc it's specified 65536 bytes :roll:

And i think that strings should be limited to 2^32 and no 2^16

[Karbon]

Not *technically* true, Flype..

Check this thread out.. viewtopic.php?t=6900

Strings can technically hold 64k bytes however most of the string handling functions (even the operators like + ) are internally buffered at 63999 bytes. This internal buffering is what limits string operations and such to 63999 bytes but you're mostly right, strings are limited to 63999 bytes if you want to use them for almost anything..

(edit) Technically they're buffered internally at 64000 bytes, but you have to account for a NULL in whatever you poke in there so your usable size is 63999 bytes.. (/edit)

I agree the docs need to be updated to reflect this - I'll try to submit a patch later today..