[Strings] Enable simultaneous support of Ascii & Unicode

[skywalk]

I have needs for simultaneous support of both Ascii and Unicode.
For network and hardware communication I use Ascii.
For gui's I prefer to use Unicode.
But, if I want both, I am forced to deal with manual "memory strings" which is a pain.
Why can't we have?

Code: Select all

Define Ascii$ = "123"
Define Unicode$$ = "123"

[Little John]

Cool idea!

[User_Russian]

I agree.
It would be great if maintained to several types of string (ASCII, UTF-8, Unicode, bstr, etc.).

[SFSxOI]

What a great idea! I like it.

[Danilo]

I have needs for simultaneous support of both Ascii and Unicode.
For network and hardware communication I use Ascii.
For gui's I prefer to use Unicode.
But, if I want both, I am forced to deal with manual "memory strings" which is a pain.
Why can't we have?

Code: Select all

Define Ascii$ = "123"
Define Unicode$$ = "123"

The idea is good in general, but your example looks like you used the other PB, where the developers always
added more special type-specifier characters.
% = Integer, & = Long Integer, && = Quad Integers, ? = Byte, ?? = Word, ??? = Double Word, ! = Single Float, # = Double Float, ## = Extended Float, @ = Currency, @@ = Extended Currency

Ugly as hell to read or write "w??? = 12 : f## = 1.02## : q&& = 123" or something like that, so please don't do that to PureBasic!

$ is only a shortcut for .s, so $$ would be a shortcut for .ss?
Compiled in UNICODE mode, default $ and .s are UNICODE, $$ and .ss are ASCII?
Compiled in ASCII mode, default $ and .s are ASCII, $$ and .ss are UNICODE?
.as = ASCII string?
.us = UNICODE string?

$$$ would be UTF8 string? Same as .sss? Better .u8s?
$$$$ could be BSTR type. Same for .ssss.

It needs for sure some more thoughts how to do it correctly and beautiful, too.

So let's talk about it, let the ideas flow!


Beside that feature request:
A start for yourself would be to write some more MemoryString functions, relying on existing PB functions to make your life easier.
Use PokeS, PeekS, CompareMemoryString, MemoryStringLength and everything with #PB_ASCII and #PB_UNICODE and #PB_UTF8
and write some functions for converting, concatenation, comparing mixed memory string types etc.
Something like a StringBuilder for ASCII, UNICODE, and UTF8. So you can build mixed strings in memory more easily, with some
functions for manipulating it.

[skywalk]

Define ASCII$ or Define UNICODE$$ was only a suggested syntax.
I was thinking Fred wouldn't like the idea of multi-character native types?
Instead of Define UNICODE$$, it could be Define UNICODE.t

I also hate decorating variables, but 'SomeString$' is actually helpful to me and not too invasive.
Like I mentioned, I already have memory functions for string manipulation as these are mandatory for speed.
PB's concatenation is only useful for very small strings.

My dilemma is compiling in ASCII, but updating a PBGadget with a Unicode string?
I don't know how to do that or if it's possible?
Obviously, I could compile in Unicode, but that negatively impacts the speed and size of my exe and data.
There are very few Unicode specific operations as most are GUI updates.

[Danilo]

My dilemma is compiling in ASCII, but updating a PBGadget with a Unicode string?
I don't know how to do that or if it's possible?

It would mean compiling PB source with ASCII, and then call the 'W' functions for all GUI commands (in Windows, that is), giving UNICODE strings to this functions only.
MessageBoxA -> MessageBoxW, CreateWindowExA -> CreateWindowExW, etc.

[STARGÅTE]

I have needs for simultaneous support of both Ascii and Unicode.
For network and hardware communication I use Ascii.
For gui's I prefer to use Unicode.
But, if I want both, I am forced to deal with manual "memory strings" which is a pain.
Why can't we have?

Code: Select all

Define Ascii$ = "123"
Define Unicode$$ = "123"

Then use Unicode and if you send a network string use: SendNetworkString(Connection, String, #PB_Ascii)
and if you receive something, you have to always use PeekS(), now with #PB_Ascii.

[skywalk]

My dilemma is compiling in ASCII, but updating a PBGadget with a Unicode string?
I don't know how to do that or if it's possible?

It would mean compiling PB source with ASCII, and then call the 'W' functions for all GUI commands (in Windows, that is), giving UNICODE strings to this functions only.
MessageBoxA -> MessageBoxW, CreateWindowExA -> CreateWindowExW, etc.

Yeah, I meant using PB Gadget/Window commands. It would be a lot of work for me to rewrite my gui.pbi.

@STARGÅTE, All my internal functions are Unicode aware, but compiling in Unicode creates subtle issues for me. As Danilo suggests, Ascii compiles force me to write OS specific gui calls to expose Unicode objects. It would be great to have both at my disposal natively.

[IdeasVacuum]

+1 for the feature request

My apps are all Unicode. I sometimes need to gather info about the graphics card's OpenGL capability, and the OpenGL function, glGetString_(), returns ASCII strings. So, the code below only works if the app is compiled in ASCII mode. Is it possible to redefine the code to work in a Unicode app?

Code: Select all

Global sgGlVersion.s, sgGlVendor.s, sgGlRender.s, sgGlExtn.s

Procedure GetOpenGL()
;--------------------
;NOTE OpenGL sub-system to be specified for compiler. (Cannot compile as unicode).

#GL_VENDOR     = $1F00
#GL_RENDERER   = $1F01
#GL_VERSION    = $1F02
#GL_EXTENSIONS = $1F03

Protected iWinId.i

      If InitSprite()

               iWinId = OpenWindow(0, 0, 0, 100, 100, "OpenGL Info",#PB_Window_Invisible)
               OpenWindowedScreen(iWinId,0,0,100,100,0,0,0)

                   sgGlVersion = Trim(PeekS(glGetString_(#GL_VERSION),#PB_Ascii))
                      sgGlExtn = Trim(PeekS(glGetString_(#GL_EXTENSIONS),#PB_Ascii))
                    sgGlVendor = Trim(PeekS(glGetString_(#GL_VENDOR),#PB_Ascii))
                    sgGlRender = Trim(PeekS(glGetString_(#GL_RENDERER),#PB_Ascii))

                If(sgGlVersion = "") : sgGlVersion = "Not verified" : EndIf
                If(   sgGlExtn = "") :    sgGlExtn = "Not verified" : EndIf
                If( sgGlVendor = "") :  sgGlVendor = "Not verified" : EndIf
                If( sgGlRender = "") :  sgGlRender = "Not verified" : EndIf

                CloseScreen()
                CloseWindow(0)
       EndIf

EndProcedure

GetOpenGL()

Debug "OpenGL Version: " + sgGlVersion
Debug "   OpenGL Extn: " + sgGlExtn
Debug " OpenGL Vendor: " + sgGlVendor
Debug " Graphics Card: " + sgGlRender

[Demivec]

+1 for the feature request

My apps are all Unicode. I sometimes need to gather info about the graphics card's OpenGL capability, and the OpenGL function, glGetString_(), returns ASCII strings. So, the code below only works if the app is compiled in ASCII mode. Is it possible to redefine the code to work in a Unicode app?

Your code should have worked.


@Edit: no bug, missing a parameter.

[Shield]

Nah it's not...

Code: Select all

sgGlVersion = Trim(PeekS(glGetString_(#GL_VERSION), -1, #PB_Ascii))
sgGlExtn = Trim(PeekS(glGetString_(#GL_EXTENSIONS),-1,#PB_Ascii))
sgGlVendor = Trim(PeekS(glGetString_(#GL_VENDOR),-1, #PB_Ascii))
sgGlRender = Trim(PeekS(glGetString_(#GL_RENDERER),-1, #PB_Ascii))

Better specify -1 for the length instead of the value of #PB_Ascii.

[IdeasVacuum]

Better specify -1 for the length instead of the value of #PB_Ascii

oops! Strange thing though, -1 fails for #GL_EXTENSIONS. Specifying a buffer does work, but the number of extensions that various cards support varies a bit. I'll take a look at the OpenGL docs to see if a max string size is specified.

I was hoping the code could be made to work in a Unicode app. Although the vars receiving the strings are Unicode whilst the strings themselves are ASCII, Unicode does cover the basic ASCII range. It would be great to initialise string vars as a specific type, as discussed (I would favour sMyString.sa/sMyString.su), that would be ideal.

Edit: In fact correcting the Peek typo by specifying length, the code does work compiled as Unicode. Thanks Shield!

[Little John]

This feature request has become even more important in the light of Fantaisie Software's recent announcement, that support for ASCII mode compilation will end after the next LTS cycle.

Beside that feature request:
A start for yourself would be to write some more MemoryString functions, relying on existing PB functions to make your life easier.
Use PokeS, PeekS, CompareMemoryString, MemoryStringLength and everything with #PB_ASCII and #PB_UNICODE and #PB_UTF8
and write some functions for converting, concatenation, comparing mixed memory string types etc.
Something like a StringBuilder for ASCII, UNICODE, and UTF8. So you can build mixed strings in memory more easily, with some
functions for manipulating it.

The reason why I'm now posting in this thread is, that I'd like to remind of idle's ModXString contribution, which can be very useful in this context
( as long as this is not built into PB itself ).