PureBasic Forums - English

hope:String and Asc() and Chr() add format
[sorry, topics edited again]
in pb,string must exist to container(like Variables),I hope have control format of container

format:#PB_Ascii,#PB_UTF8,#PB_Unicode

1 Define String.s{format}
Example:
a.s{#PB_Ascii}="nbh"
b.s{#PB_Unicode}="nbh"
Bool(a<>b);=1
a is #PB__Ascii
b is #PB_Unicode

Remarks:
all "nbh" is #PB_UTF8 in IDE when code_file's format is utf8

but a.s is #PB_Ascii after compile Converts
and b.s is #PB_Unicode after compile Converts

2 and add Syntax: "string"{format}
Example:
Bool("nbh"{#PB_Ascii}<>"nbh"{#PB_Unicode})

------------
3 Result = Asc(String$)
Return the first character value of the specified string.
4 Text$ = Chr(CharacterValue,format)
Returns a string created with the given character value.
---------
5 and Syntax promise:
a.s{#PB_Ascii}="nbh";a is #PB_Ascii
b.s{#PB_Unicode}
b=a;a Converts to b,b="nbh",b is #PB_Unicode
------
6 And add:SetFormat(Format) For Gadget And EditForm
Example:
SetFormat(#PB_Unicode)
StringGadget(0,0,0,99,22,"bnm");"bnm" will is #PB_Unicode after compile Converts
ButtonGadget(1,100,0,33,22,"bnm");"bnm" will is #PB_Unicode after compile Converts
SetFormat(#PB_UTF8)
EditorGadget(2,0,30,99,62)
SetGadgetText(2,"bnm");"bnm" and EditForm will is #PB_UTF8 after compile Converts

Not forgetting String Arrays

Define Dim sMsg.s(100, #PB_UTF8)

delete

Next version or so Ascii compilation mode will be removed, so strings are
Unicode exclusively. This requests seems like the opposite, adding Utf8 strings?

@Danilo: but #PB_Ascii,#PB_UTF8,#PB_Unicode should support

Indeed yes - ASCII is still the king on millions of Servers because the likes of PHP do not have adequate Unicode support.
See also this request: Specific Procedures ASCII or Unicode

So, if no ASCII compilation is possible, all functions that handle strings/characters need to have format flag support. Currently, there are some important omissions.

@IdeasVacuum :thanks

Disclaimer: in an argumentative mood


Current PB situation:
- 2 string modes (ASCII and Unicode)
- team has 2 versions of all* functions to maintain
- all "in-program" strings are in the same format (ASCII or Unicode)
- methods exist to convert ASCII/Unicode/UTF-8 when needed (file I/O, program I/O, memory, prototypes, ...)

* all functions which process strings

Coming soon PB situation:
- 1 string mode (Unicode only)
- team has 1 version of all* functions to maintain
- all "in-program" strings are in the same format (Unicode)
- even more methods to convert ASCII/Unicode/UTF-8 when needed

Your request (if I understand correctly):
- 1 "universal" string mode
- team has 1 version of all* functions to maintain
- "in-program" string variables can be in different encodings
- every string variable must store what encoding it's in
- every function that processes strings must be "aware of" and convert 3 different encodings
- simple operations like concatenation and comparison will need to convert encodings

This means every string variable will have to hold some metadata, and every string function will get 2-3x larger?

Define Dim sMsg.s(100, #PB_UTF8)

That syntax wouldn't work! #PB_UTF8 is just a constant with the value 2.
So your example is the same as Dim sMsg.s(100, 2)

SetFormat(#PB_Unicode)
StringGadget(0,0,0,99,22,"bnm");"bnm" is #PB_Unicode
ButtonGadget(1,100,0,33,22,"bnm");"bnm" is #PB_Unicode
SetFormat(#PB_UTF8)
EditorGadget(2,0,30,99,62)
SetGadgetText(2,"bnm");"bnm" and EditForm will is #PB_UTF8

I don't understand.
The text (such as "bnm") should display the same in Unicode or UTF-8.
The OS will store the text internally in its own choice format (UTF-16 maybe?)
(Also this would make GUI refactoring more bug-prone.)

in pb,string must exist to container(like Variables),I hope have control format of container

Well, it's your right to request it.

I see it like kenmo, and use Unicode mode as default since it's available (some years).

delete

I see it like kenmo, and use Unicode mode as default since it's available (some years).

I have always used Unicode mode so that my apps can easily support multiple languages. However, if an app needs to process data from or interact with embedded software (industrial machinery etc) or Web Servers using PHP, you need ASCII too, no sensible way round it. Currently, PB does not have enough support for ASCII when the main code is Unicode.

sorry, topics edited again

That syntax wouldn't work! #PB_UTF8 is just a constant with the value 2.

I agree it's a poor choice of construct but of course it would work if the second parameter was designated as a format flag.

Where are the PB functions for converting strings and chars between the formats? I think they should have made it into the latest release because they are already needed. I can see it is better than every string related function having two versions from a compiler perspective, but that does not alter the fact that there are occasions where apps need to handle multi-format in the same code, which is true now, before the upcoming change.

Array of UTF8 strings in a Unicode program

it would work if the second parameter was designated as a format flag.

The second (and third, fourth, ...) parameters are already used for multi-dimensional arrays.

Where are the PB functions for converting strings and chars between the formats? I think they should have made it into the latest release because they are already needed.

Currently a PB program is like a box, in which all string variables are in one encoding and processed the same. But to communicate outside (files, memory, network, etc.) there are already many ways to convert
-- and I assume the team and community will add more functions for convenience sake (I have my own includefile called StringHelper).

File I/O: you can read/write strings & chars in 3 formats
Program I/O: you can read/write strings in 3 formats
Memory: you can read/write strings in 3 formats
Network I/O: you can send a string in 3 formats (though it's common to build a packet in memory, then SendNetworkData)
API/interfaces: you can use pseudotypes to write strings in 3+ formats (though returned strings require a PeekS() in the appropriate format)
DataSections: ... well, I wish we could specify the string format since it's just binary data built at compile-time
(more cases?)

That UTF-8 array link is a good example of what I consider a program-specific problem that deserves a program-specific solution, not worth adding complexity to all the general PB string functions.
(Also, that example doesn't work 100% in PB's ASCII mode, since 8-bit ASCII text passed to an API function expecting UTF-8 can result in invalid UTF-8 sequences...)