PureBasic Forums - English

Thanks to PB Team,

I think it´s no problem with unicode.

But I need a very fast conversion between ascii and unicode.
In industry, many devices can be used with ascii protocol

I like this

@Fred
Thank you very much for one more time!

For a few months I was not online in this forum...
And now there's a new Beta with cool new features for the 3D-Engine
Dear Purebasic team, many thanks and thumbs up for you

Thank you very much!
Especially for the new 3D features.
Time to test these new additions.

Thank you very much

Thank you from heart!

Good work. Any chance for ogre terrain paging in a future update?

Is the Unicode-only mode going to affect Windows console applications?

I've just finished a console project with PB 5.42, and I had to compile it in Ascii mode because I realized that in Unicode mode error messages sent to STDERR were posing problems under certain conditions. If I used the app directly from CMD, no problems whatsoever, but if I invoked it from a batch script, redirecting its outuput, then STDERR messages would end up in the (redirected) file with NUL chars inbetween each char. So, with Unicode compilation, this approach worked fine:
but if I created a batch file with such a line: and invoked it thus:
Then the NUL inbetween chars would popup! Somehow, the problem only applied to ConsoleError().

I gather that this is because of the limited way Windows CMD handles Unicode:

CMD.exe only supports two character encodings Ascii and Unicode (CMD /A and CMD /U)

If you need full unicode support use PowerShell. There is still VERY limited support for unicode in the CMD shell, piping, redirection and most commands are still ANSI only. The only commands that work are DIR, FOR /F and TYPE, this allows reading and writing (UTF-16LE / BOM) files and filenames but not much else.
http://ss64.com/nt/chcp.html

In my case, after a few tests, I realized the problem was specific to PB's ConsoleError() function — the problem didn't show up with Print() or Print() funcs— so I ended up compiling it in Ascii mode.

With the passage to Unicode-only mode, how will PB handle such cases?

Are the console-related functionalities going to be adapted to handle such issues?

I know ... its a real bummer that Windows CMD still handles badly Unicode, especially when redirecting, and that not all of the CMD commands are Unicode compliants. Still, PowerShell is not a panacea for every situation -- and if you create a console app, chances are that people will invoke them from CMD.

I hope that coming updates of PB will address these issues, strengthening the Console library behaviour on Windows, making it true cross-platform — ie: allowing users not to worry about such issues, having one code for all OSs'.

Else, it might be a problem to continue to create console apps for Windows using PB — that is, without resorting to WinAPI libs, which would break cross-platformness of the app.

Fred wrote:You can't seek directly to a specific char, which is a problem for many string functions which requiers an offset to a char. You need to parse all the char to get the right index, which is a bottleneck when dealing with a lot of string op.

This was never a good arguments as UTF-16 has surrogate pairs and composite codepoints. Even UTF-32 does not have this guarantee.

Even if you were to limit yourself to the obsolete UCS-2 then you still have endianess to consider.

Thank you Fred for the new Beta.

All my projects compiling very well with the new version.


Greetings


NicknameFJ

My Application relies heavily on Scintilla and the Unicode stuff messes that up in a big way

aaaaaaaargh wrote:My Application relies heavily on Scintilla and the Unicode stuff messes that up in a big way

I replaced calls to scintilla with calls that manage the conversion. You really should be doing that anyway, unless you do not plan on sharing your program with anybody else, because non-ascii characters are much more common than most Americans are aware of.

Tenaja wrote:

aaaaaaaargh wrote:My Application relies heavily on Scintilla and the Unicode stuff messes that up in a big way

I replaced calls to scintilla with calls that manage the conversion. You really should be doing that anyway, unless you do not plan on sharing your program with anybody else, because non-ascii characters are much more common than most Americans are aware of.

Yeah, that is what I'll be doing soon. Well, we have the luxury of an LTS version with PB so there should be plenty of time to get those conversions done.
Many thanks to the PB Team for their continuing efforts, you rock!

No unicode -> HELLO WORLD
unicode -> chinese, and i don't speak chinese.

And it come from official base64.

supercdfr wrote:

Code: Select all

  Decoded$ = Space(1024) 
  Encoded$ = "aGVsbG8gd29ybGQ="

  Debug Base64Decoder(@Encoded$, StringByteLength(Encoded$), @Decoded$, 1024)
  Debug Decoded$

No unicode -> HELLO WORLD
unicode -> chinese, and i don't speak chinese.

And it come from official base64.

Again someone who not know how to handle that stuff.
Maybe it would be a good decission that the Decoder itself change the incoming *buffer (which is always a string) to ASCII. As hint:
Base64Decoder don't expect a string it expects a buffer with an ASCII string inside.
So you have to use Pokes(*encodebuffer, Encode$, -1, #PB_Ascii|#PB_String_NoZero)
Bernd