PureBasic Forums - English

Posted: **Wed Aug 24, 2016 6:10 pm**

Fred wrote:What do you mean by real world application ?

I mean any application that sends, reads or processes text as it's primary role.

This is not really about encoding, it is about choice.

For a long time now PB has built unicode applications by default, it's been checked in the preferences and therefore became the default for new files. I always unchecked this, but I am quite sure many users did not. That being said, most users operate within the 'bubble' of PB's built in libs, and for simple applications that use text without needing to know how to process or parse that text themselves using pointers, this would seem to be fine, for those people.

There is also overhead. Do not unicode programs which rely heavily on embedded string tables increase the size of executables? Does not allocating strings internally using unicode use twice as much memory? Negligible perhaps, but true nevertheless. If you have an application available on your own web server, does this not mean you are using more bandwidth offering that file for download? I guess it all depends on what you are developing.

I'm not complaining for complaining's sake, I am merely defending what I consider is my right to choose, a choice I no longer have.

Posted: **Wed Aug 24, 2016 6:28 pm**

Psych wrote:I mean any application that sends, reads or processes text as it's primary role.

What about the web browser you are using to read this forum or the entire MS Office suite ?
Unicode is so important these days. If your application has to be used by people in different parts of the world and they have to be able to see or type anything in their own language, things become difficult without unicode.

Posted: **Wed Aug 24, 2016 6:51 pm**

wilbert wrote:What about the web browser you are using to read this forum, php and sql on the server, the entire MS Office suite ?
Unicode is so important these days.

From this forum page's own metadata.

Code: Select all

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />

I am sure if I examined the headers, the 'text-encoding' would be UTF8 also.

Or maybe you mean that behind the scenes the page is compiled using by a sever application written in unicode, before being converted to UTF8 before being streamed to the browser? Possibly. That is their choice. Choice is all I am defending.

One could also argue that the majority of web server hosting companies are using 64 bit processors using 64 bit server software, should PB now abandon its 32 bit compiler and make us all write 64 bit applications?

Posted: **Wed Aug 24, 2016 7:08 pm**

I think I was wrong about php. As far as I understand after some reading, it looks like it doesn't use any encoding but treats strings as raw data bytes.

I also should have read the previous page from this thread more carefully. To me, UTF8 is also unicode but I have the impression it is different to you. My whole point is that for an international oriented application, it is important that it can handle and display the unicode character set. I have no problem with Ascii itself but its use is limited. It's mainly convenient for command line utilities.

Posted: **Wed Aug 24, 2016 7:18 pm**

i think you are confusing something here.

you can read/write/send/receive strings in any format you want, including ascii. internally everything is stored and processed in unicode now, like the majority of apps has been doing for more than 17 years now. the reason why ascii support was dropped, is the same reason why microsoft dropped 16bit support. one day 32bit support will be dropped too and people should better get used to it. it just doesnt make sense writing new and modern functions, while having to support some insanly outdated technology.

i have been writing apps that could be compiled as ascii, unicode, 32bit or 64bit, for almost 10 years now. everyone who missed the chance to do the same, should blame no one but himself. you people had more than enough time to migrate your code.

also are you really complaining about memory and performance, where in todays world the avarage pc has an i5 with 16gb+ ram?

c ya,
nco2k

Posted: **Wed Aug 24, 2016 8:12 pm**

nco2k wrote:i think you are confusing something here.

I don't really think I am. Even after reading various comments, I still believe I should have the choice to write an application in ascii.

nco2k wrote:the reason why ascii support was dropped, is the same reason why microsoft dropped 16bit support.

I think the two things are unrelated. I am pretty sure it was because of limitations with addressing, not because they didn't like ascii.

nco2k wrote:i have been writing apps that could be compiled as ascii, unicode, 32bit or 64bit, for almost 10 years now. everyone who missed the chance to do the same, should blame no one but himself. you people had more than enough time to migrate your code.

Good for you. That is your choice.

nco2k wrote:also are you really complaining about memory and performance, where in todays world the avarage pc has an i5 with 16gb+ ram?

You think the average PC has 16gb RAM? Try 4 or 8. I can only assume you develop applications with little or no regard for memory usage. That is also your choice.

It would be interesting to know, historically, how PB has evolved over the years, I have only been using this compiler since the late 3.x versions, however, I can easily imagine a time PB was ascii only. The reason I say this is because the option to 'Compile as Unicode' is presented as an option on a per-source basis, it has never said 'Compile as Ascii', how long has that option been there? From the very first version? Then from later versions the option was still there, but now it checked by default on a clean install, I can easily imagine a time it wasn't the default option. This tells me that ascii compiling was the most widely used scenario, and unicode executables were added as an option, a choice for users. Now unicode is the preferred method, and that has never been in doubt, and yes, I will have to get used to it, that doesn't make my disappointment about the removal of ascii any less valid.

Posted: **Wed Aug 24, 2016 8:32 pm**

Psych wrote:It would be interesting to know, historically, how PB has evolved over the years, I have only been using this compiler since the late 3.x versions, however, I can easily imagine a time PB was ascii only.

Unicode was added in 2006 with PB 4.00 .

Posted: **Wed Aug 24, 2016 8:48 pm**

Psych wrote:It would be interesting to know, historically, how PB has evolved over the years, I have only been using this compiler since the late 3.x versions, however, I can easily imagine a time PB was ascii only.

To answer your query, PureBasic added a unicode compilation option first in version 4.00. Historically, you used versions that were ASCII only.

Psych wrote:There is also overhead. Do not unicode programs which rely heavily on embedded string tables increase the size of executables? Does not allocating strings internally using unicode use twice as much memory? Negligible perhaps, but true nevertheless.

Regarding these thoughts, IMHO, the overhead is not great and the difference can be considered negligible. You can lessen the size for the storage of string tables by still storing them in ASCII encoding or by any number of different compression schemes.

Psych wrote:This is not really about encoding, it is about choice.

Respectfully, if it is only really about choice then it is a waste of time. If a user of your code can't see a difference what's the point of the choice? If it only takes some manageable modifications to your older code solutions to migrate them to a unicode compilation then what's the point of the choice? Wouldn't you rather a choice about something that makes more of a difference instead?

Posted: **Wed Aug 24, 2016 10:52 pm**

Psych wrote:I don't really think I am.

but you are. you complain about reading/writing strings and so on. it doesnt matter how the strings are internally stored. everything you could do with ascii, you can do with unicode, only that japanese, russian etc. executables will show proper characters on every pc now, no matter what language you use.

you can read an ascii string from a file, with the proper #PB_Ascii flag and it will be automatically converted to unicode. then you can modify it as much as you want and write it back as ascii if you like. nothing changes, unless you need direct in-memory access. either way, you only need a little bit more code.

you also said that you havent seen any real world applications that uses unicode. 2 byte wide char unicode (ucs2/utf16) is the default layer on any windows system since windows 2000 and 95% of apps use it, including every microsoft app and driver.

oh and speaking of overhead, the legacy ascii support is done by the unicode layer as well and everything you see on your screen is in fact unicode.

Psych wrote:However wouldn't UTF8 be better as the standard?

microsoft only has ascii and unicode functions:

Code: Select all

MessageBoxW_() (Unicode)
MessageBoxA_() (ANSI)

thats why there is no such thing as an utf8 function. utf8 is a format mainly designed for text files, which offers backward compatibility to the first 127 ascii characters. html, xml, php and even your pb source code is nothing else than a text file and can be processed by ascii and unicode executables alike. the browser you are viewing this page with, is a unicode executable.

Psych wrote:I am pretty sure it was because of limitations with addressing, not because they didn't like ascii.

limitations are exactly the reason why ascii was abandoned. same as 16bit executables were abandoned and 32bit executables will be abandoned one day too. in unicode there are currently more than 100.000 codepoints assigned, while ascii only has 255, where only the first 127 are always the same.

Psych wrote:You think the average PC has 16gb RAM? Try 4 or 8. I can only assume you develop applications with little or no regard for memory usage. That is also your choice.

4gb has been mainstream since the 64bit version of windows vista, which was released in 2007. even if you only had 1gb, you could open up a 10gb file by using file mapping, like hex editors do since many many years now. and you worry about the extra 1-2kb of your executable?

c ya,
nco2k

Posted: **Wed Aug 24, 2016 11:05 pm**

Why you feel the need to argue and belittle other peoples opinions is beyond me, maybe it's just the mob mentality, I shouldn't have expected any better.

All I did was state my opinion, but now I am stupid. Fine. A win for you.

Posted: **Wed Aug 24, 2016 11:28 pm**

how the heck am i belittling you? this has nothing to do with winning or even attacking your opinion.

your opinion is that there was no need to remove the ascii support and i agree, but sooner or later the time would come where they would have to remove it either way. so why not now?

all im saying is that unicode has been around for a very long time and that you probably didnt notice it. i mean its not like the executable you open says hello im unicode.

writing unicode apps is every bit as easy as writing ascii apps, once you start doing it.

i also didnt attack or insult you in any way. i apologize if you feel otherwise. guess thats normal on the internet nowadays, which is a shame really. only because im having a discussion with you and/or dont share your points of view, doesnt mean i hate you. i dont even know you.

c ya,
nco2k

Posted: **Thu Aug 25, 2016 12:03 am**

Psych wrote:It would be interesting to know, historically, how PB has evolved over the years, I have only been using this compiler since the late 3.x versions, however, I can easily imagine a time PB was ascii only. The reason I say this is because the option to 'Compile as Unicode' is presented as an option on a per-source basis, it has never said 'Compile as Ascii', how long has that option been there? From the very first version? Then from later versions the option was still there, but now it checked by default on a clean install, I can easily imagine a time it wasn't the default option. This tells me that ascii compiling was the most widely used scenario, and unicode executables were added as an option, a choice for users. Now unicode is the preferred method, and that has never been in doubt, and yes, I will have to get used to it, that doesn't make my disappointment about the removal of ascii any less valid.

I understand your disappointment, but just because the option was once there does not mean we can afford to keep it forever.

Here are some other options we have left behind over time:

Support for Windows 9x, NT and 2000
Support for DirectX 3
Support for DirectX 7
Support for Gtk 1 on Linux
Support for PPC processors on MacOSX
Amiga Support

As I explained in the other discussions on this topic: If we want to move into the future with new technologies, we have to leave some things behind from time to time. Do you really suggest we should sacrifice possible new features such as 64bit support to keep something outdated like DirectX 3 (three!) alive indefinitely?

We are more conservative than most when removing old features (see Windows XP support). But everything has its limits.

Give it some time. In a few years the thought of keeping Ascii support alive will sound as ridiculous as DX3 support sounds now.

Posted: **Thu Aug 25, 2016 1:16 am**

This is where the community really should focus on submitting bug reports for the LTS versions. This way if there's a feature you enjoy and you know is going to go away soon... You'll always have access to a solid version with a feature you liked.

Posted: **Thu Aug 25, 2016 1:49 am**

freak wrote:I understand your disappointment, but just because the option was once there does not mean we can afford to keep it forever.

Thank you, freak, I never expected this decision to be reversed, I just wanted to air my views on the matter.

As Darwin observed, animals evolve and adapt when their environment forces them to do so, I will also adapt.

Posted: **Thu Aug 25, 2016 8:49 am**

Psych wrote:I will also adapt.

It's not that hard or bad to adapt to Unicode; trust me. I was totally against it too, like you. Now I code every day like nothing changed. It's literally no big deal once you make the step. Just ask here if something in your strings breaks, and lots of people will show you how to fix it. That's what I did. It's all good now.

Here's an excellent article (written in 2003!) that explains why you need Unicode:

http://www.joelonsoftware.com/articles/Unicode.html

Please take the time to read it. It's important.

PureBasic Forums - English

No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?

Re: No Ascii?