Loading and displaying text with special chars on MacOS?

[Andre]

This is a small snippet from a text file, created and saved on Windows OS:

Code: Select all

France places a few missionaries in the CÙte d'Ivoire, a country it will rule more than 200 years later.
The Kingdom of England wages war against the Mashantucket Pequots.
The first opera house, Teatro San Cassiano, opens in Venice.
RenÈ Descartes writes his Discours de la MÈthode.

In original form (Windows) it contains the word in "right form", e.g. Cóte d'Ivoire or René.
But when loading and displaying the (on Windows saved) special chars on MacOS they are looking like: È Ù ...

Anyone has an idea, how to handle Windows text files containing special chars on MacOS correctly, so that on Mac also the correct special chars are displayed?

Btw... simply changing the Compiler modus to Unicode doesn't help.

Thanks!

[Andre]

Here is the same content from the same (unchanged) file, but I opened it on MS Windows:

Code: Select all

France places a few missionaries in the Côte d'Ivoire, a country it will rule more than 200 years later. 
The Kingdom of England wages war against the Mashantucket Pequots. 
The first opera house, Teatro San Cassiano, opens in Venice. 
René Descartes writes his Discours de la Méthode. 

As you can see, all special chars are displayed like they should.

Do you know any further informations about handling this problem: different display of the same (special) char on Windows and MacOS? And probably know solutions for that?

Or must I convert the file contents when loading the text file on MacOS? (converting all special chars to the right one on MacOS...)

Or....?

[Vera]

Hi Andre,

I wish I could help you but I've no chance. I do think it's less a question of unicode but more of the special character encoding of the original file itself.
Also copying your example form this html presentation doesn't enable any other MAC-user to test and find out what the reason and solution can be.

I don't think you'd need to convert the special chars themselfs but to find a suitable 'base' encoding for the files that work for both platforms. Could you upload one of of those original files to check and test it?

greetings ~ Vera

[Andre]

Hi Vera,

here is a file (saved as ANSI text file on MS Windows) for testing:
http://rapidshare.com/files/448137341/T ... lChars.txt

Hope it's possible to see the problem, as with any simple LoadFile() --> ReadString() --> Debug Routine my described problem (on MacOS) should be clear...

Thanks for your effort! )

[Vera]

Hello André,

thanks for that 'original' file that I took as base for further testing.
I bundled the whole test as Text_SpecialChars.zip including a note of what I experienced.

The conclusion I've come to is, that it might be a solution to encode your used text-files as UFT-8 with Signature (65001) as this format seems to be most compartible:
a - WIN does not revert it back to Ansi (like it does with other encodings)
b - most texteditors (including the IDE ) are able to detect this format and recode the special characters automatically for displaying
(c- no need for unicode)

Note: I didn't care about the line endings and left them in WIN-format as I don't think that the special character display depends on this.

greetings ~ Vera

ps: some interesting pages I came across these days you'd like to have a look at: A tutorial on character code issues, Character encoding, Text file, Mac OS Roman

[Shardik]

Andre,

I have downloaded your file Text_SpecialChars.txt on my iMac with MacOS X 10.6.6
and all special characters are correctly displayed in the PureBasic IDE and in TextEdit...

Update: I also loaded that file in Windows 7 x64 Home Premium (on the same iMac
within BootCamp) into Notepad and the PureBasic IDE and the special characters are
also displayed correctly. I downloaded the file twice and stored it in a FAT32 data
partition and on my Mac partition: the special characters are always displayed
correctly...

My PB IDE settings on the Mac are the default settings:
File/File format:
- Encoding: Plain Text
- Newline: MacOS (CR)

I tried loading your text first with the German language selected in the Preferences
and a second time with English.

[Andre]

Hi Vera & Shardik,

thank you very much for your effort and tipps! )

I think I have to describe it a bit further what I'm doing:

I usually don't use any text-editors (neither on Windows, or on MacOS) for editing the "text files".

The files are originally MS Excel databases, which I'm editing on Windows (with Excel2007) and on MacOS (with Excel2008) and saving them always as plain text files ".csv - comma-separated files". Furthermore I'm editing these text files (databases) using my self-written editor, which is using the standard LoadFile()... + ReadString()/WriteStringN() commands.

So I'm wondering where I should be able to switch to another text format?