Weird text file problem

[PB]

I've got a weird problem with text files... I'm editing an INI file by hand, not
with an app. It consists of lots of web addresses, in the normal text file
format of:

http://www.purebasic.com/
http://www.purebasic.com/screenshots.php3
http://www.purebasic.com/support.php3

Each line ends with CRLF, as expected for Windows. However, for some
reason, the first line now gets 3 bytes at the start of it: EF, BB, BF. Here's
an example image as seen in a hex editor:



Anyone know why my text files are saving like this now? Even if I use a

normal TXT extension instead of INI, it's there. I found a similar question
on another site, but with no answer:

http://www.webmasterworld.com/forum47/2618.htm

The INI file I'm editing is 83.1 KB in size, and the problem doesn't seem
to occur with smaller files (10 KB or so). Is this because the file is larger
than 64 KB and Notepad can't handle it? I thought Notepad under Win2K
could handle text files of any size? Any ideas on this? Thanks!

Edit: Just tried editing it with WordPad and the characters don't appear,
so yes, it seems Notepad can't handle big text files. Wow. Go Microsoft!
Looks like I'll have to associate both INI and TXT with WordPad now.

[Num3]

Never had any problems using notepad on large files (>1Mb)

Maybe the file just got corrupted and remainded corrupted...

[PB]

> Never had any problems using notepad on large files (>1Mb)

Neither had I before, which is why I was confused.

> Maybe the file just got corrupted and remainded corrupted

It even happens with new files that I created when testing. Maybe my PC
is infected with something, although I can't see how. Avast says it's clean
and there's no spyware reported with SpybotS&D. Maybe a reboot will fix
it, but I don't have time right now.

(Besides, with WordPad I can search/replace whole words, and Notepad
doesn't let you do that -- so I'll stick with WordPad from now on I think.
I just thought it may be a common problem due to the URL above where
the guy had the same problem).

[Dare2]

Thats unicode sig, UTF 8, I think.

Wordpad recognises unicode. Notepad (or some versions) doesn't. Some versions try. Some version do.

[PB]

> Thats unicode sig, UTF 8, I think

Unicode, eh? Do you think some of the URLs I have in the text file would
contain unicode characters? I've seen things like & in them, and have
removed them, but Notepad still saves it with those 3 header bytes. Most
annoying as my app needs to read the first line of the file and won't because
of these bytes.

[Dare2]

On my win2k box (and a while back) notepad started misbehaving. Even with new files I created myself. It was struggling with unicode. Then, after a time, it stopped struggling and things went plain text again.

I use automatic updates so I assume that MS mods introduced and then sorted out the misbehaviour.

Maybe something like that on your box?

Edit:

To get around the problem, perhaps binaryRead the first three bytes? If they're not the sig, fileseek back to start and go with readString. If they are the sig, readString from there on in - because, from recollection, only the start of the file had the prob, the rest was plain text.

[PB]

> I use automatic updates so I assume that MS mods introduced and
> then sorted out the misbehaviour

I'll see how it goes tomorrow after a reboot. If the problem is still there,
then I'll turn on Automatic Updates and let Microsoft play with my PC, and
then see what happens.

[Dare2]

Edited same time you were posting, see post above.

[PB]

> To get around the problem, perhaps binaryRead the first three bytes?

Hmm, that's a lot of work (not to code, but for my app to do) for a situation
that might not occur very often. When using WriteStringN() to create the
file, it's okay -- no header bytes. So making my app check for it is a waste
of time and effort. It's just that the user might edit the file himself, and thus
run into this problem. But I don't want my app to check for that if it's not a
common problem. Although I guess I should, to cover all bases. Yuck.
I don't want to. It's just reading/writing plain text -- who wants to add
some Unicode-checking code now? If it's a common problem, then maybe
Fred can make ReadString() check for it instead...

But hang on -- why does this only happen on >64 KB files? Surely that
would mean it's NOT a Unicode problem?

[Dare2]

then maybe
Fred can make ReadString() check for it instead...

Don't think it is common. But anything to do with end-users has more hazards than you poke a stick at ...

Edit:
http://www.google.com.au/search?q=unico ... 1&safe=off

So I assume now, as I did then. Funnily enough I used the old version of Arachnophilia as my editor of choice and around the same time as this was happening with notepad Arach started treating files with { inside as unicode. So it assumed everything was some sort of unknown property, etc, and showed begger all.

But all that went away magically. Love it.

[PB]

> Don't think it is common. But anything to do with end-users has more
> hazards than you poke a stick at

Hehehe, true. Actually, it's probably just easier to stick something like this
in the Troubleshooting section of my app's documentation: If the first line
of your file doesn't seem to be read, it may be that you manually edited the
file contrary to the instructions not to. Please do not edit the file manually.

Then just save the file with a first line of "; DO NOT EDIT THIS FILE!"

Ha, why not... they'll only get the problem if they edit the file, and then only
if they consciously decide to ignore the commented warning. They can bear
the blame for causing the problem by mucking around. I'm not going to add
code to my app just to check for their mischieviousness.

Edit: Thanks for the Google link!

[Dare2]

Ha, why not... they'll only get the problem if they edit the file, and then only
if they consciously decide to ignore the commented warning. They can bear
the blame for causing the problem by mucking around.

hehe.

Support call:

User. "No, honestly, I didn't touch the file! I didn't see the warning and I didn't save the file again after I didn't open it!"
Support:
- to self - *Hmmmmm. Why don't I believe you, let me count the reasons ...*
- to user - "I see. Okay, here is how we fix it ...."





Edit: Talking of strange chars, what is the 4F 7A in your sig?


Edit again. DOH Nevermind.

[PB]

> Support call

Hmm, I didn't consider that... but I'm not in business yet anyway, so I'm
excused. I'll just tell 'em to reinstall the app -- LOL -- not! Man, this sucks.
I guess I could drop off the .INI extension so they can't "accidentally" edit
it, and even mark it as Read-Only... and THEN if they still edit it, too bad.
Hohoho, it's time for me to go to bed I think. So, to recap, they'd have to:

(1) Unmark the file as read-only.
(2) Add a .INI extension to open it with Notepad.
(3) Ignore the "DO NOT EDIT" warning.
(4) Edit the file.
(5) Save it.

Then they'll call me for support and claim they didn't edit it? I'll tell them
I can't help them as something mysterious has happened on their PC, and
perhaps they have a virus and need to re-install Windows. Doesn't bother
me if they're going to treat the app like that and then lie to me.

> Edit: Talking of strange chars, what is the 4F 7A in your sig?

Come on, you can work it out...

[Dare2]

Oi Oi Oi!

[PB]