[Solved] String starts with Chr(0) somehow?

[BarryG]

SOLVED! See post further down. Windows (the OS) was the cause!

In my app (which I can't show the source) there's a line that checks FileSize(file$) for a file's location but it keeps returning -1 for not found.

So I put "SetClipboardText(file$)" before it, and it looked normal. But then I pasted the copied clipboard text into a hex editor and it shows Chr(0) at the start of the copied text. WTF? It's obviously why FileSize() is returning -1 but since when can a string have Chr(0) in it?

If I use "Debug Len(file$)" it returns 17, but file$ is ‪"D:\Temp\WTF.xlsx" which is only 16 characters. The extra 1 is due to Chr(0) at the start?

Using PureBasic 6.10 (64-bit) on Win 10 Pro, but it also happens with 6.20 Beta 2.



[Edit] I just added this debugging code and got the following result. Some strange leading character that looked like Chr(0) when pasted into HxD.

Code: Select all

For i=1 To Len(file$)
  c=Asc(Mid(file$,i,1))
  Debug Str(i)+" = "+Str(c)+" ("+Chr(c)+")"
Next

Output:

Code: Select all

1 = 8234 (‪)
2 = 68 (D)
3 = 58 (:)
4 = 92 (\)
5 = 84 (T)
6 = 101 (e)
7 = 109 (m)
8 = 112 (p)
9 = 92 (\)
10 = 87 (W)
11 = 84 (T)
12 = 70 (F)
13 = 46 (.)
14 = 120 (x)
15 = 108 (l)
16 = 115 (s)
17 = 120 (x)

[#NULL]

I don't know what in/after SetClipboardText() might happen, so better use ShowMemoryViewer() to directly look at the mem.
This is what it normally would look like:

Code: Select all

s.s = "D:\Temp\WTF.xlsx"
ShowMemoryViewer(@s, 40)
; 0000000034624070  44 00 3A 00 5C 00 54 00 65 00 6D 00 70 00 5C 00  D.:.\.T.e.m.p.\.
; 0000000034624080  57 00 54 00 46 00 2E 00 78 00 6C 00 73 00 78 00  W.T.F...x.l.s.x.
; 0000000034624090  00 00 E9 90 00 00 00 00                          ..é....

As you can see, because PB uses UTF-16 internally, every character has 2 bytes, including a null byte for each ASCII character. This is normal. The question is where and how do you get your string/filename from.

[BarryG]

This is what ShowMemoryViewer shows:



I'll have to check what's going on when assigning the file to the variable.

[DarkDragon]

We need to know how you obtained file$. Otherwise we cannot help you. The mistake is probably there. And as you can see it's not Chr(0) but 2A and 20 in hex.

[BarryG]

SOLVED! It turns out that Windows (the OS) was the cause. I had right-clicked the file to view its Properties window, and copied the path from there (see below). But Windows is adding an extra invisible character to the copy! Does that happen to anyone else?

[infratec]

Better:

Code: Select all

s.s = "D:\Temp\WTF.xlsx"
ShowMemoryViewer(@s, StringByteLength(s) + 2)

As mentioned already: your filename is wrong in memory.

Maybe it is overwritten. Use Purifier and 1, 1, 1, 1 as granularity.


https://www.compart.com/en/unicode/U+2A20

[BarryG]

It's all good now, infratec. It wasn't a coding error or PureBasic bug; it was a Windows (OS) copy issue. See post above.

[benubi]

Yes, this is a classic copy & paste mishap. There can be leading and following special characters like mostly CR/LF and things like box-drawing characters (code page 437, unicode), just like when playing with web pages.

[BarryG]

This could've been diagnosed faster if the Debug command showed all characters of a string. Will make a request for it.

[pjay]

I didn't think the standard clipboard text function could contain a null character at the begining of it, as it's a reserved control code used as a terminator. It certainly wouldn't have injected itself from copying it from the windows properties field as you're suggesting.

Your explanation doesn't make sense to me; even if it had managed to become part of your clipboard content, how would you have pasted a null character into PB's IDE for it to eventually form part of your filename string?

I'd definitely look into this more Barry, something's not right (or i'm mis-understanding it).


Edit - I take it back - replicated here - it's not a null character but a unicode LRE character.

[DarkDragon]

This could've been diagnosed faster if the Debug command showed all characters of a string. Will make a request for it.

No, there will always be invisible characters. Space, tabulator, non blocking space, zero width space, ... that's no bug and also no feature request as it depends on your font.

[Michael Vogel]

Hm, my Windows versions do not show such a strange behaviour when using the clipboard, anyhow you may use one of the following tools to check your clipboard content...

https://www.nirsoft.net/utils/inside_clipboard.html

Code: Select all

#DumpWidth=8

Procedure DebugMemory(*mem.Ascii,len)

	Protected.i b,n
	Protected.s s,t
	
	s=RSet(Hex(n),4,"0")
	
	While n<len
		s+" "+RSet(Hex(*mem\a),2,"0")
		If *mem\a<32
			t+"."
		Else
			t+Chr(*mem\a)
		EndIf
		b+1
		n+1
		If b=#DumpWidth
			b=0
			Debug s+" | "+t
			s=RSet(Hex(n),4,"0")
			t=""
		EndIf
		*mem+1
	Wend

	If b
		Debug s+Space((#DumpWidth-b)*3)+" | "+t
	EndIf

EndProcedure
Procedure DebugString(*string)

	DebugMemory(*string,MemoryStringLength(*string,#PB_ByteLength))

EndProcedure
Procedure DebugClipboardText()

	Protected s.s

	s=GetClipboardText()

	DebugString(@s)

EndProcedure

SetClipboardText("Hello Barry!")

DebugClipboardText()

[kenmo]

It's a weird behavior of that specific Windows dialog, see this Microsoft blog post:
https://devblogs.microsoft.com/oldnewth ... 0/?p=44924

The Security dialog box inserts that control character in the file name field in order to ensure that the path components are interpreted in the expected manner. Unfortunately, it also means that if you try to copy the text out of the dialog box, the Unicode formatting control character comes along for a ride. Since the character is normally invisible, it can create all sorts of silent confusion.

Your hex editor converted it to a null 0x00 because you're pasting a larger-than-255 Unicode character (0x202A) as a 1-byte-per-character string.

The PB Debugger should not be expected to display it, because it's one of (many) intentionally invisible or non-printing Unicode characters...
As mentioned in that blog post, Microsoft Notepad doesn't display it, for example.

Very interesting find, BarryG!

[BarryG]

https://devblogs.microsoft.com/oldnewth ... 0/?p=44924

Wow, so it's an actual known thing, and by design.

Unfortunately, it also means that if you try to copy the text out of the dialog box, the Unicode formatting control character comes along for a ride. Since the character is normally invisible, it can create all sorts of silent confusion.

Tell me about it, Raymond! No wonder I couldn't work out what was wrong.

Very interesting find, BarryG!

Yeah, typical that these sorts of things happen to me.

[benubi]

Oh I'm sorry then. It's not a mishap, it's a feature!