Page 1 of 1

please remove range test from chr()

Posted: Mon Oct 16, 2023 9:28 pm
by idle
please remove range test in the ide from chr()

Re: please remove range test from chr()

Posted: Mon Oct 16, 2023 10:25 pm
by Fred
The range between $D800 and $DFFF is reserved for surrogate characters, so it shouldnt be used in Chr() as it doesnt returns a valid character.

Re: please remove range test from chr()

Posted: Tue Oct 17, 2023 1:18 am
by idle
Please make it a soft warning rather than a compiler halt if you feel it's important. I only mentioned it as
It broke my 4000 line utf16 module.

Re: please remove range test from chr()

Posted: Tue Oct 17, 2023 1:57 am
by normeus
+1 for a warning instead of a fail

Norm

Re: please remove range test from chr()

Posted: Tue Oct 17, 2023 9:07 am
by Fred
Could you explain why you need it to work as it doesn't return a valid character ? The error was added to spot this issue as a bug report was raised saying Chr() didn't work as expected.

Re: please remove range test from chr()

Posted: Tue Oct 17, 2023 9:34 am
by STARGÅTE
I think the test range was added, because Len(Chr($D800)) has given 2 instead of 1.
I any way didn't use Chr() since it doesn't work with the whole unicode character range up to $10FFFF.

However:
Fred wrote: Mon Oct 16, 2023 10:25 pm The range between $D800 and $DFFF is reserved for surrogate characters, so it shouldnt be used in Chr() as it doesnt returns a valid character.
Based on this argument, you then should also check for other invalid characters like U+FFFE and U+FFFF or the NUL character, because it gives no string.

To generate surrogate pairs one can use PeekS() on a buffer with 4 bytes (the surrogate pair).

Re: please remove range test from chr()

Posted: Tue Oct 17, 2023 11:01 am
by idle
It does work for the whole unicode range with utf16 encoding which you can build on top of ucs2, which is how made the utf16 module.
I was just puzzled why and for what purpose. My understanding is that It's not the function chr place to check its the final render that's supposed to check and replace non printable chrs with □