Page 1 of 2
Need quick StringToHex() function
Posted: Mon Mar 18, 2013 1:40 pm
by Kukulkan
Hello,
I'm currently using the StringToHex() function below. But I think it should be much faster. Can you help?
The rules:
- Cross plattform
- Single Byte and Unicode capable
- In unicode mode, the unicode string must get converted to UTF8 first.
This is my attempth:
Code: Select all
EnableExplicit
; Copies string to single byte memory (make unicode to utf8)
Macro getSBStringCopy(orig, copy)
CompilerIf #PB_Compiler_Unicode
Protected copy.s = Space(StringByteLength(orig) * 2)
; make sure the result is null terminated
FillMemory(@copy, StringByteLength(copy), 0)
PokeS(@copy, orig, -1, #PB_UTF8)
CompilerElse
Protected copy.s = orig
CompilerEndIf
EndMacro
; Converts a string in hex notation at (each byte to a hex code)
Procedure.s StringToHex(Original.s)
Protected Ergebnis.s = ""
Protected x.i = 0
Protected Dummy.s = ""
getSBStringCopy(Original, sbOriginal)
For x.i = 1 To StringByteLength(sbOriginal.s)
Dummy.s = Dummy.s + RSet(Hex(PeekB(@sbOriginal + x - 1), #PB_Byte), 2, "0")
If Len(Dummy.s) > 500
Ergebnis.s = Ergebnis.s + Dummy.s
Dummy.s = ""
EndIf
Next
Ergebnis.s = Ergebnis.s + Dummy.s
ProcedureReturn Ergebnis.s
EndProcedure
Define Test.s = Space(100000)
Define start.i = ElapsedMilliseconds()
Define Result.s = StringToHex(Test.s)
Define ende.i = ElapsedMilliseconds()
MessageRequester("Test", "Needed " + Str(ende.i - start.i) + "ms")
End
By the way, I'm also interested in the reverse function if it also fits the rusel above
Thank you,
Kukulkan
Re: Need quick StringToHex() function
Posted: Mon Mar 18, 2013 2:02 pm
by wilbert
Do you want a solution using only PB commands or is ASM allowed ?
If you want the generated hex data to be stored somewhere, it might be best to always convert to UTF8 (both ascii and unicode).
Re: Need quick StringToHex() function
Posted: Mon Mar 18, 2013 2:18 pm
by Little John
I have written code for MemoryToHex and HexToMemory conversion
here (you'll have to add conversion from/to UTF-8 yourself). Other people have posted code and links in that thread, too. So you might find something there that is useful for you.
Re: Need quick StringToHex() function
Posted: Mon Mar 18, 2013 3:17 pm
by Kukulkan
wilbert wrote:Do you want a solution using only PB commands or is ASM allowed ?
Hm. Best would be without ASM. So I can understand and convert to x64, other plattform with no problems later.
wilbert wrote:If you want the generated hex data to be stored somewhere, it might be best to always convert to UTF8 (both ascii and unicode).
Good idea but not needed in my case. If single byte version, there must be no encoding as my single byte strings are already utf8 encoded versions.
Little John wrote:I have written code for MemoryToHex and HexToMemory conversion
here (it shouldn't be to hard to add string conversion to/from UTF-8). Other people have posted code and links in that thread, too. So you might find something there that is useful for you.
I already found this thread but I thought it was not that optimized for my special need. I need to encode a
string (single byte or unicode converted to utf8) to a new
string which contains hex. The functions in that thread are needing much conversion with PeekS() and PokeS() etc...
Kukulkan
Re: Need quick StringToHex() function
Posted: Mon Mar 18, 2013 4:20 pm
by wilbert
Does this work for you ?
Code: Select all
EnableExplicit
; Converts a string in hex notation at (each byte to a hex code)
Procedure.s StringToHex(Original.s)
Protected Result.s, ResultLen, i, *In.Ascii
; build static lookup table
Static Dim Table.l(256)
If Table(0) = 0
While i < 256
PokeS(@Table(i), Right(Hex(256 + i), 2))
i + 1
Wend
EndIf
; set some variables and convert to utf8 if required
CompilerIf #PB_Compiler_Unicode
i = StringByteLength(Original, #PB_UTF8)
Result = Space(i << 1)
*In = @Result + i * 3
PokeS(*In, Original, -1, #PB_UTF8)
Protected *Out.Long = @Result
CompilerElse
i = Len(Original)
Result = Space(i << 1)
*In = @Original
Protected *Out.Word = @Result
CompilerEndIf
; conversion loop
While i
CompilerIf #PB_Compiler_Unicode
*Out\l = Table(*In\a)
*Out + 4
CompilerElse
*Out\w = Table(*In\a)
*Out + 2
CompilerEndIf
*In + 1
i - 1
Wend
ProcedureReturn Result
EndProcedure
; test the procedure
Define Test.s = Space(100000)
Define start.i = ElapsedMilliseconds()
Define Result.s = StringToHex(Test.s)
Define ende.i = ElapsedMilliseconds()
MessageRequester("Test", "Needed " + Str(ende.i - start.i) + "ms")
End
Reverse function (built for speed, no check for invalid characters)
Code: Select all
; Converts hex notation to a string (each hex code to a byte)
Procedure.s HexToString(Original.s)
Protected i.a, n.a, *In.Ascii = @Original
Protected *Out.Ascii = *In
i = *In\a
While i
i & 31
If i < 7
i + 9
Else
i & 15
EndIf
If n > 15
*Out\a = n << 4 | i
*Out + 1
n = 0
Else
n = i | 16
EndIf
*In + SizeOf(Character)
i = *In\a
Wend
*Out\a = 0
CompilerIf #PB_Compiler_Unicode
ProcedureReturn PeekS(@Original, -1, #PB_UTF8)
CompilerElse
ProcedureReturn Original
CompilerEndIf
EndProcedure
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 2:39 am
by Thunder93
Good job Wilbert, thanks.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 4:07 am
by Thade
A lot of code for things you can do with one line of code
Conversions from Hex and to Hex are supported with Val() and Hex()
This should be the quickest StringToHex
or with #PB_Byte ... #PB_Unicode
Code: Select all
Result.s=Hex(Val(YourDecimalstring.s))
.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 6:58 am
by wilbert
Thade wrote:A lot of code for things you can do with one line of code
You missed the point.
Code: Select all
Debug StringToHex("PureBasic"); => "507572654261736963"
Debug HexToString("507572654261736963"); => "PureBasic"
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 9:08 am
by rsts
Nice code wilbert.
Added to toolbox. Thanks.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 10:30 am
by Little John
Kukulkan wrote:I need to encode a string (single byte or unicode converted to utf8) to a new string which contains hex. The functions in that thread are needing much conversion with PeekS() and PokeS() etc...
I know that you need to encode a
string. Strings are easily handled by those procedures as well, because with @ we can get their memory addresses. In ASCII mode, there is no PeekS() or PokeS() requiered at all. In Unicode mode, according to your needs, one PokeS() would be required ...
It is pretty fine for me when you do not use those functions, but I don't want the forum readers to get a wrong impression about them.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 11:31 am
by Kukulkan
Whow wilbert,
thank you! 
This is really fast (only 16ms instead of 4300ms)!
I will use the new function now and this really speeds up my program!
Anyway, the comment of Little John makes me curious. How would such implementation look like?
Kukulkan
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 12:38 pm
by davido
Hi wilbert,
Thank you for sharing your code! Lots to learn - must study it carefully.
Astonishing speed for string manipulation.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 1:05 pm
by flaith
Nice piece of code
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 2:16 pm
by wilbert
davido wrote:Astonishing speed for string manipulation.
That's because the main loop is kept very simple and doesn't call any string manipulation functions.
Little John wrote:Anyway, the comment of Little John makes me curious. How would such implementation look like?
As far as I understand the routines created by Little John focus on versatility (you can control a lot of output options) while I focused on speed alone.
But maybe he can explain better himself.
Re: Need quick StringToHex() function
Posted: Tue Mar 19, 2013 2:23 pm
by Little John
Hello Wilbert!
wilbert wrote:As far as I understand the routines created by Little John focus on versatility (you can control a lot of output options) while I focused on speed alone.
But maybe he can explain better himself.
I think it's not possible to explain it better.