Need quick StringToHex() function

[Kukulkan]

Hello,

I'm currently using the StringToHex() function below. But I think it should be much faster. Can you help?

The rules:
- Cross plattform
- Single Byte and Unicode capable
- In unicode mode, the unicode string must get converted to UTF8 first.

This is my attempth:

Code: Select all

EnableExplicit

; Copies string to single byte memory (make unicode to utf8)
Macro getSBStringCopy(orig, copy)
  CompilerIf #PB_Compiler_Unicode
    Protected copy.s = Space(StringByteLength(orig) * 2)
    ; make sure the result is null terminated
    FillMemory(@copy, StringByteLength(copy), 0)
    PokeS(@copy, orig, -1, #PB_UTF8)
  CompilerElse
    Protected copy.s = orig
  CompilerEndIf
EndMacro

; Converts a string in hex notation at (each byte to a hex code)
Procedure.s StringToHex(Original.s)
  Protected Ergebnis.s = ""
  Protected x.i = 0
  Protected Dummy.s = ""
  getSBStringCopy(Original, sbOriginal)
  For x.i = 1 To StringByteLength(sbOriginal.s)
    Dummy.s = Dummy.s + RSet(Hex(PeekB(@sbOriginal + x - 1), #PB_Byte), 2, "0")
    If Len(Dummy.s) > 500
      Ergebnis.s = Ergebnis.s + Dummy.s
      Dummy.s = ""
    EndIf
  Next
  Ergebnis.s = Ergebnis.s + Dummy.s
  ProcedureReturn Ergebnis.s
EndProcedure

Define Test.s = Space(100000)
Define start.i = ElapsedMilliseconds()
Define Result.s = StringToHex(Test.s)
Define ende.i = ElapsedMilliseconds()
MessageRequester("Test", "Needed " + Str(ende.i - start.i) + "ms")
End

By the way, I'm also interested in the reverse function if it also fits the rusel above

Thank you,

Kukulkan

[wilbert]

Do you want a solution using only PB commands or is ASM allowed ?
If you want the generated hex data to be stored somewhere, it might be best to always convert to UTF8 (both ascii and unicode).

[Little John]

I have written code for MemoryToHex and HexToMemory conversion here (you'll have to add conversion from/to UTF-8 yourself). Other people have posted code and links in that thread, too. So you might find something there that is useful for you.

[Kukulkan]

Do you want a solution using only PB commands or is ASM allowed ?

Hm. Best would be without ASM. So I can understand and convert to x64, other plattform with no problems later.

If you want the generated hex data to be stored somewhere, it might be best to always convert to UTF8 (both ascii and unicode).

Good idea but not needed in my case. If single byte version, there must be no encoding as my single byte strings are already utf8 encoded versions.

I have written code for MemoryToHex and HexToMemory conversion here (it shouldn't be to hard to add string conversion to/from UTF-8). Other people have posted code and links in that thread, too. So you might find something there that is useful for you.

I already found this thread but I thought it was not that optimized for my special need. I need to encode a string (single byte or unicode converted to utf8) to a new string which contains hex. The functions in that thread are needing much conversion with PeekS() and PokeS() etc...

Kukulkan

[wilbert]

Does this work for you ?

Code: Select all

EnableExplicit

; Converts a string in hex notation at (each byte to a hex code)
Procedure.s StringToHex(Original.s)
  Protected Result.s, ResultLen, i, *In.Ascii
  
  ; build static lookup table
  Static Dim Table.l(256)
  If Table(0) = 0
    While i < 256
      PokeS(@Table(i), Right(Hex(256 + i), 2))
      i + 1  
    Wend
  EndIf
  
  ; set some variables and convert to utf8 if required
  CompilerIf #PB_Compiler_Unicode
    i = StringByteLength(Original, #PB_UTF8)
    Result = Space(i << 1)
    *In = @Result + i * 3
    PokeS(*In, Original, -1, #PB_UTF8) 
    Protected *Out.Long = @Result
  CompilerElse
    i = Len(Original)
    Result = Space(i << 1)    
    *In = @Original
    Protected *Out.Word = @Result
  CompilerEndIf
  
  ; conversion loop
  While i
    CompilerIf #PB_Compiler_Unicode
      *Out\l = Table(*In\a)
      *Out + 4
    CompilerElse
      *Out\w = Table(*In\a)
      *Out + 2
    CompilerEndIf
    *In + 1
    i - 1
  Wend
  
  ProcedureReturn Result  
EndProcedure


; test the procedure
Define Test.s = Space(100000)
Define start.i = ElapsedMilliseconds()
Define Result.s = StringToHex(Test.s)
Define ende.i = ElapsedMilliseconds()
MessageRequester("Test", "Needed " + Str(ende.i - start.i) + "ms")
End

Reverse function (built for speed, no check for invalid characters)

Code: Select all

; Converts hex notation to a string (each hex code to a byte)
Procedure.s HexToString(Original.s)
  Protected i.a, n.a, *In.Ascii = @Original
  Protected *Out.Ascii = *In
  
  i = *In\a
  While i
    i & 31
    If i < 7
      i + 9
    Else
      i & 15
    EndIf
    If n > 15
      *Out\a = n << 4 | i
      *Out + 1
      n = 0 
    Else
      n = i | 16
    EndIf
    *In + SizeOf(Character)
    i = *In\a
  Wend
  *Out\a = 0
  
  CompilerIf #PB_Compiler_Unicode
    ProcedureReturn PeekS(@Original, -1, #PB_UTF8)
  CompilerElse
    ProcedureReturn Original
  CompilerEndIf
EndProcedure

[Thunder93]

Good job Wilbert, thanks.

[Thade]

A lot of code for things you can do with one line of code
Conversions from Hex and to Hex are supported with Val() and Hex()
This should be the quickest StringToHex

Code: Select all

Result.q=Val("$"+YourHexstring.s)

or with #PB_Byte ... #PB_Unicode

Code: Select all

Result.s=Hex(Val(YourDecimalstring.s))

.

[wilbert]

A lot of code for things you can do with one line of code

You missed the point.

Code: Select all

Debug StringToHex("PureBasic"); => "507572654261736963"
Debug HexToString("507572654261736963"); => "PureBasic"

[rsts]

Nice code wilbert.

Added to toolbox. Thanks.

[Little John]

I need to encode a string (single byte or unicode converted to utf8) to a new string which contains hex. The functions in that thread are needing much conversion with PeekS() and PokeS() etc...

I know that you need to encode a string. Strings are easily handled by those procedures as well, because with @ we can get their memory addresses. In ASCII mode, there is no PeekS() or PokeS() requiered at all. In Unicode mode, according to your needs, one PokeS() would be required ...
It is pretty fine for me when you do not use those functions, but I don't want the forum readers to get a wrong impression about them.

[Kukulkan]

Whow wilbert,

thank you! This is really fast (only 16ms instead of 4300ms)!

I will use the new function now and this really speeds up my program!

Anyway, the comment of Little John makes me curious. How would such implementation look like?

Kukulkan

[davido]

Hi wilbert,

Thank you for sharing your code! Lots to learn - must study it carefully.

Astonishing speed for string manipulation.

[flaith]

Nice piece of code

[wilbert]

Astonishing speed for string manipulation.

That's because the main loop is kept very simple and doesn't call any string manipulation functions.

Anyway, the comment of Little John makes me curious. How would such implementation look like?

As far as I understand the routines created by Little John focus on versatility (you can control a lot of output options) while I focused on speed alone.
But maybe he can explain better himself.

[Little John]

Hello Wilbert!

As far as I understand the routines created by Little John focus on versatility (you can control a lot of output options) while I focused on speed alone.
But maybe he can explain better himself.

I think it's not possible to explain it better.