Convert to hex (HexIt)

[infratec]

I thought it should be placed in Tricks 'n' Tips

The idea was born from this thread:
https://www.purebasic.fr/english/viewtopic.php?t=78819

Save it as HexIt.pbi:

Code: Select all

;
; https://www.purebasic.fr/english/viewtopic.php?t=78858
;

CompilerIf #PB_Compiler_IsMainFile
  EnableExplicit
CompilerEndIf


DeclareModule HexIt
  Declare.s Byte(Byte.a)
  Declare.s Word(Word.u)
  Declare.s Long(Long.l)
  Declare.s Quad(Quad.q)
  Declare.s Integer(Integer.i)
  Declare.s Memory(*Ptr.Ascii, Length.i, ValuesPerLine.i=16, ShowString.i=#True, ShowAddress.i=#True)
  Declare.s String(String$, ValuesPerLine.i=16, Format.i=#PB_Unicode, ShowTrailingZero.i=#True, ShowString.i=#True, ShowAddress.i=#True)
EndDeclareModule



Module HexIt
  
  EnableExplicit
  
  DataSection
    HexChars:
    Data.a '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'
  EndDataSection
  
  
  Structure HexCharStructure
    Nibble.a[0]
  EndStructure
  
  
  Procedure.s Byte(Byte.a)
    
    Protected Result$, *Result, *ResultPtr.Ascii, *HexChar.HexCharStructure
    
    
    *Result = AllocateMemory(2)
    If *Result
      *ResultPtr = *Result
      *HexChar = ?HexChars
    
      *ResultPtr\a = *HexChar\Nibble[Byte >> 4]
      *ResultPtr + 1
      *ResultPtr\a = *HexChar\Nibble[Byte & $0F]
      
      Result$ = PeekS(*Result, 2, #PB_Ascii)
      
      FreeMemory(*Result)
    EndIf
    
    ProcedureReturn Result$
    
  EndProcedure
  
  
  Procedure.s Word(Word.u)
    
    Protected Result$, *Result, *ResultPtr.Ascii, *HexChar.HexCharStructure, *Ptr.Ascii
    
    
    *Result = AllocateMemory(4)
    If *Result
      *ResultPtr = *Result
      *HexChar = ?HexChars
      *Ptr = @Word + 1
      
      *ResultPtr\a = *HexChar\Nibble[*Ptr\a >> 4]
      *ResultPtr + 1
      *ResultPtr\a = *HexChar\Nibble[*Ptr\a & $0F]
      *ResultPtr + 1
      
      *Ptr - 1
      
      *ResultPtr\a = *HexChar\Nibble[*Ptr\a >> 4]
      *ResultPtr + 1
      *ResultPtr\a = *HexChar\Nibble[*Ptr\a & $0F]
      
      Result$ = PeekS(*Result, 4, #PB_Ascii)
      
      FreeMemory(*Result)
    EndIf
    
    ProcedureReturn Result$
    
  EndProcedure
  
  
  Procedure.s Long(Long.l)
    
    Protected Result$, *Result, *ResultPtr.Ascii, *HexChar.HexCharStructure, *Ptr.Ascii, i.i
    
    
    *Result = AllocateMemory(8)
    If *Result
      *ResultPtr = *Result
      *HexChar = ?HexChars
      *Ptr = @Long + 3
      
      For i = 1 To 4
        *ResultPtr\a = *HexChar\Nibble[*Ptr\a >> 4]
        *ResultPtr + 1
        *ResultPtr\a = *HexChar\Nibble[*Ptr\a & $0F]
        *ResultPtr + 1
        
        *Ptr - 1
      Next i
      
      Result$ = PeekS(*Result, 8, #PB_Ascii)
      
      FreeMemory(*Result)
    EndIf
    
    ProcedureReturn Result$
    
  EndProcedure
  
  
  Procedure.s Quad(Quad.q)
    
    Protected Result$, *Result, *ResultPtr.Ascii, *HexChar.HexCharStructure, *Ptr.Ascii, i.i
    
    
    *Result = AllocateMemory(16)
    If *Result
      *ResultPtr = *Result
      *HexChar = ?HexChars
      *Ptr = @Quad + 7
      
      For i = 1 To 8
        *ResultPtr\a = *HexChar\Nibble[*Ptr\a >> 4]
        *ResultPtr + 1
        *ResultPtr\a = *HexChar\Nibble[*Ptr\a & $0F]
        *ResultPtr + 1
        
        *Ptr - 1
      Next i
      
      Result$ = PeekS(*Result, 16, #PB_Ascii)
      
      FreeMemory(*Result)
    EndIf
    
    ProcedureReturn Result$
    
  EndProcedure
  
  
  Procedure.s Integer(Integer.i)
    
    CompilerSelect #PB_Compiler_Processor
      CompilerCase #PB_Processor_x86
        ProcedureReturn Long(Integer)
      CompilerCase #PB_Processor_x64
        ProcedureReturn Quad(Integer)
    CompilerEndSelect
    
  EndProcedure
  
  
  
  Procedure.s Memory(*Ptr.Ascii, Length.i, ValuesPerLine.i=16, ShowString.i=#True, ShowAddress.i=#True)
    
    Protected Result$, ShowString$, ByteCounter.l, BytesPerLineCounter.i, Byte.a, *HexChar.HexCharStructure, MaxLines.i, i.i
    Protected *LineBuffer, *EndPtr, *LinePtr.Ascii, *StringLinePtr.Ascii, *ByteCounter.Ascii
    Protected.i AddressBytes, HexBytes, StringBytes
    
    
    MaxLines = Length / ValuesPerLine + 1
    
    If ShowAddress
      AddressBytes = 8 + 2 ; 8 digits + 2 space
    EndIf
    
    HexBytes = (ValuesPerLine * 3)  ; 2 digits + space
    
    If ShowString
      StringBytes = 1 + ValuesPerLine  ; space + ValuesPerLine
    EndIf
    
    *LineBuffer = AllocateMemory((AddressBytes + HexBytes + StringBytes + 1) * MaxLines + 1, #PB_Memory_NoClear)  ; + 1) = LF    + 1 = terminating 0
    
    If ShowString
      *StringLinePtr = *LineBuffer + AddressBytes + HexBytes
      *StringLinePtr\a = ' '
      *StringLinePtr + 1
    EndIf
    
    
    *HexChar = ?HexChars
    *EndPtr = *Ptr + Length
    *LinePtr = *LineBuffer
    *ByteCounter = @ByteCounter
    While *Ptr < *EndPtr
      
      If BytesPerLineCounter = 0 And ShowAddress
        *ByteCounter = @ByteCounter + 3
        For i = 1 To 4
          Byte = *ByteCounter\a
          *LinePtr\a = *HexChar\Nibble[Byte >> 4]
          *LinePtr + 1
          *LinePtr\a = *HexChar\Nibble[Byte & $0F]
          *LinePtr + 1
          *ByteCounter - 1
        Next i
        *LinePtr\a = ' '
        *LinePtr + 1
        *LinePtr\a = ' '
        *LinePtr + 1
      EndIf
      
      Byte = *Ptr\a
      
      *LinePtr\a = *HexChar\Nibble[Byte >> 4]
      *LinePtr + 1
      *LinePtr\a = *HexChar\Nibble[Byte & $0F]
      *LinePtr + 1
      *LinePtr\a = ' '
      *LinePtr + 1
      
      If ShowString
        If Byte >= ' '
          *StringLinePtr\a = Byte
        Else
          *StringLinePtr\a = '.'
        EndIf
        *StringLinePtr + 1
      EndIf
      
      BytesPerLineCounter + 1
      ByteCounter + 1
      *Ptr + 1
      
      If BytesPerLineCounter = ValuesPerLine
        If ShowString
          *StringLinePtr\a = #LF
          *StringLinePtr + 1
          *LinePtr = *StringLinePtr
          *StringLinePtr = *LinePtr + AddressBytes + HexBytes
          *StringLinePtr\a = ' '
          *StringLinePtr + 1
        Else
          *LinePtr\a = #LF
          *LinePtr + 1
        EndIf
        BytesPerLineCounter = 0
      EndIf
      
    Wend
    
    If ShowString
      If BytesPerLineCounter = 0
        *StringLinePtr - AddressBytes - HexBytes - 2
      Else
        While BytesPerLineCounter < ValuesPerLine
          *LinePtr\a = ' '
          *LinePtr + 1
          *LinePtr\a = ' '
          *LinePtr + 1
          *LinePtr\a = ' '
          *LinePtr + 1
          BytesPerLineCounter + 1
        Wend
      EndIf
      *StringLinePtr\a = 0
    Else
      If BytesPerLineCounter = 0
        *LinePtr - 1
      EndIf
      *LinePtr\a = 0
    EndIf
    
    Result$ = PeekS(*LineBuffer, -1, #PB_Ascii)
    
    ;Debug Result$
    
    FreeMemory(*LineBuffer)
    
    ProcedureReturn Result$
    
  EndProcedure
  
  
  
  
  Procedure.s String(String$, ValuesPerLine.i=16, Format.i=#PB_Unicode, ShowTrailingZero.i=#True, ShowString.i=#True, ShowAddress.i=#True)
    
    Protected Result$, *Ptr, *Buffer, Length.i
    
    
    Select Format
      Case #PB_Ascii
        *Buffer = Ascii(String$)
        *Ptr = *Buffer
        Length = MemorySize(*Buffer)
        If Not ShowTrailingZero
          Length - 1
        EndIf
        
      Case #PB_UTF8
        *Buffer = UTF8(String$)
        *Ptr = *Buffer
        Length = MemorySize(*Buffer)
        If Not ShowTrailingZero
          Length - 1
        EndIf
        
      Default
        *Ptr = @String$
        Length = StringByteLength(String$)
        If ShowTrailingZero
          Length + 2
        EndIf
        
    EndSelect
    
    Result$ = Memory(*Ptr, Length, ValuesPerLine, ShowString, ShowAddress)
    
    If *Buffer
      FreeMemory(*Buffer)
    EndIf
    
    ProcedureReturn Result$
    
  EndProcedure
  
EndModule


;- Examples
CompilerIf #PB_Compiler_IsMainFile
  Debug HexIt::String("Hello World äöüß")
  Debug ""
  Debug HexIt::String("Hello World äöüß", 10)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Ascii)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 8, #PB_UTF8)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Unicode, #False)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Unicode, #False, #False)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Unicode, #True, #False)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Unicode, #True, #False, #False)
  Debug ""
  Debug HexIt::String("Hello World äöüß", 16, #PB_Unicode, #True, #True, #False)
  Debug ""
  Debug HexIt::Byte($3A)
  Debug ""
  Debug HexIt::Byte($F)
  Debug ""
  Debug HexIt::Word($1A3)
  Debug ""
  Debug HexIt::Long($341A3)
  Debug ""
  Debug HexIt::Quad($9A10CDF0341A3)
  Debug ""
  Debug HexIt::Integer($DF0341A3)
  
  
  Define Filename$, File.i, *Buffer, StartTime.q, Hex$, EndTime.q
  
  Filename$ = OpenFileRequester("Choose a file", "", "All|*.*", 0)
  If Filename$
    File = ReadFile(#PB_Any, Filename$)
    If File
      *Buffer = AllocateMemory(Lof(File), #PB_Memory_NoClear)
      If *Buffer
        If ReadData(File, *Buffer, MemorySize(*Buffer)) = MemorySize(*Buffer)
          CloseFile(File)
          StartTime = ElapsedMilliseconds()
          Hex$ = HexIt::Memory(*Buffer, MemorySize(*Buffer));, 16, #True, #False)
          EndTime = ElapsedMilliseconds()
          ;Debug Hex$
          
          LoadFont(0, "Courier New", 10)
          SetGadgetFont(#PB_Default, FontID(0))
          OpenWindow(0, 0, 0, 640, 480, "HexIt::Memory example", #PB_Window_MinimizeGadget|#PB_Window_ScreenCentered)
          EditorGadget(0, 0, 0, 640, 460, #PB_Editor_ReadOnly)
          CreateStatusBar(0, WindowID(0))
          AddStatusBarField(300)
          AddStatusBarField(100)
          AddStatusBarField(100)
          StatusBarText(0, 0, GetFilePart(Filename$), #PB_StatusBar_Center)
          StatusBarText(0, 1, "Size: " + Str(MemorySize(*Buffer)), #PB_StatusBar_Center)
          StatusBarText(0, 2, "Time: " + Str(EndTime - StartTime) + "ms", #PB_StatusBar_Center)
          
          SetGadgetText(0, Hex$)
          
          Repeat
          Until WaitWindowEvent() = #PB_Event_CloseWindow
          
        EndIf
        FreeMemory(*Buffer)
      EndIf
      If IsFile(File)
        CloseFile(File)
      EndIf
    EndIf
  EndIf
CompilerEndIf

I added the possibility to show addresses in front.

A 1.5MB PDF files takes 0.7 seconds to convert.
0.6 seconds without addressses.

The long part is to show the text

[AZJIO]

infratec
I tried this file, it did not paste into the window, although it writes 0.900 seconds. My code runs 2 times faster, 484 milliseconds.

[infratec]

The values above were with debugger on.
If I switch it off, the conversion takes 0.050 seconds without addresses.

And I just tested the wav file: it is shown in the editorgadget.

[infratec]

A picture with your file:



With addresses on:



Do exactly the same with your code and post it here. (also with options inside to show or not show addresses and string)
Then I can compare the results on my PC.

If your code is really faster, then you have my full respect.
Without assembler, I can not become faster.

[AZJIO]

https://i.imgur.com/kk6EnuI.png
https://i.imgur.com/DTXUavU.png
I used your time format exactly.
It seems to me that copying 3 bytes will be faster than copying one byte by moving the pointer. I don't know what it looks like in ASM.

What is the best way to copy character by character or several. It seems to me that the iteration functionality itself adds time.

Code: Select all

CopyMemoryString(aHex(*Ptr\a) + " ", @*MemPoint)

or

Code: Select all

CopyMemoryString(aHex(*Ptr\a), @*MemPoint)
CopyMemoryString(" ", @*MemPoint)

[infratec]

Your images show not your chorus wav file.
And the comparisson on my PC with your code shows that my code is 2 times faster.

The only way to make my version faster is to use a 256 byte table for the hex digits.
Maybe I try this.

[AZJIO]

Your images show not your chorus wav file.

I said earlier that this file could not be loaded into the window when using your code. Only showed the time in the status bar.
Intel(R) Pentium(R) CPU G3260 @ 3.30GHz 3.30 GHz
Win10_x64_1809

[infratec]

I changed the type of the ByteCount variable from .i to .l
Else it will not work correctly with a x64 compiler.

Since it is very fast now, I changed the time in the example from s to ms.
I also added some more infos in the StatusBar.

The file is now closed immediately after the ReadData, to make it possible to open it from an other program.

[infratec]

Since I need always something like:

Code: Select all

RSet(Hex(Byte), 2, "0"))

I thought it would be a good idea to implement a procedure for that.

I made now a module named HexIt out of this stuff.

Happy coding.

[idle]

Thanks very useful

[BarryG]

*Buffer = AllocateMemory(Lof(File), #PB_Memory_NoClear)

The problem with this line is that it's not good to use with very large files. At the moment, your code won't work with files larger than the installed RAM of the PC, or it will use a lot of free RAM for a large file and lose that amount of RAM for other running apps. My PC has 16 GB of RAM, so using your code on a 4 GB DVD file will chew up a massive quarter of my free RAM (and take forever to parse), and that's assuming that I've even got at least 4 GB of RAM free at the time.

Hex editors like HxD only load a small part of large files at a time to display, and get the next/prev section of the file with each scroll by the user. That's the better way to do it, and overcomes all the problems I just mentioned.

[infratec]

I didn't wrote a hex-editor or something similar.
As you can see in the code: it is an example how to use the functions.

If you or someone else want to write a full program, do it.

[BarryG]

No offense was intended, infratec.

[HeX0R]

What sense would it make to look at a 4GB file in a Hex Editor?
Are you looking for errors in the Matrix?

[BarryG]

I had to use a hex editor to work out why a 2 GB video file wasn't playing, despite the headers being intact and the correct codec installed. So yes, I was looking for errors and I did find the problem. It's not an uncommon process to use when you work with large files.