Simple preprocessor for string obfuscation

Share your advanced PureBasic knowledge/code with the community.
User avatar
Keya
Addict
Addict
Posts: 1890
Joined: Thu Jun 04, 2015 7:10 am

Simple preprocessor for string obfuscation

Post by Keya »

I wanted to hide strings in my program, but keep the strings as plain readable strings in the source code because i find encrypted strings in source code annoying to create and annoying to read!

i read that Purebasic allow you to create preprocessors, so here is my simple attempt, lol. Im really happy how easy it has made it to use plain strings in my source and encrypt them only at compiletime! :)

This is just a minimalist template.

Ive set my compiled preprocessor to be always on, but selectively activated from within my sourcecode if I simply type "#XSTR=1" near the top of my code.

When it detects #XSTR=1 it goes through the temporary copy of the source code its about to compile, and encrypts all XStr("") strings. In this demo to keep things basic, encryption is simply Chr+127, so we're just shifting the entire normal character set of 0-127 into the extended range of 128-255, making it look garbled under a hex editor in one quick swoop.

So a sample program might look like this:

Code: Select all

#XSTR=1
MessageRequester("Normal", XStr("Protected"),0)
Where "Normal" remains a normal unprotected string, and "Protected" will be protected with our obfuscation/encryption at compiletime, resulting in this being compiled:

Code: Select all

Procedure.s XStr(strbuf.s)
 ;-snip-...decrypt strbuf...
EndProcedure
MessageRequester("Normal", XStr("Ðòïôåãôåä"), 0)
It couldnt be much simpler!?:)

The strings are only decrypted when they're accessed, so if you simply run the program and dump memory they should still be encrypted if you havent accessed them.

Here is XSTR.PB which is basically just the decryption code that the preprocessor string-replaces "#XSTR=1" with:

Code: Select all

CompilerIf #PB_Compiler_Unicode = 1
	MessageRequester("PREPROCESSOR", "UNICODE NOT SUPPORTED",0)
	End
CompilerEndIf
 
Procedure.s XStr(buf.s)
  If Len(buf) = 0
    ProcedureReturn
  EndIf
  Protected *pbyte.ASCII = @buf
  For i.l = 1 To Len(buf)
    *pbyte\a = *pbyte\a - 127      ;Decrypt
    *pbyte + 1
  Next i
  ProcedureReturn buf
EndProcedure

Here is the PREPROCESSOR.PB. Compile it as an EXE in the same location as the XSTR.PB file
(this version doesnt support INCLUDEs. See next post for one that does)

Code: Select all

#XSTR=0  ;Tells the preprocessor (yes, this one!) to ignore this file.

Procedure EncStr(bufaddr.l, buflen.l)
  If buflen = 0
    ProcedureReturn
  EndIf
  Protected *pbyte.ASCII = bufaddr
  For i.l = 1 To buflen
    *pbyte\a = *pbyte\a + 127        ;Encrypt
    *pbyte + 1
  Next i
EndProcedure

;---------------------------------------------------------------------------------------------------------------------

If CountProgramParameters() = 0
  MessageRequester("PreProcessor error", "No parameters!")
  End
EndIf

;Parameter = temp copy of the source code that the compiler is about to compile
sFile.s = ProgramParameter(0)
sFile = RemoveString(sFile, Chr(34))
If ReadFile(0, sFile)
  length = Lof(0)                            ; get the length of opened file
  bytes.s = ReadString(0, #PB_UTF8 | #PB_File_IgnoreEOL, length)
  CloseFile(0)
EndIf

;Check to see if the source file has been marked for preprocessing ... if not we'll simply exit (no preprocessing required)
If FindString(bytes,"#XSTR=0") <> 0 Or FindString(bytes,"#XSTR=1") = 0
  End
EndIf

;Create a lowercase version of the source code so we can search case-insensitively
lbytes.s = LCase(bytes)
Repeat
  i.l = FindString(lbytes, "xstr(", i+1)
  If i = 0
    Break
  EndIf  
  i.l = FindString(lbytes, Chr(34), i)
  i2.l = FindString(lbytes, Chr(34), i+1)
  If i = 0 Or i2 = 0
    Break
  EndIf 
  ;We've found an Xstr("") string... lets encrypt it
  EncStr(@bytes + i, i2-i-1)
Until i = 0

XstrFile.s = GetPathPart(ProgramFilename()) + "\XSTR.PB"
lFilesize.l = FileSize(XstrFile)
If lFilesize = 0
  MessageRequester("PreProcessor error", XstrFile + " filesize = 0",0)
  End
EndIf

If ReadFile(0, XstrFile)
  XStrFunc.s = ReadString(0, #PB_UTF8 | #PB_File_IgnoreEOL, lFilesize)
  CloseFile(0)
EndIf

;Replace "#XSTR=1" with the actual decryption function
bytes = ReplaceString(bytes, "#XSTR=1", #CRLF$ + XStrFunc)
If Left(bytes,1) = Chr($3F)  ;need to get rid of utf8 byte
  bytes = Right(bytes, Len(bytes) - 1)
EndIf

;Save the modified temporary source file
DeleteFile(sFile)
OpenFile(0, sFile)
WriteData(0, @bytes, Len(bytes))
CloseFile(0)

TO INSTALL
1) Compile preprocessor as an EXE, and save it and XSTR.PB to the same location
2) In Purebasic go to Tools, then Configure Tools, and click New, and add the preprocessor Exe
3) Set the Arguments to: "%COMPILEFILE"
4) Give it a name, "XStr Preprocessor" or something, and check "Wait for tool to finish"
5) for Event To Trigger The Tool, set to "Before Compile/Run"
Now repeat steps 2 to 5, but set the second Event Trigger to "Before Create Executable"

That's all! You can choose to use it on a per-case basis, but I just always leave it enabled because it does nothing if you dont specify in your code you want it (which you do simply by specifying "#XSTR=1", and then simply add XStr("") around any strings you want to protect) :)

Does anyone have any similar preprocessors, or any tips to share about this? I searched the forums but couldnt find many

ps. When designing your encryption/obfuscation routine remember that (at least in this simple implementation that uses PB's ascii native null-terminated string type) we cannot encode any character to either Chr(0) null, or Chr(34) speechmark, and I'm pretty sure Chr(13) and Chr(10) would break it also due to becoming a new line. Remember the encrypted text still needs to read as valid text within valid code. All other characters should be safe to use, including ones that dont look valid in the IDE, because the IDE isn't a part of the compile chain in this sense.
Last edited by Keya on Sat Aug 08, 2015 12:46 am, edited 19 times in total.
User avatar
aaaaaaaargh
User
User
Posts: 55
Joined: Thu Jul 27, 2006 1:24 pm

Re: Simple preprocessor for string obfuscation

Post by aaaaaaaargh »

Nice idea, and a good template for other preprocessors.
User avatar
Keya
Addict
Addict
Posts: 1890
Joined: Thu Jun 04, 2015 7:10 am

Re: Simple preprocessor for string obfuscation

Post by Keya »

This version of the preprocessor supports INCLUDEs, so we can use XStr()-obfuscated strings in include'd files too. This is how the first version should've been :)

Its PARAMETERS for usage when you Install it as a Purebasic tool are: "%COMPILEPATH" "%PATH"
(whereas the old version was just: "%COMPILEPATH")

Basically, the problem was that PB sends our preprocessor "pb_editoroutput.pb" which is a copy of the main source file that we're free to modify before it gets sent to the compiler... but the INCLUDE/XINCLUDE statements remain as they are, so we have to walk through the main source file and manually replace the include statements by inserting any included files. Not too much extra work.

Code: Select all

#XSTR=0  ;Tells the preprocessor (yes, this one!) to ignore this file.
;(Normally we dont even need to do that if we don't want to preprocess the file, but we can specify it in cases 
;like this where the source file might have the "#XSTR=1" string but we dont actually want to preprocess it.)

Procedure EncStr(bufaddr.l, buflen.l)
  If buflen = 0
    ProcedureReturn
  EndIf
  Protected *pbyte.ASCII = bufaddr
  For i.l = 1 To buflen
    *pbyte\a = *pbyte\a + 127        ;Encrypt
    *pbyte + 1
  Next i
EndProcedure

;---------------------------------------------------------------------------------------------------------------------

Procedure.l AddIfUnique(sItem.s)
  Static arCnt.l
  Static Dim arItem.s(255)  ;shouldnt be more than 255 include files!
  If arCnt = 0
    Goto AddElement
  EndIf
  For i.l = 0 To arCnt-1
    If arItem(i) = sItem 
      ProcedureReturn 0    ;Already exists in array
    EndIf
  Next i
  AddElement:
  arItem(arCnt) = sItem
  arCnt = arCnt + 1
  ProcedureReturn 1        ;Added to static array
EndProcedure


Procedure CreateMergedSourceFile(sMainSrcFile.s, sMainSrcPath.s, sIncludesPath.s, sOutputFile.s)
  sOut.s = ""
  sIncPath.s = sIncludesPath
  If Not ReadFile(0, sMainSrcPath + "\" + sMainSrcFile)   ; if the file could be read, we continue...
    MessageRequester("PreProcessor error","Couldn't open Main source file " + sMainSrcFile)
    ProcedureReturn
  EndIf
  While Eof(0) = 0
    sLine.s = ReadString(0)
    Select UCase(Left(LTrim(sLine.s),11))
      Case "INCLUDEPATH":   ;changing to different include path
        i.l = FindString(sLine, Chr(34))
        i2.l = FindString(sLine,Chr(34),i+1)
        sIncPath = LCase(Trim(Mid(sLine,i+1,i2-i-1)))        
      Case "XINCLUDEFIL", "INCLUDEFILE":    ;in this implementation i only ever include the same file once
        i.l = FindString(sLine, Chr(34))
        i2.l = FindString(sLine,Chr(34),i+1)
        sIncFile.s = LCase(Trim(Mid(sLine,i+1,i2-i-1)))
        If AddIfUnique(sIncFile) = 1        
          If Not ReadFile(1, sIncPath + "\" + sIncFile)
            MessageRequester("PreProcessor error","Couldn't open Include file " + sIncPath + "\" + sIncFile)
            ProcedureReturn
          EndIf
          sLine = ReadString(1,#PB_Ascii | #PB_File_IgnoreEOL)
          If Left(sLine,1) = Chr($3F)  ;need to get rid of utf8 byte
            sLine = Right(sLine, Len(sLine) - 1)
          EndIf
          For i = 0 To Len(sLine)-1   ;trim those weird (utf8?) chars from the start of the include file
            If PeekA(@sLine+i) < 127
              sLine = Right(sLine, Len(sLine) - i)
              Break
            EndIf
          Next i 
        EndIf
    EndSelect
    sOut = sOut + sLine + #CRLF$
  Wend
  CloseFile(0)
  If OpenFile(0, sOutputFile)
    WriteData(0, @sOut, Len(sOut))
    CloseFile(0)
  EndIf
EndProcedure


If CountProgramParameters() < 2
  MessageRequester("PreProcessor error", "Usage: " + Chr(34) + "%COMPILEFILE" + Chr(34) + " " + Chr(34) + "%PATH" + Chr(34))
  End
EndIf

;Parameter = temp copy of the source code that the compiler is about to compile
sFile.s = ProgramParameter(0)
sFile = RemoveString(sFile, Chr(34))

sIncPath.s = ProgramParameter(1)
sIncPath = RemoveString(sIncPath, Chr(34))
sMainSrcFile.s = GetFilePart(sFile)
sMainSrcPath.s = GetPathPart(sFile)
sOutputFile.s = sMainSrcPath + "\" + sMainSrcFile  ;"output.pb"
CreateMergedSourceFile(sMainSrcFile, sMainSrcPath, sIncPath, sOutputFile)


If ReadFile(0, sFile)
  length = Lof(0)                            ; get the length of opened file
  bytes.s = ReadString(0, #PB_UTF8 | #PB_File_IgnoreEOL, length)
  CloseFile(0)
EndIf

;Check to see if the source file has been marked for preprocessing ... if not we'll simply exit (no preprocessing required)
If FindString(bytes,"#XSTR=0") <> 0 Or FindString(bytes,"#XSTR=1") = 0
  End
EndIf

;Create a lowercase version of the source code so we can search case-insensitively
lbytes.s = LCase(bytes)
Repeat
  i.l = FindString(lbytes, "xstr(", i+1)
  If i = 0
    Break
  EndIf  
  i.l = FindString(lbytes, Chr(34), i)
  i2.l = FindString(lbytes, Chr(34), i+1)
  If i = 0 Or i2 = 0
    Break
  EndIf 
  ;We've found an Xstr("") string... lets encrypt it
  EncStr(@bytes + i, i2-i-1)
Until i = 0

XstrFile.s = GetPathPart(ProgramFilename()) + "\XSTR.PB"
lFilesize.l = FileSize(XstrFile)
If lFilesize = 0
  MessageRequester("PreProcessor error", XstrFile + " filesize = 0",0)
  End
EndIf

If ReadFile(0, XstrFile)
  XStrFunc.s = ReadString(0, #PB_UTF8 | #PB_File_IgnoreEOL, lFilesize)
  CloseFile(0)
EndIf

;Replace "#XSTR=1" with the actual decryption function
bytes = ReplaceString(bytes, "#XSTR=1", #CRLF$ + XStrFunc + #CRLF$)

If Left(bytes,1) = Chr($3F)  ;need to get rid of utf8 byte
  bytes = Right(bytes, Len(bytes) - 1)
EndIf
For i = 0 To Len(bytes)-1   ;trim those weird (utf8?) chars from the start of the include file
  If PeekA(@bytes+i) < 127
    bytes = Right(bytes, Len(bytes) - i)
    Break
  EndIf
Next i 

;Save the modified temporary source file
DeleteFile(sFile)
OpenFile(0, sFile)
WriteData(0, @bytes, Len(bytes))
CloseFile(0)
User_Russian
Addict
Addict
Posts: 1519
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Simple preprocessor for string obfuscation

Post by User_Russian »

Encryption strings without preprocessing.

Code: Select all

!macro ppublic name{
        !if name eq _SYS_StaticStringEnd
                !repeat $-_SYS_StaticStringStart
                        !load zczc from _SYS_StaticStringStart+%-1
                        !store zczc xor 137 at _SYS_StaticStringStart+%-1
                !end repeat
        !end if
        !public name}
!public fix ppublic
CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
  !mov edi,_SYS_StaticStringStart
  !mov ecx,_SYS_StaticStringEnd-_SYS_StaticStringStart
  !@@:
        !xor byte[edi],137
        !inc edi
        !dec ecx
CompilerElse
  !mov rdi,_SYS_StaticStringStart
  !mov rcx,_SYS_StaticStringEnd-_SYS_StaticStringStart
  !@@:
        !xor byte[rdi],137
        !inc rdi
        !dec rcx
CompilerEndIf
!jnz @b
 

MessageRequester("test test test", "Find this string in exe file")
User avatar
Keya
Addict
Addict
Posts: 1890
Joined: Thu Jun 04, 2015 7:10 am

Re: Simple preprocessor for string obfuscation

Post by Keya »

Wow, kung fu! lol :) :) :)
Thats really cool, thankyou for your sharing :)

If im understanding it correctly it gets the FASM assembler to do some preprocessing of its own, to encrypt the whole string table (so i guess this works for Win+Linux but not Mac as PB uses YASM there)

I see in the .ASM file that PB compiler generates where youre encrypting:

Code: Select all

public _SYS_StaticStringStart
_SYS_StaticStringStart:
_S2: db "Find this string in exe file",0
_S1: db "test test test",0
pb_public PB_NullString
  db     0
public _SYS_StaticStringEnd
_SYS_StaticStringEnd:

Interestingly it seems to work perfectly fine alongside my preprocessor also, two levels of obfuscation heehee :)
They do seem completely independent from each other so it doesnt seem like either method will cause the other problems

The only downside of your method on its own is that all the strings are decrypted straight away at startup (whereas mine are only decrypted on a per-string basis when they're individually needed), and I guess it has to be that way seeing as it encrypts not each string but the whole string table in one go, so if you set a breakpoint at the start of your code (but immediately after the decrypt code) in a debugger it looks as if no strings were ever encrypted, but I really love the idea of combining it with mine for two levels of obfuscation! :)

Actually im going to modify my preprocessor to also accept "#XSTRTBL=1" so it'll automagically insert your kung fu code heehee. So cool! :)

Code: Select all

If FindString(bytes, "#XSTRTBL=1") <> 0
  XStrTblFile.s = GetPathPart(ProgramFilename()) + "\XSTRTBL.PB"   ;User_Russian's fasm preprocessor kung fu code :)
  If ReadFile(1, XStrTblFile)
    XStrTblFunc.s = ReadString(1, #PB_UTF8 | #PB_File_IgnoreEOL, FileSize(XStrTblFile))
    If Left(XStrTblFunc,1) = Chr($3F)  ;need to get rid of utf8 byte
      XStrTblFunc = Right(XStrTblFunc, Len(XStrTblFunc) - 1)
    EndIf
    For i.l = 0 To Len(XStrTblFunc)-1   ;trim those weird (utf8?) chars from the start of the include file
      If PeekA(@XStrTblFunc+i) < 127
        XStrTblFunc = Right(XStrTblFunc, Len(XStrTblFunc) - i)
        Break
      EndIf
    Next i 
    CloseFile(1)
    bytes = ReplaceString(bytes, "#XSTRTBL=1", #CRLF$ + XStrTblFunc + #CRLF$)
  EndIf
EndIf

Code: Select all

#XSTR=1     ;use my preprocessor, applied on a per-string basis
#XSTRTBL=1  ;use User_Russians fasm preprocessor, applied to entire string table
MessageRequester("This string protected by XSTRTBL", XStr("This string protected by both XSTR + XSTRTBL"))
User avatar
RichAlgeni
Addict
Addict
Posts: 935
Joined: Wed Sep 22, 2010 1:50 am
Location: Bradenton, FL

Re: Simple preprocessor for string obfuscation

Post by RichAlgeni »

User_Russian wrote:Encryption strings without preprocessing.
I really like this @User_Russian, thanks!

A couple of caveats: it cannot be used inside an include, otherwise the include strings will be obfuscated, but not the parent program, or the rest of the included programs. It must be placed at the beginning of the program.

One thing it can do, and what I will use it for is to obfuscate the strings of a dll. The dll can be called with no issues from a program that is not obfuscated.
User avatar
Olliv
Enthusiast
Enthusiast
Posts: 542
Joined: Tue Sep 22, 2009 10:41 pm

Re: Simple preprocessor for string obfuscation

Post by Olliv »

Thank you Keya for opening this subject, and thank you User Russian for insuring the link and low source. I copy it to a french page.
User avatar
Olliv
Enthusiast
Enthusiast
Posts: 542
Joined: Tue Sep 22, 2009 10:41 pm

Re: Simple preprocessor for string obfuscation

Post by Olliv »

Hello,

'load' and 'store':
Have we got more informations about these expression?
See on both documentation of FASM. No found.

I suppose it belongs purebasic compiler itself?
firace
Addict
Addict
Posts: 946
Joined: Wed Nov 09, 2011 8:58 am

Re: Simple preprocessor for string obfuscation

Post by firace »

Olliv wrote:Hello,

'load' and 'store':
Have we got more informations about these expression?
See on both documentation of FASM. No found.

I suppose it belongs purebasic compiler itself?
http://flatassembler.net/docs.php?article=manual#2.2.4
User avatar
Olliv
Enthusiast
Enthusiast
Posts: 542
Joined: Tue Sep 22, 2009 10:41 pm

Re: Simple preprocessor for string obfuscation

Post by Olliv »

Exactly!
I was thinking it didn't belong to the set of FAsm directives.
Jeromyal
Enthusiast
Enthusiast
Posts: 216
Joined: Wed Jul 17, 2013 8:49 am

Re: Simple preprocessor for string obfuscation

Post by Jeromyal »

Is this all still valid with PB 6 ?
novablue
Enthusiast
Enthusiast
Posts: 177
Joined: Sun Nov 27, 2016 6:38 am

Re: Simple preprocessor for string obfuscation

Post by novablue »

User_Russian wrote: Sat Aug 08, 2015 5:44 am Encryption strings without preprocessing.

Code: Select all

!macro ppublic name{
        !if name eq _SYS_StaticStringEnd
                !repeat $-_SYS_StaticStringStart
                        !load zczc from _SYS_StaticStringStart+%-1
                        !store zczc xor 137 at _SYS_StaticStringStart+%-1
                !end repeat
        !end if
        !public name}
!public fix ppublic
CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
  !mov edi,_SYS_StaticStringStart
  !mov ecx,_SYS_StaticStringEnd-_SYS_StaticStringStart
  !@@:
        !xor byte[edi],137
        !inc edi
        !dec ecx
CompilerElse
  !mov rdi,_SYS_StaticStringStart
  !mov rcx,_SYS_StaticStringEnd-_SYS_StaticStringStart
  !@@:
        !xor byte[rdi],137
        !inc rdi
        !dec rcx
CompilerEndIf
!jnz @b
 

MessageRequester("test test test", "Find this string in exe file")
Is there something similar that works with the c backend compiler?
Olli
Addict
Addict
Posts: 1200
Joined: Wed May 27, 2020 12:26 pm

Re: Simple preprocessor for string obfuscation

Post by Olli »

Useless. The better way is having zero strings in the source code in the final compilation, whatever the source, opened or closed.
Olliv wrote: Sat Aug 15, 2015 8:59 pm Thank you Keya for opening this subject, and thank you User Russian for insuring the link and low source. I copy it to a french page.
In the french title subject which was done from this subject (french page ) you could read the "factice" word. What it does not prevent to create a native encryption for final compilation (normal source file = source without string file + encrypted strings file).

Good time.
Post Reply