simple flag for writeable code section

[Keya]

Hello, i would like to decompress code in memory as ive seen packers do, and also make optimizations to my code in memory if i detect a particular CPU, but they both require writeable+executable memory section

How about something like a "EnableCodeWrite" statement, so that when Purebasic is writing its asm it simply includes the word "writeable" to the section header statement:

section '.code' code readable writeable executable align 4096

Here is a simple example of whats then possible - http://www.xenoinc.org/code/asm/fasm/ex/self-encrypt

seems like it would be very quick and easy to add, yet a very powerful addition to Purebasic - dynamic code. Thankyou for your consider

[Fred]

I think you can reserve some space in your current code section using '!nop', and the fill it with new code.

[Keya]

Space isn't the problem - writing is - because the .code section is only Readable+Executable ... yes I can use NOPs to make space for new code, but can't write any code there without the section being Writeable also

Another wish - the ability to specify flags (Write/Execute/Read) with AllocateMemory()

(but BOTH would be great, and one doesn't render the other obsolete as they both have different pros+cons!)

[Fred]

I see, code is usually readable only for security reasons, but I sometimes it could be helpful.

[GPI]

I see, code is usually readable only for security reasons, but I sometimes it could be helpful.

One Problem would be, that many Virusscanner would detect this as malware. Also it is possible that future or current os would not support this. In my personal opinion this is a very bad programing style.

[Keya]

One Problem would be, that many Virusscanner would detect this as malware.

This might sound like a good theory but is completely impractical because simply by detecting a writeable code section as malware would be false alarm city!!!

Remember a lot of commercial programs also use packers, and some software simply cannot function without this capability. It is an important capability and you can do some amazing things with it (consider Alexia Massalin and her Synthesis kernel!)

Of the 10 primary uses listed at Wikipedia's self-modifying code page, only 1 is for potential "bad" purposes ie hiding code (but that exact same purpose is also used to protect commercial software), so please let's not be close-minded about this:

1. Semi-automatic optimizing of a state dependent loop.
2. Run-time code generation, or specialization of an algorithm in runtime or loadtime (which is popular, for example, in the domain of real-time graphics) such as a general sort utility – preparing code to perform the key comparison described in a specific invocation.
3. Altering of inlined state of an object, or simulating the high-level construction of closures.
4. Patching of subroutine (pointer) address calling, usually as performed at load/initialization time of dynamic libraries, or else on each invocation, patching the subroutine's internal references to its parameters so as to use their actual addresses. (i.e. Indirect 'self-modification').
5. Evolutionary computing systems such as genetic programming.
6. Hiding of code to prevent reverse engineering (by use of a disassembler or debugger) or to evade detection by virus/spyware scanning software and the like.
7. Filling 100% of memory (in some architectures) with a rolling pattern of repeating opcodes, to erase all programs and data, or to burn-in hardware.
8. Compressing code to be decompressed and executed at runtime, e.g., when memory or disk space is limited.
9. Some very limited instruction sets leave no option but to use self-modifying code to perform certain functions. For example, a one instruction set computer (OISC) machine that uses only the subtract-and-branch-if-negative "instruction" cannot do an indirect copy (something like the equivalent of "*a = **b" in the C language) without using self-modifying code.
10. Altering instructions for fault-tolerance.

Also it is possible that future or current os would not support this.

Why? Just because some malware use packers? Again, so does a lot of commercial software, and again some things can only be done when the code section is writeable.
And im not writing for future OS, just current... and even if future OS did for whatever reason say "ok no more writeable code sections", thats not a problem because we don't have to use it for regular programs! Its optional - you would have to enable it manually with a statement like my "EnableCodeWrite" example Code sections are ALLOWED to be writeable for a reason, and simply banning them would be on par with banning DeleteFile API because malware might use it to delete your data.
Unlike the DeleteFile API however, by very definition self-modifying code can't do any harm to anything other than itself!

And im not even sure making your exe harder to disassemble counts as harm -- that's the "worst" that self-modifying code can do. That's really it's only "crime", the only real reason it sometimes gets a bad wrap.

In my personal opinion this is a very bad programing style.

That's fine, because naturally it would be off by default - exactly as it is now, so you wouldn't even be using it, so there's no problem and you don't have to complain about other people using a feature they want to use that you dont

Again I need this functionality for legitimate reasons. There is no need for OPINION when it comes to this
So let's put opinion aside and see what actually happens when we scan with 50+ virus scanners at virustotal.com heehee ...

I created this single line program: MessageRequester("Hello","Hello") .... I disabled debugger, compiled in ascii mode, for both 32 and 64.

Its important to note here that a PB exe has two main code sections - ".text" where the entrypoint and initialization code is, and then that jumps over to the ".code" where all our user code is.
Both sections by default are RW, but not X. By comparison a typical UPX-packed exe also has two code sections .UPX0 and .UPX1, but they're both RWX. So by your theory every single UPX-packed exe would be flagged by scanners

I'll try 1) original PB exes (32bit + 64bit) that simply say MessageRequester "Hello"
2) the originals, but with .code section RW -> RWX
3) the originals, but with .code + .text sections RW -> RWX
4) the originals, but UPX packed (so, both .UPX0 and .UPX1 as RWX)

Virustotal Results:
32bit original unmodified:
1/54:
Qihoo-360: "HEUR/QVM05.1.Malware.Gen" .... i hadnt heard of Qihoo-360, its a Chinese AV. It can only be used to scan 32bit exes, it doesnt support 64 - lame!

64bit original unmodified:
0/54 (100% clean) ..... ok good, our 64bit msgbox hello world is clean.

32bit, .code section patched RW -> RWX
4/55:
Avast: "Win32:Evo-gen [Susp]"
Qihoo-360: "HEUR/QVM05.1.Malware.Gen"
TrendMicro: "PAK_Generic.001"
TrendMicro-HouseCall: "PAK_Generic.001"
... so really only TrendMicro is flagging it. Avast we can ignore (not as Generic, but as a false alert), because as we'll soon see Avast doesnt flag the same for 64bit exes, so its probably also a combination of that + something else in the exe. Qihoo we can ignore.

64bit, .code section RW -> RWX
0/53 (100% clean)

Now lets try BOTH the .code + .text sections as RWX, so that includes the section with the Entrypoint ...

32bit, .code + .text section RW -> RWX
3/55:
Qihoo-360: "HEUR/QVM05.1.Malware.Gen"
TrendMicro: "PAK_Generic.001"
TrendMicro: "PAK_Generic.001"

64bit, .code + .text section RW -> RWX
0/53 (100% clean)

Now lets try the original again, but UPX'd (latest - v3.91w) ...
32bit original, UPX'd
4/53:
Qihoo-360: "HEUR/QVM11.1.Malware.Gen"
TheHacker: "Posible_Worm32" <- anybody heard of "TheHacker" scanner? their spelling is as bad as their detection
TrendMicro: "PAK_Generic.005"
TrendMicro: "PAK_Generic.005"

64bit original, UPX'd
0/54 (100% clean)
---

So regardless of whether it's UPX-packed or we've simply manually made a Writeable executable section (or used the assembler itself ie. "section '.code' code readable writeable executable align 4096"), the answer is clearly NO - 99% of AV's will not falsely alert (only TrendMicro, and not as infected but just "generic packed" warning), so yes it's perfectly fine to use

I will still be contacting the above companies anyway to alert them of course, as all developers should do whenever they encounter a false alert from any scanner, and as recommended in this Purebasic blog

[GPI]

Why? Just because some malware use packers?

Nope. Because for example a maleware trys to overwrite the code.

About UPX-Packer: Stupid question, but does this not need more ram. the packed and unpacked?

HDD-Space is cheap. I don't really see any advantage with packing of exes.

[Dude]

HDD-Space is cheap. I don't really see any advantage with packing of exes.

Two examples are for smaller data use by email (especially when sending files by phone with small data plans), and storage on small USB sticks.

[Keya]

Why? Just because some malware use packers?

Nope. Because for example a maleware trys to overwrite the code.

Nope. Since when does malware "have to overwrite the code"?!? Polymorphic viruses perhaps, but they're obviously not the norm.

About UPX-Packer: Stupid question, but does this not need more ram. the packed and unpacked?

you have the decompression code which is usually less than 1kb, then the packed image, so the file image is tiny, and for RAM the same amount is required for the unpacked image. Of course, a sloppy packer is free to allocate more memory than it needs to if it wants to! but yes it only needs the same as the 'normal' unpacked version, no less and rarely more as the decompression code is often tiny enough to fit into slack space (otherwise simply increase the RAM by 1kb and voila).

Packed executables also load/execute faster due to less disk access, so if you consider a 500kb exe vs a 5mb exe on a slower machine with a HDD

Packed exes also means your customers on slower internet can download them faster, and your server uses less bandwidth hosting the program.

... but that's just PACKED exes of course, and there's about ten other non-packing uses for dynamic code. Keep your mind open!

I don't really see any advantage with packing of exes.

But again that's just close-minded personal opinion "YOU shouldnt program this, because I personally wouldn't"! I'm not stopping anyone from using Goto if they want! and again that's only _packed_ exes your talking about.

Purebasic isn't a nanny state! and if YOU don't want to use a particular compiler option you don't have to, but that doesn't mean you should stop others from using legitimate technologies. Please keep your mind open - the word malware shouldn't even exist in this thread.

[Kurzer]

Since I'm working on a rudimentary protection system for third party programs, this feature would be really great. I could realize completely different approaches of protection. For example, calling encrypted procedures that decode only for execution and then encrypt again immediately to make it more difficult for crackers when debugging an exe.

Yes, GPI, I know what you're thinking about it, we had already spoken in the german forum about this practice. But for me it would be a real killer feature and would improve my existing protection method extremely.

So, +1 from me

[wilbert]

I suppose JIT compilers also write code to memory and execute it.

[Keya]

wilbert, you are spot on... no dynamic code = no JIT compilers! and we don't see every java program getting pulled up as malware (unfortunately!? heehee)

Like JIT, CPU optimization is another thing im keen on, for a simple example you can detect if the CPU family is F or 6 which indicates it supports multi-byte NOP so you can then patch memory so that 9 NOPs run in 1 cycle instead of 9 - now we have speed alignment + instruction speed! and this means you only need the one copy of your algorithm, not two.

GPI seriously if you're going to blindly throw all writeable code sections in the malware basket you should also detect programs using DeleteFile() api as malware, because whereas self-modifying code can only modify/harm itself (it's really just to hinder analysis like kurzer pointed out - dynamic/self-mod code does no actual harm and whatever it does it only does to itself), calling DeleteFile actually can do harm.

But again - you don't want to use it, and that's totally fine because you don't have to use it (it'd be off by default of course), but those of us who do need it can use it and we can all be happy

kurzer, you might also be interested in my dynamic execution in writeable+executable memory alloc demo for protection purposes, which doesn't allow you to modify any existing code (that's what we need this writeable .code section for) it does allow you to do the job in new memory allocations

[Kurzer]

kurzer, you might also be interested in my dynamic execution in writeable+executable memory alloc demo for protection purposes, which doesn't allow you to modify any existing code (that's what we need this writeable .code section for) it does allow you to do the job in new memory allocations

Very impressive, Keya. Thanks for the linkt to your post.