Code optimization
-
- Addict
- Posts: 1441
- Joined: Wed Nov 12, 2008 5:01 pm
- Location: Russia
Code optimization
Why in executable file, add all of procedures of source, even those that have never called in the code (not used)?
Why the compiler can not find such procedures and to exclude them from the compilation?
Why the compiler can not find such procedures and to exclude them from the compilation?
Re: Code optimization
Because PB doesn't do optimizations, apart form very simple ones such as constant folding
and replacing instructions by more efficient ones (e.g. replace ADD foo, 1 by INC foo).
Procedures that are not referenced won't be compiled, but this is of rather limited use.
and replacing instructions by more efficient ones (e.g. replace ADD foo, 1 by INC foo).
Procedures that are not referenced won't be compiled, but this is of rather limited use.
Blog: Why Does It Suck? (http://whydoesitsuck.com/)
"You can disagree with me as much as you want, but during this talk, by definition, anybody who disagrees is stupid and ugly."
- Linus Torvalds
Re: Code optimization
Because for some reason it was not programmed to do so. Obviously it could.User_Russian wrote: Why the compiler can not find such procedures and to exclude them from the compilation?
The problem is twofold.
User procedures referenced in some way but not from your main code flow: -> http://www.purebasic.fr/english/viewtop ... 03#p300303
Library functions even if inside unreferenced user procedures : -> http://www.purebasic.fr/english/viewtop ... 01#p425401
Considering in PB you need to use includes to reuse code (the alternative is to manually cut/paste just what you need every time you start a new project), this is something that should be addressed IMO.http://www.purebasic.com/faq.php wrote: Is PureBasic really small and fast ?
Yes. All has been optimized to give maximum speed and compactness to the programs created with PureBasic.
This, and variables defined and never used (zombies forgotten in your code).
"Have you tried turning it off and on again ?"
A little PureBasic review
A little PureBasic review
Re: Code optimization
+1 ...should be moved to the "feature request" (or "bug" ) forum though.
If any of you native English speakers have any suggestions for the above text, please let me know (via PM). Thanks!
Re: Code optimization
It would be nice to summarize the most efficient tool(s) to spot unused variables and procedures? I remember luis and trond posted some logic.
It would help if PB created an expanded file from all macros & includes & modules.
The /commented asm achieves some of this, but parsing that output requires quite an investment.
It would help if PB created an expanded file from all macros & includes & modules.
The /commented asm achieves some of this, but parsing that output requires quite an investment.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Re: Code optimization
If you browse a bit there must be some tools that strip out unused procedures. CodeCaddy does but I haven't tested it on the latest versions of PureBasic...
( PB6.00 LTS Win11 x64 Asrock AB350 Pro4 Ryzen 5 3600 32GB GTX1060 6GB)
( The path to enlightenment and the PureBasic Survival Guide right here... )
( The path to enlightenment and the PureBasic Survival Guide right here... )
Re: Code optimization
No one really works for every kind of code AFAIK, since it should be able to parse the source exactly as PB does (includes, constants substitution, macro expansions, conditional compilation, etc).blueznl wrote:If you browse a bit there must be some tools that strip out unused procedures.
I'm waiting to write mine "stripper" when we will get this -> http://www.purebasic.fr/english/viewtop ... =3&t=48235, or for PB to do it from the start.
"Have you tried turning it off and on again ?"
A little PureBasic review
A little PureBasic review
Re: Code optimization
This is especially painful when I XIncludeFile for 1 or 2 member Procedures. My exe jumps by almost the full contents of the Xinclude and sure enough, the commented asm contains Procedures I have not referenced.
I read in several places that a procedure that calls another procedure would be included even if the parent is never called. But, I fear that is not really what is happening in my case.
Does someone have a test case that shows the true exe inclusion process?
I read in several places that a procedure that calls another procedure would be included even if the parent is never called. But, I fear that is not really what is happening in my case.
Does someone have a test case that shows the true exe inclusion process?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Re: Code optimization
If you don't want to write a full PB parser (incl. pre-processor and macro-processor), best thing for now could beluis wrote:No one really works for every kind of code AFAIK, since it should be able to parse the source exactly as PB does (includes, constants substitution, macro expansions, conditional compilation, etc).blueznl wrote:If you browse a bit there must be some tools that strip out unused procedures.
to process the resulting .asm output file. Scan the procedure macros, including calls to other procedures,
then filter which procedures are called in the main part (built a call graph). Exclude the PB procedure macros that are never called.
Processing this ASM is 'relatively' simple, as you just have to look for labels generally ("CALL label", "LEA ***, [label]", etc.)
PB procedure macros:
Code: Select all
macro MPnum{
_ProcedureNUM: ; label inside procedure macro
; ...
l_proc1_label1: ; scan all labels inside procedures "[_A-Za-z0-9]:"
; ...
}
Code: Select all
CALL _ProcedureNUM
LEA eax, [_ProcedureNUM]
*label_inside_procedure_macro* ; scan generally for any label names that were found inside procedure macros before [_A-Za-z0-9]
Code: Select all
MP0
MP2
MP4
...
MP200
Last edited by Danilo on Sat Dec 28, 2013 12:10 pm, edited 1 time in total.
Re: Code optimization
I don't, for the reasons explained here -> http://www.purebasic.fr/english/viewtop ... 61#p403061danilo wrote:If you don't want to write a full PB parser (incl. pre-processor and macro-processor
That was the first thing I tried, since it would have removed the need to parse the source expanding macro, evaluating conditional compilation etc.. It was working and was very easy to do but I wasn't happy with the result.Danilo wrote:best thing for now could be to process the resulting .asm output file.
A lot of time is passed (a couple of years I think) so I don't remember it well, but I recall there were some external resources/dependencies imported by PB when it detects the usage of certain commands. When you just remove the calling macros you avoid the expansion of the body of your procedures, but the resources imported somewhere are staying there.
I would have to retry the process to be more specific, sorry, but I remember I thought "this is not working well, I would have to do this at the source level".
"Have you tried turning it off and on again ?"
A little PureBasic review
A little PureBasic review
Re: Code optimization
Sorry, but I am confused by the asm output with regards to unused and unreferenced procedures (called from unused procedures). I thought it was stated these would not be included in final exe?
I never checked the asm macro assignments before now, but I see many macros = completely unused procedures.
@Danilo, can you elaborate on your approach to parse the asm to build a table?
;-- FROM sample ASM --
; Procedure.i myProc(parameters...)
macro MP72{
_Procedure72:
;Pseudo code...
1. Build table of all 'CALL _ProcedureXXX' statements.
Don't I have the same problem PB has, since I am reporting CALL's from within unused procedures.
Or do I only search between:
'PureBasicStart:' and '_PB_EndFunctions:' ?
2. Build table of all '_ProcedureXXX' statements.
3. Build table of all 'macro MPxxx' statements.
What is different about the list of MPxxx's near the end of the ASM?
Many of those are also unused and unreferenced.
4. Build table of all original source code Procedure Names from preceding line 'macro MPxxx'.
; Procedure.x yyy(parameters...)
5. Search CALL table for all known '_ProcedureXXX' and dump a list containing only the used source code Procedures.
haha, which is faster to shrink my exe?
1. Manual cut and paste from many XIncludeFiles?
or
2. Write some code to parse the asm?
or
3. Write some code to parse the original code?
This seems the most difficult since Macros are not expanded.
I never checked the asm macro assignments before now, but I see many macros = completely unused procedures.
@Danilo, can you elaborate on your approach to parse the asm to build a table?
;-- FROM sample ASM --
; Procedure.i myProc(parameters...)
macro MP72{
_Procedure72:
;Pseudo code...
1. Build table of all 'CALL _ProcedureXXX' statements.
Don't I have the same problem PB has, since I am reporting CALL's from within unused procedures.
Or do I only search between:
'PureBasicStart:' and '_PB_EndFunctions:' ?
2. Build table of all '_ProcedureXXX' statements.
3. Build table of all 'macro MPxxx' statements.
What is different about the list of MPxxx's near the end of the ASM?
Many of those are also unused and unreferenced.
4. Build table of all original source code Procedure Names from preceding line 'macro MPxxx'.
; Procedure.x yyy(parameters...)
5. Search CALL table for all known '_ProcedureXXX' and dump a list containing only the used source code Procedures.
haha, which is faster to shrink my exe?
1. Manual cut and paste from many XIncludeFiles?
or
2. Write some code to parse the asm?
or
3. Write some code to parse the original code?
This seems the most difficult since Macros are not expanded.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Re: Code optimization
My guess would be that currently, PB has a boolean flag that just indicates if a Proc is called or not.
Instead of that boolean flag, procedure references (i.e. proc calls) need to be kept. One method would be in a map (the key being the names of procs) of lists (listing all procs called within the proc). Then, you iterate through, starting with the calls from outside of any proc, and only flag the procs that can be reached from the start.
Instead of that boolean flag, procedure references (i.e. proc calls) need to be kept. One method would be in a map (the key being the names of procs) of lists (listing all procs called within the proc). Then, you iterate through, starting with the calls from outside of any proc, and only flag the procs that can be reached from the start.
Re: Code optimization
It does not have to track anything.Tenaja wrote:My guess would be that currently, PB has a boolean flag that just indicates if a Proc is called or not.
As visible in the asm output here -> http://www.purebasic.fr/english/viewtop ... 03#p300303 PB generates the code for all the procedures but wraps them in asm macros. When a proc is referenced the macro is expanded and so its code is included in the final exe. If a proc is not referenced the macro is never called, the code is not expanded and the code is not included in the final exe.
The result is something good enough most of the times for small programs with zero efforts from the compiler, and at the same time the source of the problem discussed here.
Typical worst cases are included files with a lot of procedures referencing one another but with practically nothing called from the main program.
"Have you tried turning it off and on again ?"
A little PureBasic review
A little PureBasic review
Re: Code optimization
Exactly.
Struggling to scan only the "main" portion of the asm?
This is definitely a multi-pass problem. (all 5th element jokes go here )
If I exclude all macro MPxxx's{}, I am hoping to be left with just the main calls.
Then there is that curious list of MPxxx's which I don't understand why some are and some aren't included?
Either way, it would be better if this was solved by the IDE now that we have XIncludes and Modules.
Manually stripping include files/modules is not elegant.
Struggling to scan only the "main" portion of the asm?
This is definitely a multi-pass problem. (all 5th element jokes go here )
If I exclude all macro MPxxx's{}, I am hoping to be left with just the main calls.
Then there is that curious list of MPxxx's which I don't understand why some are and some aren't included?
Either way, it would be better if this was solved by the IDE now that we have XIncludes and Modules.
Manually stripping include files/modules is not elegant.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Re: Code optimization
I believe I know how the macro wrapping works (and have, for several years)...and I do not believe your description above is entirely accurate, because your description omits a critical step, which is actually supported by your link to the 2009 thread...luis wrote:It does not have to track anything.
As visible in the asm output here -> http://www.purebasic.fr/english/viewtop ... 03#p300303 PB generates the code for all the procedures but wraps them in asm macros. When a proc is referenced the macro is expanded and so its code is included in the final exe. If a proc is not referenced the macro is never called, the code is not expanded and the code is not included in the final exe.
The macro NOT expanded when (i.e. at the location) a proc is referenced. It is expanded at the very end of the asm file--this is supported by the fact that all of the macro "call" insertions--such as MP0, MP2, MP4, etc.--are listed on consecutive lines at the bottom of the asm code. Because of that, it follows that PB is somehow flagging (tracking) which procs require that macro expansion call to be inserted, and it does not insert the call (the MP0, MP2, etc) if the flag for that proc is not set.
The "root of the problem" (I believe) is that PB flags a proc to be expanded immediately once the proc is referenced (even, for instance, within an unused proc). This technique results in some unused procs being flagged for insertion. (I wouldn't be surprised if a map or list is used to store the flags, since the macro calls are not in ascending order, nor are they in the order called.)
The solution is to use a different technique--do not flag them for insertion during parsing, but rather merely track their use such as with a call tree (or dependency graph) during parsing. Then insert required expanding macro calls as needed. One possible method is to use a map [an element for each proc] of lists [listing all procs called within that proc]) and at the end, run through the lists and only insert the procs that can possibly be executed. The "inclusion loop" would start with the non-proc code, and flagging all procs called and recursively handle each called proc for other required procs.