[Feature] - A Way to remove ununsed libraries

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
User avatar
OldSkoolGamer
Enthusiast
Enthusiast
Posts: 150
Joined: Mon Dec 15, 2008 11:15 pm
Location: Nashville, TN
Contact:

[Feature] - A Way to remove ununsed libraries

Post by OldSkoolGamer »

This would be a nice (probably not easy to implement) feature, the ability to remove unused libraries from your compiled executables somehow.

Now you include external libraries like so:

include "whatever.pbi"

What if there was a way to NOT include default libraries that are not being used in your exe from the default PB libraries, like so

Exclude #PB_HTTP_Library

something like that to keep the exe being larger for functions you are not using?

Or does PB already remove them at compile time?
Fred
Administrator
Administrator
Posts: 18162
Joined: Fri May 17, 2002 4:39 pm
Location: France
Contact:

Re: [Feature] - A Way to remove ununsed libraries

Post by Fred »

PB only includes the commands you use.
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: [Feature] - A Way to remove ununsed libraries

Post by GPI »

simple test:

Code: Select all

MessageRequester("test","test")
creates a 1kb exe (64bit)

Code: Select all

MessageRequester("test","test")
InitNetwork()
creates a 8kb exe.
User_Russian
Addict
Addict
Posts: 1518
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: [Feature] - A Way to remove ununsed libraries

Post by User_Russian »

Fred wrote:PB only includes the commands you use.
Unfortunately, in the executable file is added much extra code!
For example, this code requires gdiplus.dll (for what? :shock: :shock: ) and the program does not run on Windows 2000.

Code: Select all

OpenWindow(0, 0, 0, 400, 100, "", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
Repeat
  Event = WaitWindowEvent()  
Until Event = #PB_Event_CloseWindow
User avatar
bbanelli
Enthusiast
Enthusiast
Posts: 544
Joined: Tue May 28, 2013 10:51 pm
Location: Europe
Contact:

Re: [Feature] - A Way to remove ununsed libraries

Post by bbanelli »

User_Russian wrote:
Fred wrote:PB only includes the commands you use.
Unfortunately, in the executable file is added much extra code!
For example, this code requires gdiplus.dll (for what? :shock: :shock: ) and the program does not run on Windows 2000.
Well, this is true for PB 5.4, since 5.31 files ran on Windows NT4 (like this example).
"If you lie to the compiler, it will get its revenge."
Henry Spencer
https://www.pci-z.com/
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: [Feature] - A Way to remove ununsed libraries

Post by GPI »

User_Russian wrote:
Fred wrote:PB only includes the commands you use.
Unfortunately, in the executable file is added much extra code!
For example, this code requires gdiplus.dll (for what? :shock: :shock: ) and the program does not run on Windows 2000.

Code: Select all

OpenWindow(0, 0, 0, 400, 100, "", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
Repeat
  Event = WaitWindowEvent()  
Until Event = #PB_Event_CloseWindow
I think, when you use one command out of the window-library, the complete window-library is linked, not only parts of it. And when only one function need the gdipluss.dll, even when you don't use it, the gdipluss.dll is required.

btw: Win2000 is outdated, use it only on systems without internet-connections!
c4s
Addict
Addict
Posts: 1981
Joined: Thu Nov 01, 2007 5:37 pm
Location: Germany

Re: [Feature] - A Way to remove ununsed libraries

Post by c4s »

The problem is that PureBasic isn't very good at detecting which procedures are never executed, or let's say it could be heavily improved.

There is a buried (hint: with a "Like Post/Topic" Button great posts would be more prominent!) "hack" by Danilo called Dead Procedure Remover. I remember that it was able to remove about 30% of unused (thus dead) code and made my executables significantly smaller.
I wonder why this (or some equally effective) fix isn't an official part of PureBasic yet (after all the compiler is adding unused code for a long time now). Though I guess that it doesn't work anymore because it depends too much on the internals. Tomorrow I'll try to test if it still works with PB5.40...
If any of you native English speakers have any suggestions for the above text, please let me know (via PM). Thanks!
DontTalkToMe
Enthusiast
Enthusiast
Posts: 334
Joined: Mon Feb 04, 2013 5:28 pm

Re: [Feature] - A Way to remove ununsed libraries

Post by DontTalkToMe »

c4s wrote: I wonder why this (or some equally effective) fix isn't an official part of PureBasic yet (after all the compiler is adding unused code for a long time now). Though I guess that it doesn't work anymore because it depends too much on the internals. Tomorrow I'll try to test if it still works with PB5.40...
Considering PB has a switch to create a single source file with macros expanded and included files merged in should be feasible to process that instead of the asm source. The same thread has two PB sources doing that and both look at least as good starting points. Nevertheless this could be done easily and in a better way by the compiler itself and it should be part of its job anyway: generate the best code possible. Including dead code should be avoided if possible.
User avatar
skywalk
Addict
Addict
Posts: 4211
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: [Feature] - A Way to remove ununsed libraries

Post by skywalk »

As always, if there are stable workarounds I am satisfied. However, the last time I tried compiling a dll with modified asm using /reasm it failed with some of my settings; Threaded and Unicode + resource file.
I will try again with v54b6 and report.
Manually commenting dead procedures in my source code instead of blindly using XIncludeFile did save me 40% file size for the dll. Less than 10% and I would not care. But, I have several collections of libs(math,network,system,gui,etc.) and certainly not every app needs all procedures contained within each lib. I think this is a worthy topic to address in future releases.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
StarBootics
Addict
Addict
Posts: 1006
Joined: Sun Jul 07, 2013 11:35 am
Location: Canada

Re: [Feature] - A Way to remove ununsed libraries

Post by StarBootics »

Fred wrote:PB only includes the commands you use.
And what about unused custom procedure calling standard commands ?

Code: Select all

Procedure DummyProcedure()
  
  UseJPEG2000ImageDecoder()
  UseJPEG2000ImageEncoder()
  UseJPEGImageDecoder()
  UseJPEGImageEncoder()
  UsePNGImageDecoder()
  UsePNGImageEncoder()
  UseTGAImageDecoder()
  UseTIFFImageDecoder()
  
EndProcedure

MessageRequester("Testing", "DummyProcedure() test")
Compiling a source code like this, even if the DummyProcedure() is not being use you will end up with a compiled exe 838 ko and if you comment the entire DummyProcedure() the exe size drop to 35.4 ko.

I'm pretty sure the same thing apply for modules. In most cases I create module with many functionality and some program use all of them and other use only few of them. But in all cases unused procedure calling standard command are included regardless.

Maybe in future unused procedure will be discarded by the compiler when the exe is created.

Best regards
StarBootics
The Stone Age did not end due to a shortage of stones !
User avatar
Keya
Addict
Addict
Posts: 1890
Joined: Thu Jun 04, 2015 7:10 am

Re: [Feature] - A Way to remove ununsed libraries

Post by Keya »

I agree with User_Russian -- if it's not required, if it's not used, i'd prefer it not be included. :D
An asm-level code optimiser with garbage removal would be a great future addition.

In the case of his demo which simply creates an empty window gdiplus.dll is not used at all. Intermodular Calls shows there isn't a single call made into the DLL, but just to be sure in case of dynamics i set a breakpoint on every gdiplus.dll export, but none were triggered, not even during process termination/cleanup

the gdiplus.dll functions that are imported are: GdipDeleteFont, GdipDeleteGraphics, GdipDeleteMatrix, GdipDeletePath, GdipDeletePen, GdipDeleteStringFormat, GdipFree, GdipGetDpiX, GdipGetDpiY.
Ironically nearly all cleanup functions :D - so I can understand from that POV why they might be included, but there are no actual calls made to any of them.

ps. I tried Danilo's DeadProcedureRemover that c4s linked above, very handy addition to the Tools menu! :)
but as it does its work at PB-source-level (not fasm-source-level) it's not applicable to issues like this

Here are the extern's included in User_Russians empty window demo:

Code: Select all

extrn _PB_EndAlphaImage@0
extrn _PB_EndVectorDrawing@0
extrn _PB_Event_Free@0
extrn _PB_Event_Init@0
extrn _PB_FreeFonts@0
extrn _PB_FreeGadgets@0
extrn _PB_FreeImages@0
extrn _PB_FreeMemorys@0
extrn _PB_FreeWindows@0
extrn _PB_Init2DDrawing@0
extrn _PB_InitAlphaImage@0
extrn _PB_InitBMPImagePlugin@0
extrn _PB_InitFont@0
extrn _PB_InitGadget@0
extrn _PB_InitImage@0
extrn _PB_InitImageDecoder@0
extrn _PB_InitMap@0
extrn _PB_InitMemory@0
extrn _PB_InitVectorDrawing@0
extrn _PB_InitWindow@0
extrn _PB_OpenWindow2@28
extrn _PB_WaitWindowEvent@0
extrn _ExitProcess@4
extrn _GetModuleHandleA@4
extrn _HeapCreate@12
extrn _HeapDestroy@4
extrn _memset
extrn PB_StringBase
extrn _SYS_InitString@0
extrn _SYS_FreeStrings@0
extrn _PB_StringBasePosition
In this case if we take the unrequired VectorDrawing for example, the code (not shown here) shows that _PB_InitVectorDrawing and _PB_EndVectorDrawing are both CALL'ed (so in that sense the lib is required), but we then need to scan for all the other calls to functions within the vector lib to detect if we can remove the Init/End calls, which sounds tricky without a function name list for each lib.

But if we have such function name lists for each lib then it seems doable? reading the names part seems simple enough except for the part where it has parameter info of variable length, but i think ive figured it out and putting together some code now because well it just sounds fun loL :)
[edit] Done
Image
btw a case-insensitive unicode+ascii search for 'gdiplus' shows it only exists in two files:
\PureLibraries\VectorDrawing
\PureLibraries\Windows\Libraries\gdiplus.lib
User avatar
StarBootics
Addict
Addict
Posts: 1006
Joined: Sun Jul 07, 2013 11:35 am
Location: Canada

Re: [Feature] - A Way to remove ununsed libraries

Post by StarBootics »

Keya wrote:ps. I tried Danilo's DeadProcedureRemover that c4s linked above, very handy addition to the Tools menu! :)
It's a "Windows Only" solution out the box. Is this can be adapted to Linux maybe but no time for this in foreseeable future.

Best regards
StarBootics
The Stone Age did not end due to a shortage of stones !
freak
PureBasic Team
PureBasic Team
Posts: 5940
Joined: Fri Apr 25, 2003 5:21 pm
Location: Germany

Re: [Feature] - A Way to remove ununsed libraries

Post by freak »

User_Russian wrote:
Fred wrote:PB only includes the commands you use.
Unfortunately, in the executable file is added much extra code!
For example, this code requires gdiplus.dll (for what? :shock: :shock: ) and the program does not run on Windows 2000.

Code: Select all

OpenWindow(0, 0, 0, 400, 100, "", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
Repeat
  Event = WaitWindowEvent()  
Until Event = #PB_Event_CloseWindow
This is because of inter-library dependencies. The Window library depends on the VectorDrawing library due to the WindowVectorOutput() command. This dependency exists even if the command itself is not used. As a result, the VectorDrawing initialization function is linked and called (but no other functions from within the VectorDrawing library). As a result, gdiplus.dll is loaded.

Sure, this could be avoided by managing dependencies per function and not per library, but these dependencies are tricky enough for us to handle as it is already. E0xtending this to every function would add a lot of complexity with quite minor benefit.

Btw, Windows XP is now the minimum supported Windows version in PB. Windows 2000 is no longer relevant.
quidquid Latine dictum sit altum videtur
Fred
Administrator
Administrator
Posts: 18162
Joined: Fri May 17, 2002 4:39 pm
Location: France
Contact:

Re: [Feature] - A Way to remove ununsed libraries

Post by Fred »

BTW, if you really need to support Win2000, you can install these redistribuable for GDI+:

http://www.microsoft.com/en-us/download ... x?id=18909

But we don't support it anymore, so it can break anytime somewhere else.
User_Russian
Addict
Addict
Posts: 1518
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: [Feature] - A Way to remove ununsed libraries

Post by User_Russian »

It works on Windows 2000.
Thank you.
Post Reply