Re: Purefiler - An alternative profiler
Posted: Tue Feb 01, 2011 5:11 am
New PB releases regularly break usefull binary PB tools/libs, people leave the community or lose interest up to the point that even the binary tools disappear with them. You can find several examples of this happening on the forums. I can fully relate to your reasons though and mostly do the same, up to each author anywayDidelphodon wrote:Hm, I don't know why everyone is always horny for one's source-code
It crashed much earlier, resulting in a 166MB trace data file which should be around 2.5mil traced invocations. But to get the results i need, would indeed require many gig's of trace information this way. So it simply wont work :/Didelphodon wrote:Indeed the loading can be very slow. In the dimensions of your software I guess we're not talking about 1000000 line-invocations anymore. I guess that the traces are more about Gigs of diskspace.
Yes, loading the sourcecode file results in over 12 minutes of initialisation and afterwards it crashes. It could simply be a matter of excessive memory use as the process quickly jumps to 1.85gig memory before crashing.Didelphodon wrote:Besides that I have the guess that it's actually the loading of the source-code which is implemented kinda poorly (shame on me).
Sadly i cant do that, but i gave purefiler a closer look and noticed that you also wrap direct assembler calls with Start/Stop calls. That results in trashed registers, trashed memory and crashes shortly after.Didelphodon wrote:It initially wasn't my intention to have this Profiler for 40k lines (!!) of code. Hm, don't know how I can help you this way, though I want, but I guess you wouldn't want to give me your sourcecode or your traces (which have the source-code included).
Example source to verify this problem:
Code: Select all
Procedure Windows_Get_Prozessor_String(StringAdresse)
!mov eax,80000000h ;Test, ob CPU aktuell genug ist um den String zu liefern
!cpuid
!cmp eax,80000004h
!jae @f ;alles o.K.
!xor eax,eax ;Rückgabewert = Null für eventuelle Auswertung Fehlschlag
ProcedureReturn
!@@:
!mov esi,[p.v_StringAdresse]
!xor edi,edi
!@@:
!mov eax,80000002h
!add eax,edi
!cpuid
!mov [esi],eax
!mov [esi+4],ebx
!mov [esi+8],ecx
!mov [esi+12],edx
!inc edi
!cmp edi,3
!je @f
!add esi,16
!jmp @b
!@@:
!mov eax,1 ;Rückgabewert <> Null für Erfolg
ProcedureReturn
EndProcedure
; set CPU info string
Temp$ = Space(100)
CompilerIf #PB_Compiler_OS = #PB_OS_Windows
If (Windows_Get_Prozessor_String(@Temp$) = 0)
Temp$ = ""
EndIf
CompilerEndIf
Debug "CPUID: " + Temp$I am using a lot other commercial profiler and tracing tools in my job, including my own one. The problem they all have is the massive amount of data which is created by these kind of traces (though profiling is only the start), because the CPU's are fast and the programs complex nowadays. A simple practical solution i use, is to limit the recording by triggers.Didelphodon wrote:@All: Anyone any more wishes? I'd really like to have a whole package done at once than doing many little patches.
As an example for the most simple universal case - start the software with a flag which tells it not to record. Make an external app which is able to toggle that flag, resulting in your profiler initialising the datafile (unique timestamp via elapsedmilliseconds() as you already do). That way you can profile those situations you actually need to have a look at, without having to change and recompile a dozen times and insert TRON/TROFF or similar commands iteratively till you found what you look for. Worked wonders for me - a very simple external trigger could be a temporary file which appears. Just check for it's existance once every second, if it exists, you delete the file, toggle the recording switch. I enhanced that further by appending the content of the first line of the file to the filename of the trace data file. That way you can start profiling individual use-cases of your software and append a little note about what specifically was profiled at that moment.
ps: i would suggest to replace the dynamic forward-linked list containing individually allocated records with a static array which holds (for example) 1-mil entries. It wont fragment your memory as much, it wont slow you down with malloc/free all the time and you can just write the records by using a few large binary buffer writes instead of each value of each record individually. Plus i would definatly shrink your records and merge the Start and Stop ones to save memory.
pps: on a funny sidenote, norton sonar thinks this example is a severe threat and removed it after compilation, before execution ...
