Another tricky thing - parsing a source text

Just starting out? Need help? Post your questions and find answers here.
User avatar
Michael Vogel
Addict
Addict
Posts: 2666
Joined: Thu Feb 09, 2006 11:27 pm
Contact:

Another tricky thing - parsing a source text

Post by Michael Vogel »

I just bought a new gadget, the Forerunner 935 from Garmin. I am using GPS watches to monitor my activities already for a long time (I started with the FR101), but this is the first time I feel totally happy with a product: the GPS accuracy is okay, the battery life is incredible (3 to 4 weeks!) and there's also a lot of funny stuff like notifications on board. And you can write programs for the tool, which seems to be relatively easy using 'monkey c'...

...so I am doing my first steps here and found out, that defining constants in monkey c leads to larger (and slower) programs, because no preprocessing is done by the compiler to replace the constants by values. So this is done every time when the program is executing this part of the code. Even equations like i=Const1 + Const2 + Const3 or codes like if (i==flag1 | flag2 | flag3) are not simplified, so I'd like to do that in the source code before finalizing a program to get smaller (and faster) executables.

I did a first step to find the constants of a source file (most of them are integers, so strings, real numbers or arrays are not needed), but now I need to detect the parts which could be simplified. It starts already in the first lines where the constants are defined: const BarGap=4; is followed by const BarHalfGap=BarGap/2;, so BarHalfGap should be set to 2.

Has anyone done a similar job and have some tipps or code lines for me?

A Monkey C example file (Birdie.mc):

Code: Select all

class BirdField extends Ui.DataField {

	const LabelText = "Birdie";
	const TitleFont = Gfx.FONT_MEDIUM;
	const LabelFont = Gfx.FONT_SMALL;
	
	const MILLISECONDS_TO_SECONDS = 0.001;

	const BarCount=5;
	const BarHeight=32;
	const BarDistance=1000;

	const BarMinutes=3;	
	const BarRange=BarMinutes*60;
	const BarTarget=300;			// 5 Minuten-Ziel
	const BarBottom=BarTarget-BarMinutes*30;	// -90 oder -120
	const BarTop=BarBottom+BarRange;
	const BarGap=4;
	const BarHalfGap=BarGap/2;
	const BarBold=3;
	
	const BarColBlk=360;
	const BarColBlu=330;
	const BarColRed=BarTarget;
	const BarColYel=285;
	const BarColGrn=BarBottom;

	const colnull=-1;
	
	hidden var mx,my,mtop;
	hidden var bx,by,bw,bh,bq;
	hidden var t=new[BarCount+1];

	hidden var numbers;
	hidden var bignumbers;
	hidden var obscurity;

	// hidden var debug;
	
	const mincode = [ "'", "¼", "½", "¾" ];	//	oder mit Minuten-Symbol: [ "'", "¼'", "½'", "¾'" ];
	
	const colors = [ 	0xFF00FF,	0xFF00AA,	0x55FF55,	0x00FF00,
						0xFFFF00,	0xFFAA00,	0xFF0000,	0xA00000,
						0x55AAFF,	0x5555FF,	0xAA00FF,	0xAA00AA	];
	var psec;
		var pmin;
		var prefix;

		if (pmin<10)
		{
		psec=(pace%60)/15;
		return prefix+pmin+mincode[psec.toNumber()];
		}
		
		dist=info.elapsedDistance;
		time=info.elapsedTime*MILLISECONDS_TO_SECONDS;
				
		if (dist>=distcount*BarDistance)
		{
			trend=time*BarDistance/dist;					// Pace für Trendanzeige festhalten
			for (i=1;i<BarCount;i+=1)
			{	t[i]=t[i+1];	
				distcount=(dist+BarDistance-1)/BarDistance;
				dc.setColor(fgColor,bgColor);
				dc.drawText(mx,mtop,LabelFont,LabelText,Gfx.TEXT_JUSTIFY_CENTER);	
				width=bw-BarGap;
				width=(dist%BarDistance);
				tick=t[barindex]*BarDistance/width;
				width=2+(bw-4)*width/BarDistance;
				dc.drawText(x+bw/2,y+ShiftFont,numbers,"1"+text+BarDistance,Gfx.TEXT_JUSTIFY_CENTER);	// XTINY
				dc.fillRectangle(x+BarHalfGap,y-bh-2,width,bh+1);
				dc.setColor(colors[(z/15).toNumber()],bgColor);
				z=z*bh/BarRange;
				dc.fillRoundedRectangle(x+BarHalfGap,y-z,width,z+1,2);
				dc.setColor(fgColor,bgColor);
				dc.drawRoundedRectangle(x+BarHalfGap,y-z,width,z+1,2);
				y=by-(tick-BarBottom)*bh/BarRange;
			}		

		bx=(sx-bw*BarCount)/2-BarRange;
		bq=(sy-150)/BarMinutes/2;		// Diagrammhöhe
		bq=(sy-65)/BarMinutes/2;
		bh=(sy-35)/BarMinutes/2;
		bh=bq*BarMinutes*2;
	}
}


My code to detect constants:

Code: Select all

; Define

	Global total
	Global const

	Structure consttype
		name.s
		type.i
		value.i
		arrtext.s
	EndStructure

	Structure sourcetype
		line.s
		len.i
		const.i
	EndStructure

	Global Dim s.sourcetype(0)
	Global Dim c.consttype(0)

	Enumeration
		#ConstArray
		#ConstValue
	EndEnumeration

	Enumeration
		#SearchPosStart
		#SearchPosEnd
		#SearchPosAll
	EndEnumeration

	Global Search_Start
	Global Search_Len
	Global Search_Next
	Global Search_String.s

; EndDefine
Procedure.s MyTrim(s.s)

	Protected i
	Protected l=Len(s)*SizeOf(Character)
	Protected b

	While l
		l-SizeOf(Character)
		b=PeekA(@s+l)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
	Wend

	While i<l
		b=PeekA(@s+i)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
		i+SizeOf(Character)
	Wend

	ProcedureReturn PeekS(@s+i,l-i+1)

EndProcedure
Procedure Search(string.s,find.s,mode=#SearchPosStart)

	If CreateRegularExpression(0,find)
		If ExamineRegularExpression(0,string)
			If NextRegularExpressionMatch(0)
				Select mode
				Case #SearchPosStart
					ProcedureReturn RegularExpressionMatchPosition(0)
				Case #SearchPosEnd
					ProcedureReturn RegularExpressionMatchPosition(0)+RegularExpressionMatchLength(0)
				Case #SearchPosAll
					Search_Start=RegularExpressionMatchPosition(0)
					Search_Len=RegularExpressionMatchLength(0)
					Search_Next=Search_Start+Search_Len
					Search_String=RegularExpressionMatchString(0)
					ProcedureReturn Search_Start
				EndSelect
			EndIf
		Else
			ProcedureReturn #Null;	String not found
		EndIf
	Else
		Debug "error"
		ProcedureReturn #Null;	Find expression error
	EndIf

EndProcedure
Procedure Expression(s.s)

	s=MyTrim(s)

	ProcedureReturn Val(s)

EndProcedure
Procedure ReadSource()

	Protected s.s,t.s
	Protected c,v

	total=0

	If ReadFile(0,"Birdie.mc")

		While Eof(0)=0
			ReDim s(total)
			s=ReadString(0)

			With s(total)
				\line=s

				; Kommentare ignorieren
				If Search(s,".*//",#SearchPosAll)
					\len=Search_Next-2
					s=Left(s,\len)
				Else
					\len=Len(s)
				EndIf

				; Konstante suchen
				If Search(s,"[ \t]*const[ \t]*",#SearchPosAll)
					s=Mid(s,Search_Next)
					If Search(s,"[ \t]*[a-z,A-Z][a-z,A-Z,0-9]*[ \t]*",#SearchPosAll)
						If Mid(s,Search_Next,1)="="
							\const=#True
							ReDim c(const)
							c(const)\name=MyTrim(Search_String)
							s=MyTrim(Mid(s,Search_Next+1))
							If Left(s,1)="["
								c(const)\type=#ConstArray
								c(const)\arrtext=s
								Debug c(const)\name+" > "+s
							Else
								c(const)\type=#ConstValue
								c(const)\value=Expression(s)
								Debug c(const)\name+" < "+Str(c(const)\value)
							EndIf

							const+1
							;Debug t+" : "+s
						EndIf
					EndIf
				EndIf

			EndWith

			total+1
		Wend

	EndIf

	ProcedureReturn total

EndProcedure
If ReadSource()
EndIf
User avatar
Michael Vogel
Addict
Addict
Posts: 2666
Joined: Thu Feb 09, 2006 11:27 pm
Contact:

Re: Another tricky thing - parsing a source text

Post by Michael Vogel »

Still working on the first part to get all integer values for the integer constants, but this seems to be nearly done now - I've only need to replace Val by a fine Eval function...

Code: Select all

; Define

	Global total
	Global const

	Structure consttype
		name.s
		type.i
		value.i
		arrtext.s
	EndStructure

	Structure sourcetype
		line.s
		len.i
		const.i
	EndStructure

	Global Dim s.sourcetype(0)
	Global Dim c.consttype(0)

	Enumeration
		#ConstError
		#ConstArray
		#ConstValue
	EndEnumeration

	Enumeration
		#ExpressionSemicolon
		#ExpressionAll
	EndEnumeration

	Enumeration
		#SearchPosStart
		#SearchPosEnd
		#SearchPosAll
	EndEnumeration

	Global Search_Start
	Global Search_Len
	Global Search_Next
	Global Search_Result
	Global Search_String.s
	Global Expression_String.s
	Global Expression_Val
	Global Replace_String.s

; EndDefine
Procedure.s MyTrim(s.s)

	Protected i
	Protected l=Len(s)*SizeOf(Character)
	Protected b

	While l
		l-SizeOf(Character)
		b=PeekA(@s+l)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
	Wend

	While i<l
		b=PeekA(@s+i)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
		i+SizeOf(Character)
	Wend

	ProcedureReturn PeekS(@s+i,l-i+1)

EndProcedure
Procedure Search(string.s,find.s,mode=#SearchPosStart)

	If CreateRegularExpression(0,find)
		If ExamineRegularExpression(0,string)
			If NextRegularExpressionMatch(0)
				Select mode
				Case #SearchPosStart
					ProcedureReturn RegularExpressionMatchPosition(0)
				Case #SearchPosEnd
					ProcedureReturn RegularExpressionMatchPosition(0)+RegularExpressionMatchLength(0)
				Case #SearchPosAll
					Search_Start=RegularExpressionMatchPosition(0)
					Search_Len=RegularExpressionMatchLength(0)
					Search_Next=Search_Start+Search_Len
					Search_String=RegularExpressionMatchString(0)
					Search_Result=Bool(Search_Len=Len(string))
					ProcedureReturn Search_Start
				EndSelect
			EndIf
		Else
			ProcedureReturn #Null;	String not found
		EndIf
	Else
		Debug "error"
		ProcedureReturn #Null;	Find expression error
	EndIf

EndProcedure
Procedure.s ReplaceConstanst(s.s)

	Protected i,n

	i=const
	While i
		i-1
		If c(i)\type=#ConstValue
			s=ReplaceString(s,c(i)\name,Str(c(i)\value))
		EndIf
	Wend

	Replace_String=s
	ProcedureReturn s

EndProcedure
Procedure Expression(s.s,mode=#ExpressionSemicolon)

	Protected n
	Protected t.s

	If mode=#ExpressionSemicolon
		If Search(s,".*;",#SearchPosAll)
			Expression_String=Left(s,Search_Len-1)
			s=ReplaceConstanst(Expression_String)
			; Integer Values
			If Search(s,"[0-9]+",#SearchPosAll) And Search_Result
				Expression_Val=Val(s)
				ProcedureReturn #True
			EndIf
			; Hex Values
			If Search(s,"0x[0-9]+",#SearchPosAll) And Search_Result
				Expression_Val=Val("$"+Mid(s,3))
				ProcedureReturn #True
			EndIf
		EndIf
	EndIf

	ProcedureReturn 0

EndProcedure
Procedure ReadSource()

	Protected s.s,t.s
	Protected c,v

	total=0

	If ReadFile(0,"C:\Program Files\System\Connect IQ\Samples\Datafield\Birdie\source\Birdie.mc")

		While Eof(0)=0
			ReDim s(total)
			s=ReadString(0)

			With s(total)
				\line=s

				; Kommentare ignorieren
				If Search(s,".*//",#SearchPosAll)
					\len=Search_Next-2
					s=Left(s,\len)
				Else
					\len=Len(s)
				EndIf

				; Konstante suchen
				If Search(s,"[ \t]*const[ \t]*",#SearchPosAll)
					s=Mid(s,Search_Next)
					If Search(s,"[ \t]*[a-z,A-Z][a-z,A-Z,0-9]*[ \t]*",#SearchPosAll)
						If Mid(s,Search_Next,1)="="
							\const=#True
							ReDim c(const)
							c(const)\name=MyTrim(Search_String)
							s=MyTrim(Mid(s,Search_Next+1))
							If Left(s,1)="["
								c(const)\type=#ConstArray
								c(const)\arrtext=s
								Debug c(const)\name+" > "+s
							Else
								If Expression(s)
									c(const)\type=#ConstValue
									c(const)\value=Expression_Val
									Debug c(const)\name+" < "+Str(c(const)\value)+" < '"+Replace_String+"' < '"+Expression_String+"'"
								Else
									c(const)\type=#ConstError
									c(const)\value=0
									Debug c(const)\name+" < Error < '"+Replace_String+"' < '"+Expression_String+"'"
								EndIf

							EndIf

							const+1
							;Debug t+" : "+s
						EndIf
					EndIf
				EndIf

			EndWith

			total+1
		Wend

	EndIf

	ProcedureReturn total

EndProcedure
If ReadSource()
EndIf
...but then I need to replace the constants within regular code, this could me maybe a little bit trickier :wink:
User avatar
Michael Vogel
Addict
Addict
Posts: 2666
Joined: Thu Feb 09, 2006 11:27 pm
Contact:

Re: Another tricky thing - parsing a source text

Post by Michael Vogel »

A new problem for regex experts :)

I need to get all variable names from a source code, which is not critical if they are defined by a simple var xxx; or var yyy=1; but there are more variants allowed, like var a = 2, b , c =a * b ; which could span multiple lines as well. Here I would need to detect the variables 'a', 'b' and 'c' !

You can see my actual code below which tries to detect the variable definitions (and classes, functions and constants). The target for this program is to optimize the placement of variables (and constants) as they (both) do perform 10 times quicker when used locally compared to global and 3 times quicker when they are defined in the same class.
So after making a list of variables and their assignment to certain classes and functions, I need to check all occurences of variables and constants in the source to see if they are used within certain classes/functions only.

Code: Select all

#ShowCode=		0
#ShowClasses=		0
#ShowFunctions=	0
#ShowVariables=		1
#ShowConstants=	0

; Define

	#Q=#DOUBLEQUOTE$

	#Names=	"[a-z,A-Z,_][a-z,A-Z,0-9,_]*"
	#Spaces=	"[ \t]*"
	#Delimiter=	"[ =;\,\t\[]*"

	Global total
	Global const

	Structure consttype
		name.s
		type.i
		value.i
		arrtext.s
	EndStructure

	Structure sourcetype
		line.s
		len.i
		const.i
	EndStructure

	Global Dim s.sourcetype(0)
	Global Dim c.consttype(0)

	Enumeration
		#ConstError
		#ConstArray
		#ConstValue
	EndEnumeration

	Enumeration
		#ExpressionSemicolon
		#ExpressionAll
	EndEnumeration

	Enumeration
		#SearchGetStart
		#SearchGetEnd
		#SearchGetAll
	EndEnumeration

	Global Search_Start
	Global Search_Len
	Global Search_Next
	Global Search_Result
	Global Search_String.s
	Global Expression_String.s
	Global Expression_Val
	Global Replace_String.s

	DataSection


		Data.s "using Toybox.WatchUi as Ui;"
		Data.s "using Toybox.Application as App;"
		Data.s "using Toybox.Graphics as Gfx;"
		Data.s "using Toybox.System as Sys;"
		Data.s "using Toybox.Time as Time;"
		Data.s "using Toybox.Activity as Act;"
		Data.s ""
		Data.s ""
		Data.s "class BirdField extends Ui.DataField {"
		Data.s ""
		Data.s "	const ShiftBottomline = 6;						// Pixelversatz - const doesnotexist;"
		Data.s "	const MILLISECONDS_TO_SECONDS = 0.001;"
		Data.s ""
		Data.s "	const BarCount=6;"
		Data.s "	const BarHeight=32;"
		Data.s "	const BarDistance=1000;"
		Data.s ""
		Data.s "	const BarMinutes=3;							// var nothere=3;"
		Data.s "	const BarRange=BarMinutes*60;"
		Data.s "	const BarTarget=300;						// 5"
		Data.s "	const BarBottom=BarTarget-BarMinutes*30;	// -"
		Data.s "	const BarTop=BarBottom+BarRange;"
		Data.s "	const BarGap=4;"
		Data.s "	const BarHalfGap=BarGap/2;"
		Data.s "	const BarBold=3;"
		Data.s "	"
		Data.s "	const BarColBlk=360;"
		Data.s "	const BarColBlu=330;"
		Data.s "	const BarColRed=BarTarget;"
		Data.s "	const BarColYel=285;"
		Data.s "	const BarColGrn=BarBottom;"
		Data.s ""
		Data.s "	const colnull=-1;"
		Data.s ""
		Data.s "var test1, test2,"
		Data.s "test3"
		Data.s "test4;"
		Data.s ""
		Data.s "var variante1 , variante2 ,"
		Data.s "variante3 ;"
		Data.s ""
		Data.s "	hidden var timermode;"
		Data.s "	hidden var time;"
		Data.s "	hidden var dist;"
		Data.s "	hidden var laptime;"
		Data.s "	hidden var trend;"
		Data.s "	hidden var distcount;"
		Data.s "	hidden var barindex;"
		Data.s "	hidden var ShiftHeadline;"
		Data.s ""
		Data.s "	hidden var sx,sy;"
		Data.s "	hidden var mx,my,mtop;"
		Data.s "	hidden var bx,by,bw,bh,bq;"
		Data.s "	hidden var t=new[BarCount+1];"
		Data.s ""
		Data.s "	hidden var numbers;"
		Data.s "	hidden var bignumbers;"
		Data.s "	hidden var obscurity;"
		Data.s ""
		Data.s "	// hidden var debug;"
		Data.s "	"
		Data.s "	// const mincode = [ ~'~, ~¼~, ~½~, ~¾~ ];	// tricky */ const whatthe=666;"
		Data.s "	"
		Data.s "	const colors = [ 	0xFF00FF,	0xFF00AA,	0x55FF55,	0x00FF00,"
		Data.s "						0xFFFF00,	0xFFAA00,	0xFF0000,	0xA00000,"
		Data.s "						0x55AAFF,	0x5555FF,	0xAA00FF,	0xAA00AA	];"
		Data.s ""
		Data.s ""
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	function updateTimerMode()"
		Data.s "	{"
		Data.s "		timermode=Act.getActivityInfo().timerState;"
		Data.s "	}"
		Data.s "	"
		Data.s "	"
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	function initialize()"
		Data.s "	{"
		Data.s "		// Sys.println(~[Initialize] ~+time);"
		Data.s ""
		Data.s "		var i;"
		Data.s ""
		Data.s "		time=0;			// but not notime"
		Data.s "		dist=0;			// also not nodist"
		Data.s ""
		Data.s "		trend=0;		// "
		Data.s "		laptime=0;		// "
		Data.s "		distcount=1;	// "
		Data.s "		barindex=1;		// "
		Data.s "		"
		Data.s "		for (i=0; i<=BarCount; i+=1)"
		Data.s "		{	t[i]=0;	}"
		Data.s "		"
		Data.s "	}"
		Data.s ""
		Data.s ""
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	function pacestr(pace,mode)"
		Data.s "	{"
		Data.s "		var psec;"
		Data.s "		var pmin;"
		Data.s "		var prefix;"
		Data.s ""
		Data.s "		{"
		Data.s "			pace=BarTarget-pace;"
		Data.s "			if (pace<0)"
		Data.s "			{"
		Data.s "				pace=-pace;"
		Data.s "				prefix=~-~;"
		Data.s "			}"
		Data.s "			else if (pace>0)"
		Data.s "			{	prefix=~+~;	}"
		Data.s "			else"
		Data.s "			{	return ~0~;	}"
		Data.s "			"
		Data.s "			if (pace<60)"
		Data.s "			{"
		Data.s "			}"
		Data.s "		}"
		Data.s "		"
		Data.s "	}"
		Data.s ""
		Data.s ""
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "    function compute(info)"
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	{"
		Data.s "		// Sys.println(~[Compute]~);"
		Data.s ""
		Data.s "		var i;"
		Data.s "		"
		Data.s "		t[barindex]=time-laptime;"
		Data.s ""
		Data.s "		if (dist>=distcount*BarDistance)"
		Data.s "		{"
		Data.s "			trend=time*BarDistance/dist;					// remark"
		Data.s "			if (barindex<BarCount)"
		Data.s "			{	barindex=barindex+1;	}"
		Data.s "		"
		Data.s "			laptime=time;"
		Data.s "			t[barindex]=0;"
		Data.s "		}"
		Data.s "    }"
		Data.s ""
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	function redraw(dc)"
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	{"
		Data.s "		// Sys.println(~[Redraw]~);"
		Data.s ""
		Data.s "		var tick=0;"
		Data.s "		var width;"
		Data.s "		var text;"
		Data.s "		var i,c,o,x,y,z;"
		Data.s "		"
		Data.s "		var bgColor=getBackgroundColor();"
		Data.s "		var fgColor;"
		Data.s ""
		Data.s "        if (bgColor==Gfx.COLOR_WHITE)"
		Data.s "		{	"
		Data.s "			fgColor=Gfx.COLOR_BLACK; 	// internal const"
		Data.s "			c=0xaaffaa;					// local var "
		Data.s "		}"
		Data.s "		else"
		Data.s "		{"
		Data.s "			fgColor=Gfx.COLOR_WHITE; 	// blah  "
		Data.s "			c=0x000055;					// blah "
		Data.s "		}"
		Data.s "		"
		Data.s ""
		Data.s "         dc.setColor(fgColor,bgColor);"
		Data.s "         dc.clear();"
		Data.s ""
		Data.s "    "
		Data.s "		if (time==0 && timermode!=Act.TIMER_STATE_ON)"
		Data.s "		{"
		Data.s "			// Werte zurücksetzen //"
		Data.s "			trend=0;		"
		Data.s "			laptime=0;		"
		Data.s "			distcount=1;"
		Data.s "			$.barindex=1;	// global variables may be referenced by $"
		Data.s "			"
		Data.s "			dc.setColor(fgColor,bgColor);"
		Data.s "			dc.drawText(mx,mtop,Gfx.FONT_SMALL,~Birdie~,Gfx.TEXT_JUSTIFY_CENTER);	"
		Data.s "			x=z.sec%5;"
		Data.s "			if (x)	{	dc.setColor(0xFF0000,bgColor);	}"
		Data.s "			else	{	dc.setColor(0xFF5555,bgColor);	}"
		Data.s "			z=x/2; "
		Data.s "			}"
		Data.s ""
		Data.s "		}"
		Data.s ""
		Data.s "		else"
		Data.s ""
		Data.s "		{"
		Data.s "			// aktuelle Kilometer als Titelzeile //"
		Data.s "			dc.setColor(fgColor,bgColor);"
		Data.s "			o=dist/1000+~.~;									// 12."
		Data.s "			z=((dist/10)%100).format(~%02d~);					// ³°"
		Data.s "			x=dc.getTextWidthInPixels(o,Gfx.FONT_LARGE);		// "
		Data.s "			y=dc.getTextWidthInPixels(z,Gfx.FONT_MEDIUM);		// "
		Data.s "			i=mx-(x+y+37)/2;									// "
		Data.s "			dc.drawText(i,20,numbers,dc.getTextWidthInPixels(~0km~,Gfx.FONT_TINY),Gfx.TEXT_JUSTIFY_LEFT);		// "
		Data.s ""
		Data.s "			y=by;"
		Data.s "			x=bx+bw-2;"
		Data.s "			for	(i=BarBottom+30;i<=BarTop;i+=30)"
		Data.s "			{"
		Data.s "			y-=bq;"
		Data.s "			if (i%60==0)"
		Data.s "			dc.drawLine(x,y,sx-x,y);"
		Data.s "			}"
		Data.s ""
		Data.s ""
		Data.s "			x=bx;"
		Data.s "			dc.setColor(fgColor, bgColor);"
		Data.s "			for (i=1; i<=BarCount and i<=barindex; i+=1)"
		Data.s "			{"
		Data.s "				text=i+distcount-barindex;"
		Data.s "				width=bw-BarGap;"
		Data.s "				if (i<barindex)"
		Data.s "				{	width=(dist%BarDistance);"
		Data.s "					width=2+(bw-4)*width/BarDistance;"
		Data.s "				}"
		Data.s "			"
		Data.s "				x+=bw;"
		Data.s "				if (text<0)"
		Data.s "				{"
		Data.s "					width+=BarBold;"
		Data.s "					dc.drawText(x+bw/2,y+ShiftBottomline,numbers,~1~+text,Gfx.TEXT_JUSTIFY_CENTER);	// XTINY"
		Data.s "				}"
		Data.s "				else"
		Data.s ""
		Data.s "				if (tick==0)"
		Data.s "				{"
		Data.s "					z=z*bh/BarRange;"
		Data.s "					dc.fillRoundedRectangle(x+BarHalfGap,y-z,width,z+1,2);"
		Data.s "					dc.drawText(x+bw/2,y-bh-ShiftHeadline,bignumbers,pacestr(tick,0),Gfx.TEXT_JUSTIFY_CENTER);"
		Data.s "				}"
		Data.s ""
		Data.s "				if (text>0)"
		Data.s "				{	x-=BarBold;		}"
		Data.s "			}"
		Data.s "			"
		Data.s "			for (i=BarCount;i>=0;i-=1)"
		Data.s "			{	dc.drawLine(z,by,z,by+3);"
		Data.s "				if (i==1 && barindex<distcount) { z-=4; }	"
		Data.s "			}"
		Data.s ""
		Data.s ""
		Data.s "			c=tick;										"
		Data.s "			tick=time*1000/dist;							"
		Data.s "			if (sy>200)"
		Data.s "			{	dc.drawText(mx,mtop,Gfx.FONT_LARGE,pacestr(tick,2),Gfx.TEXT_JUSTIFY_CENTER);"
		Data.s "				if (trend)"
		Data.s "				{	c-=trend;				"
		Data.s ""
		Data.s "		"
		Data.s "					dc.drawText(mx,sy-20,bignumbers,o,Gfx.TEXT_JUSTIFY_CENTER);"
		Data.s "				}"
		Data.s "			}"
		Data.s "			if (tick>=BarBottom and tick<=BarTop)"
		Data.s "			{	if (tick>BarTarget)"
		Data.s "				{	z=0xAA0000;		}	// gfx_COLOR_DK_RED;"
		Data.s "				else"
		Data.s "				{	z=0x00AA00;		}	// gfx_COLOR_DK_GREEN;"
		Data.s "				x=bx+bw-5;"
		Data.s "				y=by-(tick-BarBottom)*bh/BarRange;"
		Data.s "				dc.setColor(z,bgColor);"
		Data.s "				dc.drawLine(x,y,sx-x,y);"
		Data.s "			}"
		Data.s "			else if (sy>200)"
		Data.s "			{	dc.drawText(mx,mtop+8,bignumbers,~X~,Gfx.TEXT_JUSTIFY_CENTER);"
		Data.s "			}"
		Data.s "							"
		Data.s "		}"
		Data.s ""
		Data.s "    }"
		Data.s "	"
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	function onLayout(dc)"
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s "	{"
		Data.s "		sx=dc.getWidth();"
		Data.s "		sy=dc.getHeight();"
		Data.s "		mx=sx/2;"
		Data.s "		my=sy/2;"
		Data.s ""
		Data.s "		obscurity=DataField.getObscurityFlags();"
		Data.s "		"
		Data.s "		bw=sx/(BarCount+1)-1;				// "
		Data.s "		bx=(sx-bw*BarCount)/2-bw;			// "
		Data.s "		"
		Data.s "		if (sy>180)"
		Data.s "		{"
		Data.s "			mtop=5;							// "
		Data.s "			bq=(sy-150)/BarMinutes/2;		// "
		Data.s "			by=sy-74;						// "
		Data.s "		}"
		Data.s "		else if (sy>100)"
		Data.s "		{"
		Data.s "			mtop=2;"
		Data.s "			bq=(sy-65)/BarMinutes/2;"
		Data.s "			if (sx>120)"
		Data.s "			{	by=sy-22;	}"
		Data.s "			else"
		Data.s "			{	by=sy-35;	}"
		Data.s "		}"
		Data.s "		else"
		Data.s "		{"
		Data.s "			mtop=0;"
		Data.s "			bh=(sy-35)/BarMinutes/2;"
		Data.s "			by=sy-22;"
		Data.s "		}"
		Data.s "		bh=bq*BarMinutes*2;"
		Data.s ""
		Data.s "		ShiftHeadline=dc.getFontHeight(5678+bignumbers+1234);"
		Data.s ""
		Data.s "    }"
		Data.s ""
		Data.s "	// / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / //"
		Data.s ""
		Data.s "    function onUpdate(dc)"
		Data.s "	{"
		Data.s "		redraw(dc);"
		Data.s "	}	"
		Data.s ""
		Data.s "}"
		Data.s ""
		Data.s "class Birdie extends App.AppBase"
		Data.s "{"
		Data.s ""
		Data.s "    function getInitialView()"
		Data.s "	{	return [new BirdField()];	}"
		Data.s ""
		Data.s "    function initialize()"
		Data.s "	{	AppBase.initialize();		}"
		Data.s ""
		Data.s "    function onStart(state)"
		Data.s "	{								}"
		Data.s ""
		Data.s "    function onStop(state)"
		Data.s "	{	return false;				}"
		Data.s "}"
		Data.s ""
		Data.s "/*"
		Data.s "class MyDelegate extends Ui.BehaviorDelegate"
		Data.s "{"
		Data.s "	var menu;"
		Data.s "	"
		Data.s "	function initialize(view)"
		Data.s "	{	Sys.println(~* MyDelegate Init (~+menu+~)~);"
		Data.s "		BehaviorDelegate.initialize();"
		Data.s "	}"
		Data.s ""
		Data.s "}"
		Data.s "*/"
		Data.s "EOF"

	EndDataSection
; EndDefine

Procedure.l Eval(Input.s)

	Protected k
	Protected last_op,last_op2,lastchrnum
	Protected P2_Start_Op,P2_Result,Result
	Protected number.s
	Protected *pInput.byte,*r.byte
	Protected Dim Priority.s(20)

	k=0
	*pInput=@Input

	Repeat
		If *pInput\b='('
			k+1

		ElseIf *pInput\b=')'
			*r=@Priority(k)
			last_op=0
			number=""
			lastchrnum=1
			P2_Start_Op=-1
			Repeat
				If *r\b >= $30 And *r\b <= $39
					number+Chr(*r\b)
					lastchrnum=1

				Else ;*r\b ist keine ASCII-Ziffer


					last_op2=last_op
					If lastchrnum
						If *r\b='+'
							If last_op > 2
								If last_op=3
									P2_Result*Val(number)
								ElseIf last_op=4
									P2_Result/Val(number)
								EndIf
								If P2_Start_Op=0
									Result=P2_Result
								ElseIf P2_Start_Op=1
									Result + P2_Result
								ElseIf P2_Start_Op=2
									Result - P2_Result
								EndIf
								P2_Start_Op=-1
							EndIf

							last_op=1
						ElseIf *r\b='-'
							If last_op > 2
								If last_op=3
									P2_Result * Val(number)
								ElseIf last_op=4
									P2_Result / Val(number)
								EndIf
								If P2_Start_Op=0
									Result=P2_Result
								ElseIf P2_Start_Op=1
									Result + P2_Result
								ElseIf P2_Start_Op=2
									Result - P2_Result
								EndIf
								P2_Start_Op=-1
							EndIf

							last_op=2
						ElseIf *r\b='*'
							If last_op < 3
								P2_Start_Op=last_op
								P2_Result=Val(number)
							EndIf
							last_op=3
						ElseIf *r\b='/'
							If last_op < 3
								P2_Start_Op=last_op
								P2_Result=Val(number)
							EndIf
							last_op=4
						EndIf

						If last_op2=0 And last_op < 3
							Result=Val(number)
						ElseIf last_op2=1 And last_op < 3
							Result + Val(number)
						ElseIf last_op2=2 And last_op < 3
							Result - Val(number)
						ElseIf last_op2=3
							P2_Result * Val(number)
						ElseIf last_op2=4
							P2_Result / Val(number)
						EndIf

						number=""
					ElseIf *r\b='-'
						If number=""
							number="-"
						Else
							number=""
						EndIf
					EndIf

					lastchrnum=0
				EndIf

				*r+*r+SizeOf(Character)
			Until *r\b=0

			If last_op=0
				Result=Val(number)
			ElseIf last_op=1
				Result + Val(number)
			ElseIf last_op=2
				Result - Val(number)
			ElseIf last_op=3
				P2_Result * Val(number)
			ElseIf last_op=4
				P2_Result / Val(number)
			EndIf

			If P2_Start_Op=0
				Result=P2_Result
			ElseIf P2_Start_Op=1
				Result + P2_Result
			ElseIf P2_Start_Op=2
				Result - P2_Result
			EndIf

			k-1
			If k < 0
				ProcedureReturn 0
			EndIf
			Priority(k)+Str(Result)

		ElseIf *pInput\b=0 ; ---ENDE vom String---
			*r.BYTE=@Priority(k)
			last_op=0
			number=""
			lastchrnum=1
			P2_Start_Op=-1

			Repeat
				If *r\b >= $30 And *r\b <= $39
					number + Chr(*r\b)
					lastchrnum=1

				Else ;*r\b ist keine ASCII-Ziffer
					last_op2=last_op
					If lastchrnum

						If *r\b='+'
							If last_op > 2
								If last_op=3
									P2_Result * Val(number)
								ElseIf last_op=4
									P2_Result / Val(number)
								EndIf
								If P2_Start_Op=0
									Result=P2_Result
								ElseIf P2_Start_Op=1
									Result + P2_Result
								ElseIf P2_Start_Op=2
									Result - P2_Result
								EndIf
								P2_Start_Op=-1
							EndIf
							last_op=1

						ElseIf *r\b='-'
							If last_op > 2
								If last_op=3
									P2_Result * Val(number)
								ElseIf last_op=4
									P2_Result / Val(number)
								EndIf
								If P2_Start_Op=0
									Result=P2_Result
								ElseIf P2_Start_Op=1
									Result + P2_Result
								ElseIf P2_Start_Op=2
									Result - P2_Result
								EndIf
								P2_Start_Op=-1
							EndIf
							last_op=2

						ElseIf *r\b='*'
							If last_op < 3
								P2_Start_Op=last_op
								P2_Result=Val(number)
							EndIf
							last_op=3

						ElseIf *r\b='/'
							If last_op < 3
								P2_Start_Op=last_op
								P2_Result=Val(number)
							EndIf
							last_op=4

						EndIf

						If last_op2=0 And last_op < 3
							Result=Val(number)
						ElseIf last_op2=1 And last_op < 3
							Result + Val(number)
						ElseIf last_op2=2 And last_op < 3
							Result - Val(number)
						ElseIf last_op2=3
							P2_Result * Val(number)
						ElseIf last_op2=4
							P2_Result / Val(number)
						EndIf

						number=""
					ElseIf *r\b='-'
						If number=""
							number="-"
						Else
							number=""
						EndIf
					EndIf
					lastchrnum=0

				EndIf
				*r+SizeOf(Character)
			Until *r\b=0

			If last_op=0
				Result=Val(number)
			ElseIf last_op=1
				Result + Val(number)
			ElseIf last_op=2
				Result - Val(number)
			ElseIf last_op=3
				P2_Result * Val(number)
			ElseIf last_op=4
				P2_Result / Val(number)
			EndIf

			If P2_Start_Op=0
				Result=P2_Result
			ElseIf P2_Start_Op=1
				Result + P2_Result
			ElseIf P2_Start_Op=2
				Result - P2_Result
			EndIf

			ProcedureReturn Result

		Else
			Priority(k) + Chr(*pInput\b)

		EndIf

		*pInput+SizeOf(Character)
	ForEver

EndProcedure
Procedure.s MyTrim(s.s)

	#CharacterBits=SizeOf(Character)>>1

	Protected i
	Protected l=(Len(s)-1)<<#CharacterBits
	Protected b

	While i<l
		b=PeekA(@s+i)
		;Debug Str(i)+": "+Str(b)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
		i+SizeOf(Character)
	Wend

	While l>=i
		b=PeekA(@s+l)
		;Debug Str(l)+": "+Str(b)
		If (b<>' ') And (b<>#TAB)
			Break
		EndIf
		l-SizeOf(Character)
	Wend

	If l>=0
		;Debug Str(i)+", "+Str(l)+" - '"+s+"' = '"+PeekS(@s+i,l-i)+"'"
		ProcedureReturn PeekS(@s+i,(l-i)>>#CharacterBits+1)
	Else
		;Debug "*"+Str(i)+", "+Str(l)+" - '"+s+"'"
		ProcedureReturn ""
	EndIf

EndProcedure
Procedure.s Clean(s.s)

	Protected q1,q2

	q1=#True
	While q1
		; Debug "..."+q1+": "+Mid(s,q1)
		q1=FindString(s,#Q,q1)
		If q1
			q2=q1+1
			Repeat
				q2=FindString(s,#Q,q2)
				; Debug ":"+q1+", "+q2+": "+Mid(s,q1)
				If q2
					If Mid(s,q2-1,1)="\" And Mid(s,q2-2,1)<>"\";	behandelt '\"' und '\\"', nicht jedoch '\\\"' !
						q2+1
					Else
						s=Left(s,q1)+Mid(s,q2)
						q1+2
						q2=#False
					EndIf
				Else
					q1=#False
				EndIf
			Until q2=#Null
		EndIf
		; Debug "..."+q1
	Wend

	ProcedureReturn s

EndProcedure
Procedure Search(string.s,find.s,mode=#SearchGetStart)

	If CreateRegularExpression(0,find)
		If ExamineRegularExpression(0,string)
			If NextRegularExpressionMatch(0)
				Select mode
				Case #SearchGetStart
					ProcedureReturn RegularExpressionMatchPosition(0)
				Case #SearchGetEnd
					ProcedureReturn RegularExpressionMatchPosition(0)+RegularExpressionMatchLength(0)
				Case #SearchGetAll
					Search_Start=RegularExpressionMatchPosition(0)
					Search_Len=RegularExpressionMatchLength(0)
					Search_Next=Search_Start+Search_Len
					Search_String=RegularExpressionMatchString(0)
					Search_Result=Bool(Search_Len=Len(string))
					ProcedureReturn Search_Start
				EndSelect
			EndIf
		Else
			ProcedureReturn #Null;	String not found
		EndIf
	Else
		Debug "error"
		ProcedureReturn #Null;	Find expression error
	EndIf

EndProcedure
Procedure.s ReplaceConstanst(s.s)

	Protected i,n

	i=const
	While i
		i-1
		If c(i)\type=#ConstValue
			s=ReplaceString(s,c(i)\name,Str(c(i)\value))
		EndIf
	Wend

	Replace_String=s
	ProcedureReturn s

EndProcedure
Procedure Expression(s.s,mode=#ExpressionSemicolon)

	Protected n
	Protected t.s

	If mode=#ExpressionSemicolon
		If Search(s,".*;",#SearchGetAll)
			Expression_String=Left(s,Search_Len-1)
			s=ReplaceConstanst(Expression_String)
			; Integer Values
			If Search(s,"[0-9]+",#SearchGetAll) And Search_Result
				Expression_Val=Val(s)
				ProcedureReturn #True
			EndIf
			; Hex Values
			If Search(s,"0x[0-9]+",#SearchGetAll) And Search_Result
				Expression_Val=Val("$"+Mid(s,3))
				ProcedureReturn #True
			EndIf
			; Other Values
			Expression_Val=EVal(s)
			ProcedureReturn #True

		EndIf
	EndIf

	ProcedureReturn 0

EndProcedure
Procedure ReadSource()

	Protected s.s,t.s
	Protected c,v

	total=0

	If 1;								ReadFile(0,"Birdie.mc")

		zeile=0
		While 1;						Eof(0)=0
			ReDim s(total)

			Read.s s;					s=ReadString(0)
			s=ReplaceString(s,"~",#Q)
			If s="EOF"
				Break
			EndIf

			zeile+1
			With s(total)
				\line=s

				; Kommentare ignorieren
				If Search(s,".*?//",#SearchGetAll)
					s=Left(s,Search_Next-3)
				EndIf
				If remark
					If Search(s,".*?\*/",#SearchGetAll)
						s=Mid(s,Search_Next)
						remark=#False
					EndIf
				Else
					If Search(s,".*?/\*",#SearchGetAll)
						remark=#True
						s=""
					EndIf
				EndIf
				If remark=#False
					s=MyTrim(s);			Remove Tabs
					If s
						s=Clean(s);			Remove text (which could cotain 'names') within quotes, e.g. i="i+j"; > i="";
						s=ReplaceString(s,#TAB$," ")
						If #ShowCode
							Debug RSet(Str(zeile),5,"0")+" "+s
						EndIf

						; CLASS
						If Search(s,"[ \t]*class[ \t]*",#SearchGetAll)
							s=Mid(s,Search_Next)
							If Search(s,#Spaces+#Names+#Spaces,#SearchGetAll)
								If #ShowClasses
									Debug "*CLS* '"+MyTrim(Search_String)+"'"
								EndIf
							EndIf
						EndIf

						; FUNCTION
						If Search(s,"[ \t]*function[ \t]*",#SearchGetAll)
							s=Mid(s,Search_Next)
							If Search(s,#Spaces+#Names+#Spaces,#SearchGetAll)
								If #ShowFunctions
									Debug "*FNC* '"+MyTrim(Search_String)+"'"
								EndIf
							EndIf
						EndIf

						; VAR
						If Search(s,"[ \t]*var[ \t]*",#SearchGetAll)
							s=Mid(s,Search_Next)
							If Search(s,#Spaces+#Names+#Delimiter,#SearchGetAll)
								If #ShowVariables
									Debug "*VAR* '"+MyTrim(Search_String)+"'"
								EndIf
							EndIf
						EndIf

						; CONST
						If Search(s,"[ \t]*const[ \t]*",#SearchGetAll)
							s=Mid(s,Search_Next)
							If Search(s,#Spaces+#Names+#Spaces,#SearchGetAll)
								If #ShowConstants
									Debug "*CNS* '"+MyTrim(Search_String)+"'"
								EndIf
							EndIf
						EndIf

				EndIf
			EndIf

		EndWith

		total+1
	Wend

EndIf

ProcedureReturn total

EndProcedure

Procedure main()

Protected i

If ReadSource()
	i=0
	While i<total
		;Debug s(i)\line
		i+1
	Wend

EndIf

EndProcedure

main()
User avatar
Lunasole
Addict
Addict
Posts: 1091
Joined: Mon Oct 26, 2015 2:55 am
Location: UA
Contact:

Re: Another tricky thing - parsing a source text

Post by Lunasole »

Man that's surely crazy idea with parsing C/C++ code ^^
I've played previously with such things, but failed to make something universal (only few parsers, limited and specific for some projects). I also tried to take some C/C++ compiler and use data which it generates - that's THEORETICALLY gives really cool results, but it takes so many time and efforts to modify compiler on practice that I've abandoned that.
Also in your case you trying to parse OOP classes, which is even more than crazy idea :3
Michael Vogel wrote:A new problem for regex experts :)

I need to get all variable names from a source code, which is not critical if they are defined by a simple var xxx; or var yyy=1; but there are more variants allowed, like var a = 2, b , c =a * b ; which could span multiple lines as well. Here I would need to detect the variables 'a', 'b' and 'c' !
As for me RegExp will not help here, maybe should just search for "var " pattern without regexps, then parse every match (checking if it is really variable declaration, extracting vars from it), taking also string before it.
Typical "dirty" solution I'm using in such parsers ^^ Good luck with it all

PS. However something like this (or expressions) anyway looks not enough to check where variables are used , need to have whole map of code, logically separated on "scopes". Maybe should just try using some IDE or static code analyzer ?
"W̷i̷s̷h̷i̷n̷g o̷n a s̷t̷a̷r"
Post Reply