Page 1 of 1

[All Platforms] Tokenise (Parse) Command-Line Strings

Posted: Mon Dec 12, 2011 12:24 am
by Env
Hey all,

Here's a procedure that will take a CL-style string (delimited by spaces, defining multi-word tokens with quotation marks) and split it into tokens.

Code (Contains usage example)

Code: Select all

 ; ----------------------------------------------------------------------------------------------------
; Title:        Commandline String Tokeniser (Procedural)
; Description:  Procedure that provides a String Tokeniser for Commandlines.
; Author(s):    Michael R. King (mrking2910@gmail.com)
; Revision:     1
; Support:      Cross-Platform
;
; Notes:        Will use spaces as delimiters, and consider strings encapsulated 
;								by double quotes as a single token.
; ----------------------------------------------------------------------------------------------------

EnableExplicit

CompilerIf Defined(_PBI_CLTOK_, #PB_Constant) = #False
	#_PBI_CLTOK_ = #True
	
	Procedure.l CLTok(String$, List OutputList.s())
		Protected cCnt.l, cIx.l, c.s, tQuot.a, t.s
		ClearList(OutputList())
		cCnt = Len(String$)
		For cIx = 1 To (cCnt + 1)
			c = Mid(String$, cIx, 1)
			If c = #DQUOTE$
				tQuot = 1 - tQuot
				If tQuot = 0
					If t <> ""
						AddElement(OutputList()) : OutputList() = t
					EndIf
					t = ""
				EndIf
			EndIf
			If tQuot = #False
				If c = " " Or cIx = (cCnt + 1)
					If t <> ""
						AddElement(OutputList()) : OutputList() = t
					EndIf
					t = ""
				Else
					If c <> #DQUOTE$
						t = t + c
					EndIf
				EndIf
			Else
				If c <> #DQUOTE$
					t = t + c
				EndIf
			EndIf
		Next
		If tQuot = 1
			If t <> ""
				AddElement(OutputList()) : OutputList() = t
			EndIf
		EndIf
		ProcedureReturn ListSize(OutputList())
	EndProcedure
	
CompilerEndIf ;_PBI_CLTOK_

; -------------------------------------------------------------------------------------
; - DEMONSTRATION CODE - DEMONSTRATION CODE - DEMONSTRATION CODE - DEMONSTRATION CODE -
; -------------------------------------------------------------------------------------

; - Define our test string -
Define TestString$ = "hello world " + #DQUOTE$ + "this is" + #DQUOTE$ + "a test"

; - Create an Output List -
NewList Output.s()

; - Tokenise the Test String -
CLTok(TestString$, Output())

; - Display results in the Debug Window -
ForEach Output()
	Debug Output()
Next

; - Free the List -
FreeList(Output())

End
Hope someone finds it useful :)

Thanks :D

Re: [All Platforms] Tokenise (Parse) Command-Line Strings

Posted: Tue Dec 13, 2011 7:24 am
by Mistrel
I posted an alternative example of this a while ago except it's a 'C' compatible argc/arv pair, for those who are interested in this type of implementation:

http://www.purebasic.fr/english/viewtop ... 12&t=47009