GoScintilla - 2.7 (Purebasic 4.5 onwards)

[ts-soft]

GoSci is designed to work with a userdefined Lexer, userdefined Lexer is always Lexer 0!
You can't use other lexers with gosci.

[Tenaja]

GoSci is designed to work with a userdefined Lexer, userdefined Lexer is always Lexer 0!
You can't use other lexers with gosci.

Yes, I know that; I am trying to disable Gosci on the fly. Most code editors permit syntax highlighting changes on the fly, and I am trying to implement that. Since there is no point in reinventing the wheel with C, etc, I am trying to disable Gosci and use the standard SCI lexers.

Is the only way to do that to Remove the gadget created by Gosci and create a new one? That seems quite laborious, considering Scintilla, on its own, lets you toggle lexers at will. With a simple Scintilla test, you can change lexers as easy as this:

Code: Select all

ScintillaSendMessage(1, #SCI_SETLEXER, #SCLEX_CPP, 0)
ScintillaSendMessage(1, #SCI_COLOURISE, 0, -1
Debug ScintillaSendMessage(id, #SCI_GETLEXER)    ; displays 3, as it should

Yet once you use Gosci, the GetLexer returns 0 all the time. I am trying to figure out what Gosci does that prohibits toggling to the std lexers--the SCI_SETLEXER command is supposed to select between container (i.e. Gosci) and standard lexers.

[srod]

I've never used one of the Scintilla lexers, so can't really comment.

However, why does the following not work; why does the lexer value not change? Am I missing something daft here or is it as simple that the version compiled for PB does not allow the use of the built in lexers?

Code: Select all

 If OpenWindow(0, 0, 0, 320, 90, "ScintillaGadget", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  
     If InitScintilla()
        ScintillaGadget(0, 10, 10, 300, 70, #Null)
        Debug ScintillaSendMessage(0, #SCI_GETLEXER)
        ScintillaSendMessage(0, #SCI_SETLEXER, #SCLEX_CPP, 0)
        Debug ScintillaSendMessage(0, #SCI_GETLEXER) 
        
        ; Output set to red color
        ScintillaSendMessage(0, #SCI_STYLESETFORE, 0, RGB(255, 0, 0))
        
        ; Set the initial text to the ScintillaGadget
        ScintillaSendMessage(0, #SCI_SETTEXT, 0, @"This is a simple ScintillaGadget with text...")
        
        ; Adding a second line of text with linebreak before
        Text$ = Chr(10) + "Second line"
        ScintillaSendMessage(0, #SCI_APPENDTEXT, Len(Text$), @Text$)
     EndIf
     
     Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
  EndIf

[Tenaja]

SRod,
I had a short sample working, converted from another sample. Mine was very similar, but with more colors, and it worked. The only significant difference I could see was the library...

Your example doesn't work because it uses InitScintilla(). Switch it to InitScintillaStaticFull() and it works.
I also tried this, and it works, too:

Code: Select all

	TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll")
	If TestLexDLL = 0
		MessageRequester("Failure", "TestLexDLL Failed to load.")
	EndIf

I tried this, with failure:

Code: Select all

InitScintilla()
TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll")
	If TestLexDLL = 0
		MessageRequester("Failure", "TestLexDLL Failed to load.")
	EndIf

Everything is solved. The bottom line is the InitScintilla() function appears to either be missing the Lexers, or does not play friendly with them. The solution is to use one of these methods to init scintilla:

Code: Select all

InitScintillaStaticFull()    ; Success.

Code: Select all

TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll")   ; Success

Now we can happily switch back & forth between our custom Gosci lexer, and a standard Scintilla Lexer. Thanks for the help...

[ts-soft]

However, why does the following not work; why does the lexer value not change?

Hello stephen,
there is no other lexer in scintilla.dll shipped by PB

Cheers, Thomas

[srod]

Ah I see, Tenaja is using a different version of the lib.

Thanks for the tip Tenaja; glad that you've sorted it.

[Tenaja]

Ah I see, Tenaja is using a different version of the lib.

Thanks for the tip Tenaja; glad that you've sorted it.

Yes, it took your sample to realize just how much PB's library is lacking. Thanks for the help!

I actually started using the Static Library so it would make a single exe without the DLL, and did not initially realize it was even different. I toggled back and forth for testing, and initially found no differences.

[Justin]

Folding does not work. Try to collapse the first function in Blockcommentstyler.pb.

[srod]

Ah yes, the folding works, but a lot of the close-fold terminators are not being recognised.

There's only one thing which can cause that. One moment...

[srod]

@Justin : the problem was caused by a recent bug fix; fix one bug, create another, that's my philosophy!

I've uploaded a new version (2.6) which I hope sorts out code folding once and for all! I do find the Scintilla control quite strange in the way it decides on which parts of which lines need restyling at any given time, and it is this which is causing the majority of the problems thus far.


@ALL GoScintilla users : one tip I can give regarding GoScintilla and the various functions which allow you to opt not to restyle the document (or part of it) is that if you do opt to switch off restyling then Scintilla itself will usually still decide to force a restyling of the visible lines in the control (+ a few additional lines!)

As an example, consider the GOSCI_SetText() function which we could use to add, say, 1000 lines of text to the control in one go. If we set the 'reStyle' parameter to #False (to perhaps speed things up) then GoScintilla will not set about styling the 1000 lines. However, Scintilla itself will kick up a stink and will force us to style at least those lines which are visible, say the first 50 lines. Additional lines will then be styled when they are brought into view (through scrolling and such like).

So far so good then, no problem with this.

However, I point this out because there are potential pitfalls here. It means, for example, that those lines which have not been styled at least once will not have had their 'code folding' indexes calculated. If you then write some code which trawls through all 1000 lines looking for those lines marked as 'open fold headers', then you will miss all those lines which not been restyled as yet!

Take a look at the following code for example (taken from one of our demos) in which we populate a Scintilla control line by line by use of the GOSCI_InsertLineOfText() function.

Code: Select all

    If ReadFile(1, "exampleSource.c")
      While Eof(1) = #False
        text$ = ReadString(1)
        GOSCI_InsertLineOfText(1, -1, text$, #False) ;#False means that only the visible lines will be styled (more or less!)
      Wend
      CloseFile(1)
    EndIf

Note the #False parameter in the GOSCI_InsertLineOfText() function call. This tells the GOSCI_InsertLineOfText() function not to restyle the newly added line which will speed matters up considerably (if you are adding lots of lines). Now, in this case, when we are done, Scintilla will, because we haven't styled our newly added lines, force us to style the lines which will be made visible which is only right and just.

The question I need to ask is whether I need every line styling (even the ones not yet visible) so that things like code-folding indexes are set etc? Not a problem unless we need to write some code which examines individual lines etc. because even the lines not yet visible will eventually be styled when the user brings them into view and all the code folding would (should!) work fine.

Something to bear in mind.

Of course if you wish to force a control wide restyle at any time then simply use the GOSCI_SetState() function.

[Justin]

Thanks, folding works. But as you mentioned styling is lost when folding a block, in the same example folding the first function leaves the text below not styled until we use true in GOSCI_InsertLineOfText(), it is supposed to work like this?
Is there a folding event so we can force styling when it occurs?

[srod]

Yes, sorry that is indeed the way it works. My bad!

I said that Scintilla is 'strange' in the way it handles styling; always seeking to minimize the amount of restyling it does.

To the block comment demo you can add

Code: Select all

GOSCI_SetState(1, #GOSCI_RESTYLEDOCUMENT)

after the loop which populates the control. This will ensure that all lines have received their styling and will sort your problem out. Seems that collapsing a group does not automatically result in newly visible lines being restyled. An inconsistency on Scintilla's part by the looks of it, though I guess it is not unreasonable of Scintilla to expect that all lines will have received their styling bytes beforehand! To be honest, GOSCI_InsertLineOfText() is not a good way of populating the entire control; GOSCI_SetText() is better, far more efficient.

The truth is that the 'reStyle' parameters were added to GoScintilla on request by Tenaja and these parameters need to be used with some care (more care than I exhibited in those demos! Doh!)

[Perkin]

I've had a need to add search & replace capabilities, here's my efforts...
(Uses PB's PCRE and extends that with back-reference replacing.)

Save this alongside the GoScintilla.pbi as GoScintilla_Regex.pbi

Code: Select all

XIncludeFile "GoScintilla.pbi"

CompilerIf Defined(pcre_exec,#PB_Procedure) = 0
	CompilerIf #PB_Compiler_OS = #PB_OS_Linux
		ImportC ""
			pcre_exec(*pcre, *extra, subject, length, startoffset, options, *ovector, ovecsize) As "pb_pcre_exec"
		EndImport
	CompilerElse
		ImportC ""
			CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
				pcre_exec(*pcre,*extra,subject,length,startoffset,options,*ovector,ovecsize) As "pb_pcre_exec"
			CompilerElse
				pcre_exec(*pcre,*extra,subject,length,startoffset,options,*ovector,ovecsize) As "_pb_pcre_exec"
			CompilerEndIf
		EndImport
	CompilerEndIf
CompilerEndIf


#PCRE_CASELESS				= $00000001
#PCRE_MULTILINE				= $00000002 ; Same as #PB_RegularExpression_MultiLine
#PCRE_DOTALL					= $00000004 ; Same as #PB_RegularExpression_DotAll
#PCRE_EXTENDED        = $00000008 ; Same as #PB_RegularExpression_Extended
#PCRE_UNGREEDY        = $00000200
#PCRE_NEWLINE_CR			= $00100000
#PCRE_NEWLINE_LF			= $00200000
#PCRE_NEWLINE_CRLF		= $00300000
#PCRE_NEWLINE_ANY			= $00400000
#PCRE_NEWLINE_ANYCRLF	= $00500000 ; Same as #PB_RegularExpression_AnyNewLine

; Selects the first match of regex search from given position.
; Also scrolls that match into view
; returns position of match if found or -1 if not found
Procedure GOSCI_RegexSearch(id, text$, pos = 0, flags = 0)
	Protected *Buffer, length, regex_h, result=-1
	Protected Dim ov(2)
	If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
		If text$ = "" : ProcedureReturn -1 : EndIf
		regex_h = CreateRegularExpression(#PB_Any,text$,flags)
		If regex_h
			length = ScintillaSendMessage(id,#SCI_GETLENGTH)
			*Buffer = AllocateMemory(length)
			If *Buffer
				ScintillaSendMessage(id, #SCI_GETTEXT, length, *Buffer)
				pcre_exec(PeekI(regex_h), 0, *Buffer, length, pos, 0, @ov(), ArraySize(ov()))
				If ov(0) > -1
					ScintillaSendMessage(id, #SCI_SETSEL, ov(0), ov(1))
					result = ov(0)
				EndIf
				FreeMemory(*Buffer)
			EndIf
			FreeRegularExpression(regex_h)
		EndIf
	EndIf
	ProcedureReturn result
EndProcedure

; Replaces All occurences of search text with replace text
Procedure GOSCI_RegexReplaceAll(id, searchtext$, replacetext$, flags=0, clearUndoStack=#False)
	Protected text$
	Protected Dim ov(30)
	Protected utf8Len, subBuf$, reg_rep$, i
	If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
		If searchtext$ = "" : ProcedureReturn -1 : EndIf
		*regex_h.Integer = CreateRegularExpression(#PB_Any, searchtext$, flags)
		If *regex_h
			text$ = GOSCI_GetText(id)
			Repeat
				utf8Len = StringByteLength(text$, #PB_UTF8)
				subBuf$ = Space(utf8Len+2)
				PokeS(@subBuf$, text$, -1, #PB_UTF8)
				If pcre_exec(*regex_h\i, 0, @subBuf$, utf8Len, ov(1), 0, @ov(), ArraySize(ov())) < 0
					Break
				EndIf
				reg_rep$=replacetext$
				For i = 1 To 9
					If FindString(reg_rep$,"\"+Str(i),1)
						reg_rep$=ReplaceString(reg_rep$,"\"+Str(i),PeekS(@subBuf$+ov(i*2),ov(i*2+1)-ov(i*2), #PB_UTF8))
					EndIf
				Next
				text$ = PeekS(@subBuf$, ov(0), #PB_UTF8) + reg_rep$ + PeekS(@subBuf$+ov(1), utf8Len-ov(1), #PB_UTF8)
			ForEver
			FreeRegularExpression(*regex_h)
		EndIf
		GOSCI_SetText(id, text$, clearUndoStack)
	EndIf
	ProcedureReturn result
EndProcedure

Procedure GOSCI_RegexReplaceInSelected(id, searchtext$, replacetext$, flags=0, replace_all=#False)
	Protected text$
	Protected Dim ov(30)
	Protected utf8Len, subBuf$, reg_rep$, i
	If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
		If searchtext$ = "" : ProcedureReturn -1 : EndIf
		*regex_h.Integer = CreateRegularExpression(#PB_Any, searchtext$, flags)
		If *regex_h
			text$ = GOSCI_GetSelectedText(id)
			Repeat
				utf8Len = StringByteLength(text$, #PB_UTF8)
				subBuf$ = Space(utf8Len+2)
				PokeS(@subBuf$, text$, -1, #PB_UTF8)
				pcre_exec(*regex_h\i, 0, @subBuf$, utf8Len, ov(1), 0, @ov(), ArraySize(ov()))
				If ov(0)>-1
					reg_rep$=replacetext$
					For i = 1 To 9
						If FindString(reg_rep$,"\"+Str(i),1)
							reg_rep$=ReplaceString(reg_rep$,"\"+Str(i),PeekS(@subBuf$+ov(i*2),ov(i*2+1)-ov(i*2), #PB_UTF8))
						EndIf
					Next
					text$ = PeekS(@subBuf$, ov(0), #PB_UTF8) + reg_rep$ + PeekS(@subBuf$+ov(1), utf8Len-ov(1), #PB_UTF8)
				Else
					Break
				EndIf
				If replace_all=#False : Break : EndIf
			ForEver
			GOSCI_ReplaceSelectedText(id, text$)
			FreeRegularExpression(*regex_h)
		EndIf
	EndIf
	ProcedureReturn result
EndProcedure

Then for a test, here's an example, save it again alongside the GoScintilla.pbi or add an include path as in other provided examples and save alongside them.

Code: Select all

XIncludeFile "GoScintilla.pbi"
XIncludeFile "GoScintilla_Regex.pbi"

CompilerIf  #PB_Compiler_OS = #PB_OS_Windows 
	InitScintilla()
CompilerEndIf

Enumeration
	#g_sci
	#g_reset
	#g_search
	#g_replace
	#g_replaceALL
	#g_search2
	#g_replace2
EndEnumeration

If OpenWindow(0,0,0,650,270,"GO_Scintilla Regex Search + Replace test.",#PB_Window_ScreenCentered|#PB_Window_SystemMenu)
	RemoveKeyboardShortcut(0, #PB_Shortcut_Tab) ;Required for the tab key to function correctly when the Scintilla control has the focus.
	GOSCI_Create(#g_sci, 10, 40, 630, 220, 0, #GOSCI_AUTOSIZELINENUMBERSMARGIN)
	GOSCI_SetAttribute(#g_sci, #GOSCI_LINENUMBERAUTOSIZEPADDING, 10)
	GOSCI_SetAttribute(#g_sci, #GOSCI_WRAPLINES, #True)
	GOSCI_SetColor(#g_sci, #GOSCI_CARETLINEBACKCOLOR, $A0FFFF)
	GOSCI_SetFont(#g_sci, "Courier New", 12)
	GOSCI_SetTabs(#g_sci, 2)
	
	ButtonGadget(#g_reset,        5, 5, 90, 30, "Reset")
	ButtonGadget(#g_search,     105, 5, 90, 30, "Search 1")
	ButtonGadget(#g_replace,    205, 5, 90, 30, "Replace 1")
	ButtonGadget(#g_replaceALL, 305, 5, 90, 30, "All")
	ButtonGadget(#g_search2, 405, 5, 90, 30, "Search 2")
	ButtonGadget(#g_replace2, 505, 5, 90, 30, "Replace 2")
	
	flags=#PCRE_MULTILINE|#PCRE_NEWLINE_ANYCRLF|#PCRE_DOTALL|#PCRE_CASELESS
	
	test.s = "This is regex powered searching."+#CRLF$
	test.s + "This is his whisper."+#CRLF$+"Also handles"+#CRLF$+"newlines in search."+#CRLF$
	test.s + "Search 1 searches for the word 'regex' and scrolls into view if needed."+#CRLF$
	test.s + "Replace 1 replaces the selected text with 'super'."+#CRLF$
	test.s + "All replaces all occurences of 'is' which are followed with whitespace, with '_is_'."+#CRLF$
	test.s + "Search 2 searches for lines that end with lowercase letter and begin the next line with lowercase as well."+#CRLF$
	test.s + "Replace 2 searches the selected text and does same search as for Search 2 and replaces the linebreak with a space."
	
	GOSCI_SetText(#g_sci,test)
	
	Repeat
		ev=WaitWindowEvent()
		Select ev
			Case #PB_Event_Gadget
				EvG=EventGadget()
				
				Select EvG
					Case #g_reset
						GOSCI_SetText(#g_sci,test)
					Case #g_search
						search.s = "regex"
						GOSCI_RegexSearch(#g_sci,search,0,flags)
					Case #g_replace
						If GOSCI_GetSelectedText(#g_sci)
							GOSCI_ReplaceSelectedText(#g_sci,"super",#True)
						EndIf
					Case #g_replaceALL
						search.s = "(is)(?=\s)"
						replace.s= "_\1_"
						GOSCI_RegexReplaceAll(#g_sci,search,replace,flags)
					Case #g_search2
						search.s = "([a-z])\R([a-z])"
						GOSCI_RegexSearch(#g_sci,search,0,flags)
					Case #g_replace2
						search.s = "([a-z])\R([a-z])"
						replace.s= "\1 \2"
						GOSCI_RegexReplaceInSelected(#g_sci,search,replace,flags)
				EndSelect
			Case #PB_Event_CloseWindow
				QUIT=#True
		EndSelect
	Until QUIT=#True
EndIf

I tried to copy srod' layout for the procedures.

Please feel free to scrutinise/criticise/adapt as you see fit.
CompilerIf's at top may need tweaking for 64bit and MacOS.

This has been tried on WinXP, Win7 and Xubuntu 11.10 - all 32bit.

Edit: adjusted code - thanks srod.
Edit2: changed GOSCI_Search() to GOSCI_RegexSearch() : - avoid clash with tssoft's GoSCI_Search()

[srod]

Hi,

nice work; though, personally, I would prefer to use the native scintilla search and replace in target messages etc. I rarely use reg ex's myself.

Just puzzled with your use of the GOSCI_ReplaceSelectedText(id, text$, clearUndoStack) call.

My GOSCI_ReplaceSelectedText() function has no 'clearUndoStack' parameter! Are you using a modified version of GoScintilla here?

[Perkin]

Hi,

nice work; though, personally, I would prefer to use the native scintilla search and replace in target messages etc. I rarely use reg ex's myself.

Just puzzled with your use of the GOSCI_ReplaceSelectedText(id, text$, clearUndoStack) call.

My GOSCI_ReplaceSelectedText() function has no 'clearUndoStack' parameter! Are you using a modified version of GoScintilla here?

Thanks, good spot, I think that may have been a leftover from when I had it as GOSCI_SetText().
Will adjust that code now.