GoScintilla - 2.7 (Purebasic 4.5 onwards)

Tenaja · **Joined:** Tue Nov 09, 2010 10:15 pm **Posts:** 791

I am using GoScintilla for some custom lexering (

) and I thought it would be nice to be able to add the ability to also edit "other" files, since Scintilla has so many lexers built in.

I have successfully gotten the other lexers to work on their own. I pasted that code into my large editor file, and the text just goes plain, with no colors at all. I am wondering what is missing? Do I need to remove the Gosci callback?

Has anybody gotten Scintilla to work, switching back/forth from Gosci to Scintilla's lexers?

Thanks.

Tenaja · **Joined:** Tue Nov 09, 2010 10:15 pm **Posts:** 791

Tenaja wrote:

I am using GoScintilla for some custom lexering (

) and I thought it would be nice to be able to add the ability to also edit "other" files, since Scintilla has so many lexers built in.

I have successfully gotten the other lexers to work on their own. I pasted that code into my large editor file, and the text just goes plain, with no colors at all. I am wondering what is missing? Do I need to remove the Gosci callback?

Has anybody gotten Scintilla to work, switching back/forth from Gosci to Scintilla's lexers?

I have it isolated to the SetLexer command failing. In the simple sample, I use this, and see the debug values change as expected (0, then 3):

Code:

;lex setup
Debug ScintillaSendMessage(#Scintilla, #SCI_GETLEXER)
ScintillaSendMessage(#Scintilla, #SCI_SETLEXER, #SCLEX_CPP, 0)
Debug ScintillaSendMessage(#Scintilla, #SCI_GETLEXER)

However, in the program that uses Gosci, both of those debug statements result in 0. Why would the Scintilla gadget fail to assign the lexer using the same code, after using Gosci?

ts-soft · **Posted:** Wed Jun 13, 2012 5:39 pm

GoSci is designed to work with a userdefined Lexer, userdefined Lexer is always Lexer 0!
You can't use other lexers with gosci.

Tenaja · **Joined:** Tue Nov 09, 2010 10:15 pm **Posts:** 791

ts-soft wrote:

GoSci is designed to work with a userdefined Lexer, userdefined Lexer is always Lexer 0!
You can't use other lexers with gosci.

Yes, I know that; I am trying to disable Gosci on the fly. Most code editors permit syntax highlighting changes on the fly, and I am trying to implement that. Since there is no point in reinventing the wheel with C, etc, I am trying to disable Gosci and use the standard SCI lexers.

Is the only way to do that to Remove the gadget created by Gosci and create a new one? That seems quite laborious, considering Scintilla, on its own, lets you toggle lexers at will. With a simple Scintilla test, you can change lexers as easy as this:

Code:

ScintillaSendMessage(1, #SCI_SETLEXER, #SCLEX_CPP, 0)
ScintillaSendMessage(1, #SCI_COLOURISE, 0, -1
Debug ScintillaSendMessage(id, #SCI_GETLEXER)    ; displays 3, as it should

Yet once you use Gosci, the GetLexer returns 0 all the time. I am trying to figure out what Gosci does that prohibits toggling to the std lexers--the SCI_SETLEXER command is supposed to select between container (i.e. Gosci) and standard lexers.

srod · **Posted:** Wed Jun 13, 2012 8:08 pm

I've never used one of the Scintilla lexers, so can't really comment.

However, why does the following not work; why does the lexer value not change? Am I missing something daft here or is it as simple that the version compiled for PB does not allow the use of the built in lexers?

Code:

 If OpenWindow(0, 0, 0, 320, 90, "ScintillaGadget", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  
     If InitScintilla()
        ScintillaGadget(0, 10, 10, 300, 70, #Null)
        Debug ScintillaSendMessage(0, #SCI_GETLEXER)
        ScintillaSendMessage(0, #SCI_SETLEXER, #SCLEX_CPP, 0)
        Debug ScintillaSendMessage(0, #SCI_GETLEXER) 
        
        ; Output set to red color
        ScintillaSendMessage(0, #SCI_STYLESETFORE, 0, RGB(255, 0, 0))
        
        ; Set the initial text to the ScintillaGadget
        ScintillaSendMessage(0, #SCI_SETTEXT, 0, @"This is a simple ScintillaGadget with text...")
        
        ; Adding a second line of text with linebreak before
        Text$ = Chr(10) + "Second line"
        ScintillaSendMessage(0, #SCI_APPENDTEXT, Len(Text$), @Text$)
     EndIf
     
     Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
  EndIf

Tenaja · **Joined:** Tue Nov 09, 2010 10:15 pm **Posts:** 791

SRod,
I had a short sample working, converted from another sample. Mine was very similar, but with more colors, and it worked. The only significant difference I could see was the library...

Your example doesn't work because it uses InitScintilla(). Switch it to InitScintillaStaticFull() and it works.
I also tried this, and it works, too:

Code:

   TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll")
   If TestLexDLL = 0
      MessageRequester("Failure", "TestLexDLL Failed to load.")
   EndIf

I tried this, with failure:

Code:

InitScintilla()
TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll")
   If TestLexDLL = 0
      MessageRequester("Failure", "TestLexDLL Failed to load.")
   EndIf

Everything is solved. The bottom line is the InitScintilla() function appears to either be missing the Lexers, or does not play friendly with them. The solution is to use one of these methods to init scintilla:

Code:

InitScintillaStaticFull() ; Success.

Code:

TestLexDLL = OpenLibrary(#PB_Any, "SciLexer.dll") ; Success

Now we can happily switch back & forth between our custom Gosci lexer, and a standard Scintilla Lexer. Thanks for the help...

ts-soft · **Posted:** Wed Jun 13, 2012 8:55 pm

srod wrote:

However, why does the following not work; why does the lexer value not change?

Hello stephen,
there is no other lexer in scintilla.dll shipped by PB :wink:

Cheers, Thomas

srod · **Posted:** Wed Jun 13, 2012 8:57 pm

Ah I see, Tenaja is using a different version of the lib.

Thanks for the tip Tenaja; glad that you've sorted it.

Tenaja · **Joined:** Tue Nov 09, 2010 10:15 pm **Posts:** 791

srod wrote:

Ah I see, Tenaja is using a different version of the lib.

Thanks for the tip Tenaja; glad that you've sorted it.

Yes, it took your sample to realize just how much PB's library is lacking. Thanks for the help!

I actually started using the Static Library so it would make a single exe without the DLL, and did not initially realize it was even different. I toggled back and forth for testing, and initially found no differences.

Justin · **Joined:** Sat Apr 26, 2003 2:49 pm **Posts:** 348

Folding does not work. Try to collapse the first function in Blockcommentstyler.pb.

srod · **Posted:** Sat Jun 16, 2012 2:25 pm

Ah yes, the folding works, but a lot of the close-fold terminators are not being recognised.

There's only one thing which can cause that. One moment...

srod · **Posted:** Sat Jun 16, 2012 6:17 pm

@Justin : the problem was caused by a recent bug fix; fix one bug, create another, that's my philosophy!

I've uploaded a new version (2.6) which I hope sorts out code folding once and for all! I do find the Scintilla control quite strange in the way it decides on which parts of which lines need restyling at any given time, and it is this which is causing the majority of the problems thus far.

@ALL GoScintilla users : one tip I can give regarding GoScintilla and the various functions which allow you to opt not to restyle the document (or part of it) is that if you do opt to switch off restyling then Scintilla itself will usually still decide to force a restyling of the visible lines in the control (+ a few additional lines!)

As an example, consider the GOSCI_SetText() function which we could use to add, say, 1000 lines of text to the control in one go. If we set the 'reStyle' parameter to #False (to perhaps speed things up) then GoScintilla will not set about styling the 1000 lines. However, Scintilla itself will kick up a stink and will force us to style at least those lines which are visible, say the first 50 lines. Additional lines will then be styled when they are brought into view (through scrolling and such like).

So far so good then, no problem with this.

However, I point this out because there are potential pitfalls here. It means, for example, that those lines which have not been styled at least once will not have had their 'code folding' indexes calculated. If you then write some code which trawls through all 1000 lines looking for those lines marked as 'open fold headers', then you will miss all those lines which not been restyled as yet!

Take a look at the following code for example (taken from one of our demos) in which we populate a Scintilla control line by line by use of the GOSCI_InsertLineOfText() function.

Code:

    If ReadFile(1, "exampleSource.c")
      While Eof(1) = #False
        text$ = ReadString(1)
        GOSCI_InsertLineOfText(1, -1, text$, #False) ;#False means that only the visible lines will be styled (more or less!)
      Wend
      CloseFile(1)
    EndIf

Note the #False parameter in the GOSCI_InsertLineOfText() function call. This tells the GOSCI_InsertLineOfText() function not to restyle the newly added line which will speed matters up considerably (if you are adding lots of lines). Now, in this case, when we are done, Scintilla will, because we haven't styled our newly added lines, force us to style the lines which will be made visible which is only right and just.

The question I need to ask is whether I need every line styling (even the ones not yet visible) so that things like code-folding indexes are set etc? Not a problem unless we need to write some code which examines individual lines etc. because even the lines not yet visible will eventually be styled when the user brings them into view and all the code folding would (should!) work fine.

Something to bear in mind.

Of course if you wish to force a control wide restyle at any time then simply use the GOSCI_SetState() function.

Justin · **Joined:** Sat Apr 26, 2003 2:49 pm **Posts:** 348

Thanks, folding works. But as you mentioned styling is lost when folding a block, in the same example folding the first function leaves the text below not styled until we use true in GOSCI_InsertLineOfText(), it is supposed to work like this?
Is there a folding event so we can force styling when it occurs?

srod · **Posted:** Sat Jun 16, 2012 10:00 pm

Yes, sorry that is indeed the way it works. My bad!

I said that Scintilla is 'strange' in the way it handles styling; always seeking to minimize the amount of restyling it does.

To the block comment demo you can add

Code:

GOSCI_SetState(1, #GOSCI_RESTYLEDOCUMENT)

after the loop which populates the control. This will ensure that all lines have received their styling and will sort your problem out. Seems that collapsing a group does not automatically result in newly visible lines being restyled. An inconsistency on Scintilla's part by the looks of it, though I guess it is not unreasonable of Scintilla to expect that all lines will have received their styling bytes beforehand! To be honest, GOSCI_InsertLineOfText() is not a good way of populating the entire control; GOSCI_SetText() is better, far more efficient.

The truth is that the 'reStyle' parameters were added to GoScintilla on request by Tenaja and these parameters need to be used with some care (more care than I exhibited in those demos! Doh!)

Perkin · **Posted:** Mon Jul 02, 2012 11:12 am

I've had a need to add search & replace capabilities, here's my efforts...
(Uses PB's PCRE and extends that with back-reference replacing.)

Save this alongside the GoScintilla.pbi as GoScintilla_Regex.pbi

Code:

XIncludeFile "GoScintilla.pbi"

CompilerIf Defined(pcre_exec,#PB_Procedure) = 0
   CompilerIf #PB_Compiler_OS = #PB_OS_Linux
      ImportC ""
         pcre_exec(*pcre, *extra, subject, length, startoffset, options, *ovector, ovecsize) As "pb_pcre_exec"
      EndImport
   CompilerElse
      ImportC ""
         CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
            pcre_exec(*pcre,*extra,subject,length,startoffset,options,*ovector,ovecsize) As "pb_pcre_exec"
         CompilerElse
            pcre_exec(*pcre,*extra,subject,length,startoffset,options,*ovector,ovecsize) As "_pb_pcre_exec"
         CompilerEndIf
      EndImport
   CompilerEndIf
CompilerEndIf


#PCRE_CASELESS            = $00000001
#PCRE_MULTILINE            = $00000002 ; Same as #PB_RegularExpression_MultiLine
#PCRE_DOTALL               = $00000004 ; Same as #PB_RegularExpression_DotAll
#PCRE_EXTENDED        = $00000008 ; Same as #PB_RegularExpression_Extended
#PCRE_UNGREEDY        = $00000200
#PCRE_NEWLINE_CR         = $00100000
#PCRE_NEWLINE_LF         = $00200000
#PCRE_NEWLINE_CRLF      = $00300000
#PCRE_NEWLINE_ANY         = $00400000
#PCRE_NEWLINE_ANYCRLF   = $00500000 ; Same as #PB_RegularExpression_AnyNewLine

; Selects the first match of regex search from given position.
; Also scrolls that match into view
; returns position of match if found or -1 if not found
Procedure GOSCI_RegexSearch(id, text$, pos = 0, flags = 0)
   Protected *Buffer, length, regex_h, result=-1
   Protected Dim ov(2)
   If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
      If text$ = "" : ProcedureReturn -1 : EndIf
      regex_h = CreateRegularExpression(#PB_Any,text$,flags)
      If regex_h
         length = ScintillaSendMessage(id,#SCI_GETLENGTH)
         *Buffer = AllocateMemory(length)
         If *Buffer
            ScintillaSendMessage(id, #SCI_GETTEXT, length, *Buffer)
            pcre_exec(PeekI(regex_h), 0, *Buffer, length, pos, 0, @ov(), ArraySize(ov()))
            If ov(0) > -1
               ScintillaSendMessage(id, #SCI_SETSEL, ov(0), ov(1))
               result = ov(0)
            EndIf
            FreeMemory(*Buffer)
         EndIf
         FreeRegularExpression(regex_h)
      EndIf
   EndIf
   ProcedureReturn result
EndProcedure

; Replaces All occurences of search text with replace text
Procedure GOSCI_RegexReplaceAll(id, searchtext$, replacetext$, flags=0, clearUndoStack=#False)
   Protected text$
   Protected Dim ov(30)
   Protected utf8Len, subBuf$, reg_rep$, i
   If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
      If searchtext$ = "" : ProcedureReturn -1 : EndIf
      *regex_h.Integer = CreateRegularExpression(#PB_Any, searchtext$, flags)
      If *regex_h
         text$ = GOSCI_GetText(id)
         Repeat
            utf8Len = StringByteLength(text$, #PB_UTF8)
            subBuf$ = Space(utf8Len+2)
            PokeS(@subBuf$, text$, -1, #PB_UTF8)
            If pcre_exec(*regex_h\i, 0, @subBuf$, utf8Len, ov(1), 0, @ov(), ArraySize(ov())) < 0
               Break
            EndIf
            reg_rep$=replacetext$
            For i = 1 To 9
               If FindString(reg_rep$,"\"+Str(i),1)
                  reg_rep$=ReplaceString(reg_rep$,"\"+Str(i),PeekS(@subBuf$+ov(i*2),ov(i*2+1)-ov(i*2), #PB_UTF8))
               EndIf
            Next
            text$ = PeekS(@subBuf$, ov(0), #PB_UTF8) + reg_rep$ + PeekS(@subBuf$+ov(1), utf8Len-ov(1), #PB_UTF8)
         ForEver
         FreeRegularExpression(*regex_h)
      EndIf
      GOSCI_SetText(id, text$, clearUndoStack)
   EndIf
   ProcedureReturn result
EndProcedure

Procedure GOSCI_RegexReplaceInSelected(id, searchtext$, replacetext$, flags=0, replace_all=#False)
   Protected text$
   Protected Dim ov(30)
   Protected utf8Len, subBuf$, reg_rep$, i
   If IsGadget(id) And GadgetType(id) = #PB_GadgetType_Scintilla
      If searchtext$ = "" : ProcedureReturn -1 : EndIf
      *regex_h.Integer = CreateRegularExpression(#PB_Any, searchtext$, flags)
      If *regex_h
         text$ = GOSCI_GetSelectedText(id)
         Repeat
            utf8Len = StringByteLength(text$, #PB_UTF8)
            subBuf$ = Space(utf8Len+2)
            PokeS(@subBuf$, text$, -1, #PB_UTF8)
            pcre_exec(*regex_h\i, 0, @subBuf$, utf8Len, ov(1), 0, @ov(), ArraySize(ov()))
            If ov(0)>-1
               reg_rep$=replacetext$
               For i = 1 To 9
                  If FindString(reg_rep$,"\"+Str(i),1)
                     reg_rep$=ReplaceString(reg_rep$,"\"+Str(i),PeekS(@subBuf$+ov(i*2),ov(i*2+1)-ov(i*2), #PB_UTF8))
                  EndIf
               Next
               text$ = PeekS(@subBuf$, ov(0), #PB_UTF8) + reg_rep$ + PeekS(@subBuf$+ov(1), utf8Len-ov(1), #PB_UTF8)
            Else
               Break
            EndIf
            If replace_all=#False : Break : EndIf
         ForEver
         GOSCI_ReplaceSelectedText(id, text$)
         FreeRegularExpression(*regex_h)
      EndIf
   EndIf
   ProcedureReturn result
EndProcedure

Then for a test, here's an example, save it again alongside the GoScintilla.pbi or add an include path as in other provided examples and save alongside them.

Code:

XIncludeFile "GoScintilla.pbi"
XIncludeFile "GoScintilla_Regex.pbi"

CompilerIf  #PB_Compiler_OS = #PB_OS_Windows 
   InitScintilla()
CompilerEndIf

Enumeration
   #g_sci
   #g_reset
   #g_search
   #g_replace
   #g_replaceALL
   #g_search2
   #g_replace2
EndEnumeration

If OpenWindow(0,0,0,650,270,"GO_Scintilla Regex Search + Replace test.",#PB_Window_ScreenCentered|#PB_Window_SystemMenu)
   RemoveKeyboardShortcut(0, #PB_Shortcut_Tab) ;Required for the tab key to function correctly when the Scintilla control has the focus.
   GOSCI_Create(#g_sci, 10, 40, 630, 220, 0, #GOSCI_AUTOSIZELINENUMBERSMARGIN)
   GOSCI_SetAttribute(#g_sci, #GOSCI_LINENUMBERAUTOSIZEPADDING, 10)
   GOSCI_SetAttribute(#g_sci, #GOSCI_WRAPLINES, #True)
   GOSCI_SetColor(#g_sci, #GOSCI_CARETLINEBACKCOLOR, $A0FFFF)
   GOSCI_SetFont(#g_sci, "Courier New", 12)
   GOSCI_SetTabs(#g_sci, 2)
   
   ButtonGadget(#g_reset,        5, 5, 90, 30, "Reset")
   ButtonGadget(#g_search,     105, 5, 90, 30, "Search 1")
   ButtonGadget(#g_replace,    205, 5, 90, 30, "Replace 1")
   ButtonGadget(#g_replaceALL, 305, 5, 90, 30, "All")
   ButtonGadget(#g_search2, 405, 5, 90, 30, "Search 2")
   ButtonGadget(#g_replace2, 505, 5, 90, 30, "Replace 2")
   
   flags=#PCRE_MULTILINE|#PCRE_NEWLINE_ANYCRLF|#PCRE_DOTALL|#PCRE_CASELESS
   
   test.s = "This is regex powered searching."+#CRLF$
   test.s + "This is his whisper."+#CRLF$+"Also handles"+#CRLF$+"newlines in search."+#CRLF$
   test.s + "Search 1 searches for the word 'regex' and scrolls into view if needed."+#CRLF$
   test.s + "Replace 1 replaces the selected text with 'super'."+#CRLF$
   test.s + "All replaces all occurences of 'is' which are followed with whitespace, with '_is_'."+#CRLF$
   test.s + "Search 2 searches for lines that end with lowercase letter and begin the next line with lowercase as well."+#CRLF$
   test.s + "Replace 2 searches the selected text and does same search as for Search 2 and replaces the linebreak with a space."
   
   GOSCI_SetText(#g_sci,test)
   
   Repeat
      ev=WaitWindowEvent()
      Select ev
         Case #PB_Event_Gadget
            EvG=EventGadget()
            
            Select EvG
               Case #g_reset
                  GOSCI_SetText(#g_sci,test)
               Case #g_search
                  search.s = "regex"
                  GOSCI_RegexSearch(#g_sci,search,0,flags)
               Case #g_replace
                  If GOSCI_GetSelectedText(#g_sci)
                     GOSCI_ReplaceSelectedText(#g_sci,"super",#True)
                  EndIf
               Case #g_replaceALL
                  search.s = "(is)(?=\s)"
                  replace.s= "_\1_"
                  GOSCI_RegexReplaceAll(#g_sci,search,replace,flags)
               Case #g_search2
                  search.s = "([a-z])\R([a-z])"
                  GOSCI_RegexSearch(#g_sci,search,0,flags)
               Case #g_replace2
                  search.s = "([a-z])\R([a-z])"
                  replace.s= "\1 \2"
                  GOSCI_RegexReplaceInSelected(#g_sci,search,replace,flags)
            EndSelect
         Case #PB_Event_CloseWindow
            QUIT=#True
      EndSelect
   Until QUIT=#True
EndIf

I tried to copy srod' layout for the procedures.

Please feel free to scrutinise/criticise/adapt as you see fit.
CompilerIf's at top may need tweaking for 64bit and MacOS.

This has been tried on WinXP, Win7 and Xubuntu 11.10 - all 32bit.

Edit: adjusted code - thanks srod.
Edit2: changed GOSCI_Search() to GOSCI_RegexSearch() : - avoid clash with tssoft's GoSCI_Search()

PureBasic Forum

GoScintilla - 2.7 (Purebasic 4.5 onwards)

Who is online