Syntax highlighting control

[mindfulvector]

I was wondering how XIDE accomplishes its syntax highlighting? I am working on a server monitoring tool which allows the user to write scripts in XSharp or T-SQL. I have a basic working syntax highlighting system built on RichText control, but have run into issues controlling the viewport of the RichText control when updating it from a "backbuffer" (another RichText control) after handling the actual highlighting there.

Any hints or code would be deeply appreciated!

[Chris]

Hi Isaac,

The editor in XIDE is completely custom made, every character is printed one by one on screen, so it's easy to do anything with it. Unfortunately I have no experience with doing anything like that with the RichEdit or any other standard control.

I had thought of offering the editor from XIDE as a separate tool, but the code is completely tied up with the rest of the IDE, so unfortunately it would be a lot of work to make it a standalone library. Maybe something for a rainy day...

[mindfulvector]

Understandable. I wonder if compiling the entire IDE as a DLL file and exposing a method to inject the main form (with all UI hidden except the editor panel and tabs) would be viable?

In case anyone else comes across this post, here is what I'm using for RichText control based highlighting. I had done something similar in VB6 many years ago, but can't quite recall how I solved the scrolling issue. I'll keep poking at it.

Code: Select all

    PROTECTED oRtbScriptCode AS RichTextBox
    PROTECTED oRtbHighlighBackBuffer AS RichTextBox
    // Syntax highlighting timer
    PROTECTED oHighlightTimer AS Timer

    PROTECCTED METHOD InitializeScriptEditorPanel() AS VOID
        SELF:oRtbScriptCode := RichTextBox{}
        SELF:oRtbScriptCode:Location := System.Drawing.Point{10, nYPos}
        SELF:oRtbScriptCode:Anchor := AnchorStyles.Top | AnchorStyles.Bottom | AnchorStyles.Left | AnchorStyles.Right
        SELF:oRtbScriptCode:Size := Size{275, 150}
        SELF:oRtbScriptCode:Font := System.Drawing.Font{"Consolas", 10, FontStyle.Regular}
        SELF:oRtbScriptCode:ScrollBars := RichTextBoxScrollBars.ForcedVertical
        SELF:oRtbScriptCode:WordWrap := TRUE
        SELF:oRtbScriptCode:AcceptsTab := TRUE
        SELF:oRtbScriptCode:TextChanged += RtbScriptCode_TextChanged

        SELF:oRtbHighlighBackBuffer := RichTextBox{}
        SELF:oRtbHighlighBackBuffer:Visible := FALSE
        SELF:oRtbScriptCode:Font := System.Drawing.Font{"Consolas", 10, FontStyle.Regular}
        
        // Initialize syntax highlighting timer
        SELF:oHighlightTimer := Timer{}
        SELF:oHighlightTimer:Interval := 500 // 0.5 seconds
        SELF:oHighlightTimer:Tick += HighlightTimer_Tick

    END METHOD
    PRIVATE METHOD RtbScriptCode_TextChanged(sender AS OBJECT, e AS EventArgs) AS VOID
        // Ignore text changes triggered by undo/redo operations
        IF SELF:lIgnoreTextChanges
            RETURN
        ENDIF

        SELF:lScriptModified := TRUE
        SELF:UpdateFormTitle()

        // Reset syntax highlighting timer
        SELF:oHighlightTimer:Stop()
        SELF:oHighlightTimer:Start()
    END METHOD

    PRIVATE METHOD HighlightTimer_Tick(sender AS OBJECT, e AS EventArgs) AS VOID
        SELF:oHighlightTimer:Stop()

        // Push current state to undo stack before applying syntax highlighting
        SELF:oScriptCodeUndoRedoManager:PushState(SELF:oRtbScriptCode:Text)

        SELF:oRtbScriptCode:ClearUndo()

        // Apply syntax highlighting
        SELF:ApplySyntaxHighlighting()
    END METHOD
    PRIVATE METHOD ApplySyntaxHighlighting() AS VOID
        STATIC scLastText := "" AS STRING
        IF SELF:oRtbScriptCode:Text:Length == 0 .OR. SELF:oRtbScriptCode:Text == scLastText
            RETURN
        ENDIF

        scLastText := SELF:oRtbScriptCode:Text
        SELF:oRtbHighlighBackBuffer:Text := SELF:oRtbScriptCode:Text

        // Save current selection and scroll position
        LOCAL nSelStart := SELF:oRtbScriptCode:SelectionStart AS INT
        LOCAL nSelLength := SELF:oRtbScriptCode:SelectionLength AS INT

        TRY
            // Disable redrawing during highlighting
            SELF:oRtbHighlighBackBuffer:SuspendLayout()

            // Reset all formatting
            SELF:oRtbHighlighBackBuffer:SelectAll()
            SELF:oRtbHighlighBackBuffer:SelectionColor := System.Drawing.Color.Black
            SELF:oRtbHighlighBackBuffer:SelectionFont := System.Drawing.Font{"Consolas", 10, FontStyle.Regular}

            // Apply language-specific highlighting
            LOCAL cLanguage := SELF:oCboLanguage:SelectedItem:ToString():ToUpper() AS STRING

            SWITCH cLanguage
            CASE "T-SQL"
                SELF:ApplySqlSyntaxHighlighting()
            CASE "POSTGRESQL"
                SELF:ApplyPostgreSqlSyntaxHighlighting()
            CASE "XSHARP"
                SELF:ApplyXSharpSyntaxHighlighting()
            END SWITCH

        FINALLY
            SELF:oRtbScriptCode:Rtf := SELF:oRtbHighlighBackBuffer:Rtf

            // Restore selection and scroll position
            SELF:oRtbScriptCode:Select(nSelStart, nSelLength)
            SELF:oRtbScriptCode:ResumeLayout()
        END TRY
    END METHOD

    PRIVATE METHOD ApplySqlSyntaxHighlighting() AS VOID
        LOCAL cText := SELF:oRtbScriptCode:Text AS STRING

        // SQL Keywords
        LOCAL aSqlKeywords := <STRING>{"SELECT", "FROM", "WHERE", "INSERT", "UPDATE", "DELETE", "CREATE", "ALTER", "DROP", ;
                                      "TABLE", "INDEX", "VIEW", "PROCEDURE", "FUNCTION", "DECLARE", "SET", "IF", "ELSE", ;
                                      "BEGIN", "END", "WHILE", "FOR", "CASE", "WHEN", "THEN", "AS", "AND", "OR", "NOT", ;
                                      "IN", "EXISTS", "BETWEEN", "LIKE", "IS", "NULL", "ORDER", "BY", "GROUP", "HAVING", ;
                                      "UNION", "JOIN", "INNER", "OUTER", "LEFT", "RIGHT", "ON", "DISTINCT", "TOP"} AS STRING[]

        // Highlight keywords
        FOREACH cKeyword AS STRING IN aSqlKeywords
            SELF:HighlightPattern(i"\b{cKeyword}\b", System.Drawing.Color.Blue, FontStyle.Bold, TRUE)
        NEXT

        // Highlight strings
        SELF:HighlightPattern("'[^']*'", System.Drawing.Color.Red, FontStyle.Regular, FALSE)

        // Highlight comments
        SELF:HighlightPattern("--.*$", System.Drawing.Color.Green, FontStyle.Italic, FALSE)
        SELF:HighlightPattern("/\*.*?\*/", System.Drawing.Color.Green, FontStyle.Italic, FALSE)

        // Highlight numbers
        SELF:HighlightPattern("\b\d+(\.\d+)?\b", System.Drawing.Color.Purple, FontStyle.Regular, FALSE)
    END METHOD

    PRIVATE METHOD ApplyPostgreSqlSyntaxHighlighting() AS VOID
        LOCAL cText := SELF:oRtbScriptCode:Text AS STRING

        // PostgreSQL Keywords (similar to SQL with some additions)
        LOCAL aPgKeywords := <STRING>{"SELECT", "FROM", "WHERE", "INSERT", "UPDATE", "DELETE", "CREATE", "ALTER", "DROP", ;
                                     "TABLE", "INDEX", "VIEW", "FUNCTION", "DECLARE", "SET", "IF", "ELSE", ;
                                     "BEGIN", "END", "WHILE", "FOR", "CASE", "WHEN", "THEN", "AS", "AND", "OR", "NOT", ;
                                     "IN", "EXISTS", "BETWEEN", "LIKE", "ILIKE", "IS", "NULL", "ORDER", "BY", "GROUP", "HAVING", ;
                                     "UNION", "JOIN", "INNER", "OUTER", "LEFT", "RIGHT", "ON", "DISTINCT", "LIMIT", "OFFSET"} AS STRING[]

        // Highlight keywords
        FOREACH cKeyword AS STRING IN aPgKeywords
            SELF:HighlightPattern(i"\b{cKeyword}\b", System.Drawing.Color.Blue, FontStyle.Bold, TRUE)
        NEXT

        // Highlight strings
        SELF:HighlightPattern("'[^']*'", System.Drawing.Color.Red, FontStyle.Regular, FALSE)

        // Highlight comments
        SELF:HighlightPattern("--.*$", System.Drawing.Color.Green, FontStyle.Italic, FALSE)
        SELF:HighlightPattern("/\*.*?\*/", System.Drawing.Color.Green, FontStyle.Italic, FALSE)

        // Highlight numbers
        SELF:HighlightPattern("\b\d+(\.\d+)?\b", System.Drawing.Color.Purple, FontStyle.Regular, FALSE)
    END METHOD

    PRIVATE METHOD ApplyXSharpSyntaxHighlighting() AS VOID
        LOCAL cText := SELF:oRtbScriptCode:Text AS STRING

        // XSharp Keywords
        LOCAL aXSharpKeywords := <STRING>{"USING", "CLASS", "METHOD", "FUNCTION", "PROCEDURE", "INTERFACE", "PROPERTY", ;
                                         "PUBLIC", "PRIVATE", "PROTECTED", "STATIC", "VIRTUAL", "OVERRIDE", "ABSTRACT", ;
                                         "IF", "ELSE", "ELSEIF", "ENDIF", "FOR", "FOREACH", "NEXT", "WHILE", "ENDDO", ;
                                         "DO", "CASE", "OTHERWISE", "ENDCASE", "TRY", "CATCH", "FINALLY", "END", ;
                                         "RETURN", "LOCAL", "AS", "IS", "AND", "OR", "NOT", "TRUE", "FALSE", "NULL"} AS STRING[]

        // Highlight keywords
        FOREACH cKeyword AS STRING IN aXSharpKeywords
            SELF:HighlightPattern(i"\b{cKeyword}\b", System.Drawing.Color.Blue, FontStyle.Bold, TRUE)
        NEXT

        // Highlight strings
        SELF:HighlightPattern('"[^"]*"', System.Drawing.Color.Red, FontStyle.Regular, FALSE)
        SELF:HighlightPattern("'[^']*'", System.Drawing.Color.Red, FontStyle.Regular, FALSE)

        // Highlight comments
        SELF:HighlightPattern("//.*$", System.Drawing.Color.Green, FontStyle.Italic, FALSE)
        SELF:HighlightPattern("/\*.*?\*/", System.Drawing.Color.Green, FontStyle.Italic, FALSE)
        SELF:HighlightPattern("\*.*$", System.Drawing.Color.Green, FontStyle.Italic, FALSE)

        // Highlight numbers
        SELF:HighlightPattern("\b\d+(\.\d+)?\b", System.Drawing.Color.Purple, FontStyle.Regular, FALSE)
    END METHOD

    PRIVATE METHOD HighlightPattern(cPattern AS STRING, oColor AS System.Drawing.Color, oStyle AS FontStyle, lIgnoreCase AS LOGIC) AS VOID
        TRY
            LOCAL oRegex AS Regex
            LOCAL oOptions := RegexOptions.Multiline AS RegexOptions

            IF lIgnoreCase
                oOptions |= RegexOptions.IgnoreCase
            ENDIF

            oRegex := Regex{cPattern, oOptions}
            LOCAL oMatches := oRegex:Matches(SELF:oRtbHighlighBackBuffer:Text) AS MatchCollection

            FOREACH oMatch AS Match IN oMatches
                SELF:oRtbHighlighBackBuffer:Select(oMatch:Index, oMatch:Length)
                SELF:oRtbHighlighBackBuffer:SelectionColor := oColor
                SELF:oRtbHighlighBackBuffer:SelectionFont := System.Drawing.Font{"Consolas", 10, oStyle}
            NEXT

        CATCH oEx AS Exception
            // Ignore regex errors for now
        END TRY
    END METHOD

[Chris]

Hi Isaac,

That gave me an idea: Please create a new core dialect app, add references to System.Windows.Forms, System.Drawing and the XIDE dlls XIDE.Base.dll, XIDE.Main.dll, XIDE.PluginSystem.dll and XIDE.Debugger.Core.dll and use this code:

Code: Select all

USING System.Windows.Forms
USING System.Drawing
USING Xide

[STAThreadAttribute] FUNCTION Start() AS VOID
	LOCAL oParams AS StartUpParams
	
	Application.EnableVisualStyles()
	Application.DoEvents()
	
	Glo.glDoNotLoadPlugins := TRUE
	Glo.glOpenFloating := TRUE
	
	Initializer{}
	//	Glo.gcGlobalCfg := Application.StartupPath + "\XIDE.cfg"
	
	oParams := GetStartUpParams()
	oParams:eAction := StartUpAction.LoadFile
	oParams:cFile := "c:\test\the file to open.prg"
	
	LOCAL oIde AS PyrgasIde
	oIde := PyrgasIde{oParams}
	
	LOCAL oFile AS FileControlBase
	oFile := oIde:GetFile(1)
	oIde:AddEditorForm(oFile, FALSE, Point{20, 20})
	oIde:Hide()
	
	Application.Run( oFile:FindForm() )

This should open the file in a floating window and you can edit it, save, etc. It's far from ideal (for example the main IDE is first shown before getting hidden etc), but if that's close to what you are looking for, I can make some changes to the main IDE to improve using it like that.

[mindfulvector]

Dang this is super cool, will experiment and let you know!

[mindfulvector]

I see the initial IDE window briefly, then the editor window comes up, then is hidden and the popup editor window appears.

One thing I noticed is that the requested file to edit is only opened in the initial IDE window. Is it possible to open it in the popup editor window?

[Chris]

Hi Isaac,

With the code I posted, it should open it in the floating window, that's what the call to AddEditorForm() does.

But anyway, if this standalone editor is what you're looking for, I'll make some adjustments to the core IDE code, so that it makes it easier to open a file in this detached window, without the problems of first showing the full IDE, projects initializations etc.

[mindfulvector]

Yeah, that would be perfect. All my windows in this application are popups like this already so this would fit in well with the overall workflow I have so far. Really appreciate it!

[Chris]

Hi Isaac,

I started looking again into this to implement what we discussed, but I noticed you have not subscribed to the FOX program. We normally do such custom development only to paying customers, as they are the ones that help funding this project to keep it going, so would you be willing to purchase at least a half year subscription? Also please send me a private email at chris at xsharp eu, so I can send you back the updated XIDE with this feature once I'm done with it.

[ic2]

Hello Isaac,

As a personal note from a Fox subscriber to Chris' remark: I can really recommend you to take Fox. Contrary to users who are waiting anxiously for X# 3.x I am not - the main thing I welcome is that editing in VS would be more comfortable like it is in VO, which is after all these years still miles ahead on VS.

But the times that Chris has personally looked into some malfunctioning code and always found the reason why alone has made my Fox subscription fully worth.

Dick