Lua string multiline not respected

[sonoro1234]

Hi,

I dont know if there are other languages with multiline strings where you need to read several lines to reach end of string.
At least this happens with Lua. For what I have understood in the code, detecting things that need several lines (multiline comments) is done in ColorizeInternal (before calling ColorizeRange) where some flags are set in Gliphs: mComment, mMultiLineComment or mPreprocessor so that mColorIndex is respected or not in GetGlyphColor.

If all above is correct, the only way to process multiline strings and also multiline comments in Lua as

--[=[
one line
second line
--[[
another comment inside the other
which is multiline
]]
]=]

would be that ColorizeInternal instead of doing the same for all Languages, would call a function for doing this work depending on the language (If there is not such function would call a default one) and also having a flag in Glyph called mString.

What do you think about?

[santaclose]

there are several problems with syntax highlighting regarding strings and multiline comments. I was working on refactoring all of the colorizing logic but the work is not finished yet. I will upload what I have to a separate branch if you want to look at that. The idea is that from changes made to the undo stack, the Colorizer can run and update glyph colors, being able to do different things depending on the language.

I guess the solution you talk about can be a temporary approach as well.

[sonoro1234]

I will upload what I have to a separate branch if you want to look at that.

Great.

[sonoro1234]

I guess the solution you talk about can be a temporary approach as well.

But I am not interested in temporary work and prefer to wait for your new proposal

[santaclose]

uploaded the branch:
https://github.com/santaclose/ImGuiColorTextEdit/commits/syntax_highlighting_work/

[sonoro1234]

gonna try it!!!

[sonoro1234]

GetKeyIndex does not let me compile!!
Neither cmath include
I could modify above and compiled. I see only C language is done. It seems more performant when writting new characters than the master branch (But not for multiline comments)

[santaclose]

yeah there is still work to do, i wouldn't consider C language to be done, as i was just testing the interface i made (as far as i remember). My idea for multiline is that whenever a multiline sequence is entered or removed from the editor, a new thread (it's using the main thread for now) should probably start iterating from that location towards the end of the file writing the new colors for every glyph (I can't think of a case where we also need to update colors towards the beginning of the file).

Another thing i have been thinking about, but not that related to colorizing the text, is that characters should be stored in a 32 bit datatype and not bytes, as this would reduce complexity for utf-8 sequences longer than 1 byte (just so you know what my current plans are).

[sonoro1234]

I could compare with main branch of your fork adding multiline comment to big Cpp file (imgui.cpp). It is more performant in main branch.