[reST] refactor

[jrappen]

Note

⚠️ This PR is a work-in-progress. If you have suggestions for stuff that
should be fixed, leave a comment below.

Additions:

• correctly hightlight directive options and their values as
  mapping.pair while checking value format
• default keymap
• symbol index now lists explicit and implicit hyperlink
  targets (sections, footnotes, citations)
• highlight embedded python and raw-code directives (css, js, json,
  jsonc, html, python, toml, yaml), compare [reStructuredText] Code-blocks are not handled #3158
• explicit hyperlink targets and anonymous explicit hyperlink
  targets
• correctly highlight grid tables with optional inline markup
• correctly highlight inline interpreted text, optionally
  with roles
• correctly highlight substitution definitions
• completions with strict scope limits
• indentation settings based upon official docs
• neg. tests for comments
• highlight empty comments and section separators between empty
  lines
• add metadata to enable folding
• added window command to make selections a literal command
• added window command to make selections an interpreted text role

Fixes:

• line blocks (with optional inline markup)
• footnotes and citations
• pop explicit markup blocks before the next explicit markup block
• correctly highlight (valid) comments as block comments
• do not highlight paragraphs starting with ... as comments
• include inline markup within block quotes
• allowed symbols for heading markers, compare [reStructuredText] Headers using some of the supported characters aren't highlighted #2894
• inline internal targets may contain symbol characters, compare
  [RestructuredText] single quotes break link targets #1204
• inline hyperlink references
• section headings:
  • with overline and underline
  • with underline only
  • optional inset when with overline and underline
• literal blocks:
  • meta scope
  • literal blocks with expanded form
  • literal blocks with partially minimized form
  • literal blocks with fully minimized form
  • literal blocks with quoted form
• enumerated lists
• field lists
• punctuation of inline substitution references
• tests for embedded python
• inline literal for single backtick

Changes:

• the syntax file now uses branching to correctly highlight
  section headings and their punctuation
• scope names have been adjusted with regard to the syntax
  scope naming guide
• match punctuation.definition.end rules for inline items
  before invalid.illegal.newline
• pop line blocks on empty line
• pop explicit markup blocks before next explicit markup block
• moved some rules from block-quote-block context to literal-block
• removed inline markup from literal blocks
• updated list of allowed characters for internal link labels,
  compare [reST] Internal link labels should allow more character types #793
• changed auto_complete_selector default
• re-ordered test file to match order of syntax file
• include linebreak in match for first line of directives
• differentiate known (constant.language) and unknown (constant.other)
  directive names

BREAKING CHANGES:

• this pull request requires py3.13 and Sublime Text v4201+

References:

• fixes [reST] Internal link labels should allow more character types #793
• fixes [RestructuredText] single quotes break link targets #1204
• fixes [reStructuredText] Headers using some of the supported characters aren't highlighted #2894
• fixes [reStructuredText] Code-blocks are not handled #3158
• duplicate the work @deathaxe had done in [Common] Add git conflict marker highlighting #4047, so his changes to the
  syntax don't get lost

Thanks to these contributors:

• deathaxe deathaxe@users.noreply.github.com
• Giampaolo Rodola giampaolo@users.noreply.github.com
• Victor Wheeler vwheeler63@users.noreply.github.com

[vwheeler63]

@jrappen More syntax bugs:

• I think I found a good one for you. I just encountered this. I don't understand why that whole section is blue and underlined. I'm guessing a run-away regex?

  .. _sometext:
  
  Subsection heading
  ******************
  
  .. note:
    more text

  

  • Compare the docs in this source: https://github.com/lvgl/lvgl/tree/master/docs/
  • Published here: https://docs.lvgl.io/9.3/

• With inline literals within a paragraph that span across a line break:

  Some text ``inline
  literal`` more text.

  

[vwheeler63]

@jrappen

Also as a reminder, the https://sphinx-themes.org/sample-sites/furo/kitchen-sink/ web page has several sub-pages that by clicking the "eye" icon, you can get to the .rst source code. And they are real, production-quality reST examples of a wide variety of reST formatting, that if I were you, I would be using for test files for syntax-highlighting and formatting tests!

You're also welcome to use the docs at LVGL User Documentation for some production-quality .rst docs to use for testing.

[jrappen]

Excluded punctuation from explicit hyperlink target symbols in latest force-push.

[jrappen]

Added requested custom window commands.

[vwheeler63]

Hi, @jrappen !

I finally got around to putting this branch in my Packages/RestructuredText/ directory and a few things still aren't working correctly. I have found 2 so far and they are below.

1. This seems to be being triggered by the link target above the title:

   

2. Given this external link target

   .. _Format Specification Mini-Language:  https://docs.python.org/3/library/string.html#formatspec

   the link reference is still displayed un-highlighted with the hyphen (which is legal):

   

   whereas if I make it into a space, it IS highlighted:

   

[giampaolo]

Hi @jrappen. In my own override of the RsT syntax I'm using this:

    # class directive
    - match: '^(\s*)((\.\.)\s+class(::))\s+(?:\w+\.)*(\w+)'
      captures:
        2: meta.other.directive.restructuredtext
        3: punctuation.definition.directive.restructuredtext
        4: punctuation.separator.key-value.restructuredtext
        5: entity.name.class.restructuredtext
      push:
        - match: '^(?!\1\s+)(?=\s*\S+)'
          pop: true
        - include: inline

    # method directive
    - match: '^(\s*)((\.\.)\s+method(::))\s+(?:\w+\.)*(\w+)'
      captures:
        2: meta.other.directive.restructuredtext
        3: punctuation.definition.directive.restructuredtext
        4: punctuation.separator.key-value.restructuredtext
        5: entity.name.function.restructuredtext
      push:
        - match: '^(?!\1\s+)(?=\s*\S+)'
          pop: true
        - include: inline

    # function directive
    - match: '^(\s*)((\.\.)\s+function(::))\s+(?:\w+\.)*(\w+)'
      captures:
        2: meta.other.directive.restructuredtext
        3: punctuation.definition.directive.restructuredtext
        4: punctuation.separator.key-value.restructuredtext
        5: entity.name.function.restructuredtext
      push:
        - match: '^(?!\1\s+)(?=\s*\S+)'
          pop: true
        - include: inline

    # data directive
    - match: '^(\s*)((\.\.)\s+data(::))\s+(?:\w+\.)*(\w+)'
      captures:
        2: meta.other.directive.restructuredtext
        3: punctuation.definition.directive.restructuredtext
        4: punctuation.separator.key-value.restructuredtext
        5: entity.name.constant.restructuredtext
      push:
        - match: '^(?!\1\s+)(?=\s*\S+)'
          pop: true
        - include: inline

    # attribute directive
    - match: '^(\s*)((\.\.)\s+attribute(::))\s+(?:\w+\.)*(\w+)'
      captures:
        2: meta.other.directive.restructuredtext
        3: punctuation.definition.directive.restructuredtext
        4: punctuation.separator.key-value.restructuredtext
        5: entity.name.attribute.restructuredtext
      push:
        - match: '^(?!\1\s+)(?=\s*\S+)'
          pop: true
        - include: inline

With this I can highlight all the functions / methods / classes / attributes / CONSTANTs.

But most importantly, with this I can navigate the symbols of the .rst doc (show_overlay -> goto -> @).

Do you think you can integrate it into your work?

[jrappen]

Appreciate the feedback, will take another look later tonight.