From 2b12abded2ab10d9168920462ae3df1360b0c240 Mon Sep 17 00:00:00 2001 From: Vinicius Stock Date: Thu, 4 May 2023 13:57:45 -0400 Subject: [PATCH 1/2] Add semantic highlighting documentation for themes --- README.md | 3 +++ SEMANTIC_HIGHLIGHTING.md | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 SEMANTIC_HIGHLIGHTING.md diff --git a/README.md b/README.md index cf6ec832a..0cd3c7a84 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,9 @@ end See the [documentation](https://shopify.github.io/ruby-lsp) for more in-depth details about the [supported features](https://shopify.github.io/ruby-lsp/RubyLsp/Requests.html). +For creating rich themes for Ruby using the semantic highlighting information, see the [semantic highlighting +documentation](SEMANTIC_HIGHLIGHTING.md). + ## Learn More * [RubyConf 2022: Improving the development experience with language servers](https://www.youtube.com/watch?v=kEfXPTm1aCI) ([Vinicius Stock](https://github.com/vinistock)) diff --git a/SEMANTIC_HIGHLIGHTING.md b/SEMANTIC_HIGHLIGHTING.md new file mode 100644 index 000000000..e607db4d3 --- /dev/null +++ b/SEMANTIC_HIGHLIGHTING.md @@ -0,0 +1,37 @@ +# Semantic highlighting + +The Ruby LSP supports semantic highlighting. This feature informs editors about the right token types for each part of +the code to allow for richer and accurate highlighting. The strategy taken by the Ruby LSP is to only return tokens for +syntax that is ambiguous in Ruby (as opposed to all existing tokens) to optimize for performance. + +An example of ambiguous syntax in Ruby are local variables and method calls. If you look at this line in isolation: +```ruby +foo +``` +it is not possible to tell if `foo` is a local variable or a method call. It depends on whether `foo` was assigned to +something before or not. This is one scenario where semantic highlighting removes the ambiguity for themes, returning +the correct token type by statically analyzing the code. + +To enhance a theme's Ruby syntax highlighting using the Ruby LSP, check the inform below. You may also want to check out +the [Spinel theme](https://github.com/Shopify/vscode-shopify-ruby/blob/main/themes/dark_spinel.json) as an example, +which uses all of the Ruby LSP's semantic highlighting information. + +## Token types + +According to the LSP specification, language servers can either use token types and modifiers from the [default +list](https://microsoft.github.io/language-server-protocol/specification#semanticTokenTypes) or contribute new semantic +tokens of their own. Currently, the Ruby LSP does not contribute any new semantic tokens and only uses the ones +contained in the default list. + +## Token list + +| Syntax | Type.Modifier | Note | +| ------------- | ------------- | ------------- | +| Sorbet annotation methods such as `let` or `cast` | type | Not every annotation is handled | +| Method calls with any syntax | method | | +| Constant references (including classes and modules) | namespace | We don't yet differentiate between module and class references | +| Method definition | method.declaration | | +| self | variable.default_library | | +| Method, block and lambda arguments | parameter | | +| Class declaration | class.declaration | | +| Module declaration | class.declaration | | From d1ef7fe5ebbff2906bf35a7395deb62a4bd083f0 Mon Sep 17 00:00:00 2001 From: Vinicius Stock Date: Thu, 4 May 2023 13:57:56 -0400 Subject: [PATCH 2/2] Fix module declaration token type --- SEMANTIC_HIGHLIGHTING.md | 8 ++++---- .../requests/semantic_highlighting.rb | 2 +- .../module_and_reference.exp.json | 19 +++++++++++++++++++ 3 files changed, 24 insertions(+), 5 deletions(-) create mode 100644 test/expectations/semantic_highlighting/module_and_reference.exp.json diff --git a/SEMANTIC_HIGHLIGHTING.md b/SEMANTIC_HIGHLIGHTING.md index e607db4d3..833edda32 100644 --- a/SEMANTIC_HIGHLIGHTING.md +++ b/SEMANTIC_HIGHLIGHTING.md @@ -1,7 +1,7 @@ # Semantic highlighting The Ruby LSP supports semantic highlighting. This feature informs editors about the right token types for each part of -the code to allow for richer and accurate highlighting. The strategy taken by the Ruby LSP is to only return tokens for +the code to allow for rich and accurate highlighting. The strategy taken by the Ruby LSP is to only return tokens for syntax that is ambiguous in Ruby (as opposed to all existing tokens) to optimize for performance. An example of ambiguous syntax in Ruby are local variables and method calls. If you look at this line in isolation: @@ -12,9 +12,9 @@ it is not possible to tell if `foo` is a local variable or a method call. It dep something before or not. This is one scenario where semantic highlighting removes the ambiguity for themes, returning the correct token type by statically analyzing the code. -To enhance a theme's Ruby syntax highlighting using the Ruby LSP, check the inform below. You may also want to check out -the [Spinel theme](https://github.com/Shopify/vscode-shopify-ruby/blob/main/themes/dark_spinel.json) as an example, -which uses all of the Ruby LSP's semantic highlighting information. +To enhance a theme's Ruby syntax highlighting using the Ruby LSP, check the information below. You may also want to +check out the [Spinel theme](https://github.com/Shopify/vscode-shopify-ruby/blob/main/themes/dark_spinel.json) as an +example, which uses all of the Ruby LSP's semantic highlighting information. ## Token types diff --git a/lib/ruby_lsp/requests/semantic_highlighting.rb b/lib/ruby_lsp/requests/semantic_highlighting.rb index 140d4535e..a4fd9c921 100644 --- a/lib/ruby_lsp/requests/semantic_highlighting.rb +++ b/lib/ruby_lsp/requests/semantic_highlighting.rb @@ -347,7 +347,7 @@ def visit_class(node) def visit_module(node) return super unless visible?(node, @range) - add_token(node.constant.location, :class, [:declaration]) + add_token(node.constant.location, :namespace, [:declaration]) visit(node.bodystmt) end diff --git a/test/expectations/semantic_highlighting/module_and_reference.exp.json b/test/expectations/semantic_highlighting/module_and_reference.exp.json new file mode 100644 index 000000000..e90909a1b --- /dev/null +++ b/test/expectations/semantic_highlighting/module_and_reference.exp.json @@ -0,0 +1,19 @@ +{ + "params": [], + "result": [ + { + "delta_line": 0, + "delta_start_char": 7, + "length": 3, + "token_type": 0, + "token_modifiers": 1 + }, + { + "delta_line": 3, + "delta_start_char": 0, + "length": 3, + "token_type": 0, + "token_modifiers": 0 + } + ] +}