Add semantic highlighting documentation for themes

README.md

@@ -39,6 +39,9 @@ end
 See the [documentation](https://shopify.github.io/ruby-lsp) for more in-depth details about the
 [supported features](https://shopify.github.io/ruby-lsp/RubyLsp/Requests.html).
 
+For creating rich themes for Ruby using the semantic highlighting information, see the [semantic highlighting
+documentation](SEMANTIC_HIGHLIGHTING.md).
+
 ## Learn More
 
 * [RubyConf 2022: Improving the development experience with language servers](https://www.youtube.com/watch?v=kEfXPTm1aCI) ([Vinicius Stock](https://github.com/vinistock))

SEMANTIC_HIGHLIGHTING.md

@@ -0,0 +1,37 @@
+# Semantic highlighting
+
+The Ruby LSP supports semantic highlighting. This feature informs editors about the right token types for each part of
+the code to allow for richer and accurate highlighting. The strategy taken by the Ruby LSP is to only return tokens for
+syntax that is ambiguous in Ruby (as opposed to all existing tokens) to optimize for performance.
+
+An example of ambiguous syntax in Ruby are local variables and method calls. If you look at this line in isolation:
+```ruby
+foo
+```
+it is not possible to tell if `foo` is a local variable or a method call. It depends on whether `foo` was assigned to
+something before or not. This is one scenario where semantic highlighting removes the ambiguity for themes, returning
+the correct token type by statically analyzing the code.
+
+To enhance a theme's Ruby syntax highlighting using the Ruby LSP, check the inform below. You may also want to check out
+the [Spinel theme](https://github.com/Shopify/vscode-shopify-ruby/blob/main/themes/dark_spinel.json) as an example,
+which uses all of the Ruby LSP's semantic highlighting information.
+
+## Token types
+
+According to the LSP specification, language servers can either use token types and modifiers from the [default
+list](https://microsoft.github.io/language-server-protocol/specification#semanticTokenTypes) or contribute new semantic
+tokens of their own. Currently, the Ruby LSP does not contribute any new semantic tokens and only uses the ones
+contained in the default list.
+
+## Token list
+
+| Syntax  | Type.Modifier | Note |
+| ------------- | ------------- | ------------- |
+| Sorbet annotation methods such as `let` or `cast`  | type | Not every annotation is handled |
+| Method calls with any syntax  | method | |
+| Constant references (including classes and modules)  | namespace | We don't yet differentiate between module and class references |
+| Method definition  | method.declaration | |
+| self  | variable.default_library | |
+| Method, block and lambda arguments | parameter | |
+| Class declaration | class.declaration | |
+| Module declaration | class.declaration | |