Refactoring the way rule severity works

[ssbarnea]

As we plan to soon introduce profiles, ones that can enable/disable various rules and customize the severity of various rule violations, we are likely to want to change the current implementations.

Now each rule class as a .severity property which is a string with a limited set of values: VERY_HIGH, HIGH, MEDIUM, LOW, VERY_LOW.

That is limiting the usability as these are not naturally sortable and it would be extremely hard to allocated new values between these.

Going for numeric values would seem like something that would make more sense.

Notable implementations

SARIF Standard

Latest 2.1.0 spec has two fields:

• level: an enum [ "none", "note", "warning", "error" ] with default warning.
• rank: float with default and min value of -1.0 and max value of 100.0. Described as "A number representing the priority or importance of the result."

Some interesting notes about rank can be seen at https://docs.oasis-open.org/sarif/sarif/v2.0/csprd02/sarif-v2.0-csprd02.html#_Toc10127854

Not sure about the meaning of the negative value but I suppose their purpose was to use it as null/None value and avoid having a field that can also be null?

vscode problems

We need to identify what it supports as I was not able to find a link to it. It is important as that a key place where we expect to display outcomes.

Based on https://michaelheap.com/getting-started-problem-matchers/ it seems that they have a numeric severity. A grep on the codebase indicates that they might use something like

export enum MarkerSeverity {
	Hint = 1,
	Info = 2,
	Warning = 4,
	Error = 8,
}