Add rule persistence support and refactor dashboard for better scalability

[cdfive]

Describe what this PR does / why we need it

Sentinel dashboard provide web UI where we can setting rules, viewing machines, viewing monitoring, etc, it's very useful and convenient for user, especially new coming user.

By default, rules are stored in memory, and users may need to persist the rule by different own way.
But at present, dashboard is a single module, and user-defined modification can only be made on the source code of the whole project, which is not very convenient for continuous upgrade and keep up with the latest version.

Does this pull request fix one issue?

Fixes #1241, #1257
Related #1375, #1415

Describe how you did it

To make it simple to use, a configuration parameter and a properties file is used to support one type of repository.For example, using nacos,
Add VM paramteter -Drule.repository.type=nacos,
Add a properties file Nacos.properties in the same folder path as sentinel-dashboard.jar, and in the properties file we can configure settings for Nacos.

Considering independence and extensibility, I tried to layer the dashboard project into the following separate modules:

• sentinel-dashboard-common
• sentinel-dashboard-entity
• sentinel-dashboard-repository
• sentinel-dashboard-repository-memory
• sentinel-dashboard-repository-redis
• sentinel-dashboard-repository-nacos
• sentinel-dashboard-repository-zookeeper
• sentinel-dashboard-repository-apollo
• sentinel-dashboard-service
• sentinel-dashboard-web

In sentinel-dashboard-common, common use class(config, util) , such as DashboardConfig,VersionUtils etc.

In sentinel-dashboard-entity , all entities used by dashboard, include rule entity, machine/app entity, metric entity.

In sentinel-dashboard-repository, based on DynamicRuleProvider/DynamicRulePublisher interface, abstract the common logic for persist rules with two abstract class AbstractRuleProvider/AbstractRulePublisher, methods are reserved for the realization by specific sub modules of different repository type.

• AtractRuleProvider#fetchRules
• AbstractRulePublisher#publishRules

The encoder/decoder package include serialization/deserialization for rules of different type, and the logic is for common use.

The RuleKeyBuilder interface define method String buildRuleKey(String app, String ip, Integer port); for generate the persistent key of rule.

For example, the flow rule, in Nacos:
If it's applied to one machine instance, the rule key is appName-ip-port-flow-rules, such as
sentinel-dashboard-192.168.1.100-8719-flow-rules.

If we set the rule which take application as dimension, that is all machine instances of this application apply the same rule, if have four instances, we'll persist four properties with four key in Nacos, the keys are:
app-ip1-port1-flow-rules
app-ip2-port2-flow-rules
app-ip3-port3-flow-rules
app-ip4-port4-flow-rules

Pay attention to that the RuleKeyBuilder#buildRuleKey method is a default implementation to generate ruleKey, which is alternative, and user can customize it in specific repository modules.

In specific repository modules, such as sentinel-dashboard-repository-nacos,sentinel-dashboard-repository-redis etc, the implementation is simple, we just need to focus on how to get and push rules with the specific repository. For example, NacosRuleProvider/NacosRulePublisher class extends AbstractRuleProvider/AbstractRulePublisher class, and override fetchRules/publishRules method.

NacosProperties is used for reading properties from file.

@PropertySource(value = {"classpath:repository/nacos.properties", "file:nacos.properties"}, ignoreResourceNotFound = true)

Pay attention to that there are two files, the first file are in /resources/repository/nacos.properties, which is convenient for debug in IDE environment, and the second file nacos.properties is useful
when package the sentinel-dashboard.jar, and it has higher priority.

In sentinel-dashboard-service, define vo(req/resp), api and implementation of CRUD functions for rule of different type, the key logic is encapsulated in AbstractRuleService class.

In sentinel-dashboard-web, are web UI(page/javascript/css), http RESTFul controller, web config and filters, it will be better if we split it into complete front and back-end separation in the furture.

Previous we have flow_v1,flow_v2, which may confuse new users, hope to support both set rules via machine or whole application, so the flow_v1,flow_v2 are merged into one file, in this way it's flexible and could meet most scenes.

Describe how to verify it

• Package sentinel-dashboard.jar, test with different type of repository(Nacos,Redis,Zookeeper etc)
  For Nacos repository:
  1.Start Nacos server
  2.Start dashboard with -Drule.repository.type=nacos

  Copy nacos.properties file from sentinel-dashboard/src/main/resource/repository/nacos.properties to the same folder path of `sentinel-dashboard.jar

  nacos.properties:

  nacos.serverAddr=localhost
  nacos.sentinelGroup=SENTINEL_GROUP

  Note that if Nacos server is in the same machine as sentinel-dashboard.jar, and use default setting
  , the nacos.properties can be omitted.

  Start dashboard
  java -jar -Dproject.name=sentinel-dashboard -Dcsp.sentinel.dashboard.server=localhost:8080
-Drule.repository.type=nacos sentinel-dashboard.jar

  3.Test with dashboard application itself, test CRUD functions for rules of different type in
  dashboard app, and view the rules in Nacos web console.

  4.Test with demo project, start sentinel-demo-annotation-spring-aop application with two different instances, test CRUD functions for rules of different type via application or single machine.

• Test in IDE environment

• Run test cases

Special notes for reviews

Now persistence for five types of rules have been completed(flow/degrade/system/authority/paramFlow), persistence for the gateway rule and cluster have not implement yet.

With the design of sentinel-dashboard-repository and sentinel-dashboard-service, I find it easy to implement specific repository except sentinel-dashboard-repository-memory, since it needs to communicate with sentinel-transport http api. For simplicity, I add id/app/ip/port fields in AbstractRule , but this class is in core module. Is there any better way, please give some help.

A new module sentinel-transport-client is added, in order to implement sentinel-dashboard-repository-memory, I prefer put the client module in transport module. In this way, it is an independent module, and easy to use and expand.

Pay attention to the id field of RuleEntity, previous version use AtomicLong ids and increment one when adding a new rule. This may have some problems when dashboard restart and memory data lost. To solve this I add IdGenerator interface and DefaultIdGenerator class, to generate unique and self increasing id for a new rule.

[codecov-commenter]

Codecov Report

Merging #1453 into master will increase coverage by 0.39%.
The diff coverage is 87.03%.

@@             Coverage Diff              @@
##             master    #1453      +/-   ##
============================================
+ Coverage     43.96%   44.36%   +0.39%     
- Complexity     1719     1750      +31     
============================================
  Files           376      376              
  Lines         10660    10752      +92     
  Branches       1418     1426       +8     
============================================
+ Hits           4687     4770      +83     
- Misses         5408     5414       +6     
- Partials        565      568       +3     

Impacted Files	Coverage Δ	Complexity Δ
...a/csp/sentinel/slots/system/SystemRuleManager.java	70.39% <83.33%> (+5.39%)	21.00 <2.00> (+2.00)
...alibaba/csp/sentinel/slots/block/AbstractRule.java	70.73% <100.00%> (+12.11%)	22.00 <8.00> (+8.00)
...nel/adapter/dubbo/SentinelDubboProviderFilter.java	62.06% <0.00%> (-21.27%)	4.00% <0.00%> (+2.00%)	⬇️
...csp/sentinel/adapter/dubbo/config/DubboConfig.java	66.66% <0.00%> (-13.34%)	8.00% <0.00%> (-1.00%)
...nel/adapter/dubbo/SentinelDubboConsumerFilter.java	89.28% <0.00%> (-5.96%)	12.00% <0.00%> (+9.00%)	⬇️
...aba/csp/sentinel/adapter/servlet/CommonFilter.java	87.80% <0.00%> (-2.44%)	11.00% <0.00%> (-1.00%)
...l/cluster/server/connection/ConnectionManager.java	74.00% <0.00%> (-2.00%)	10.00% <0.00%> (-1.00%)
...ource/spring/cloud/config/SentinelRuleLocator.java	36.71% <0.00%> (-0.98%)	5.00% <0.00%> (ø%)
...alibaba/csp/sentinel/adapter/dubbo/DubboUtils.java	96.15% <0.00%> (+0.32%)	11.00% <0.00%> (+2.00%)
...tinel/slots/block/flow/param/ParamFlowChecker.java	56.66% <0.00%> (+0.96%)	31.00% <0.00%> (+2.00%)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46076b3...b1668df. Read the comment docs.

[CLAassistant]

All committers have signed the CLA.

[kyiln]

请问 网关的现在实现了吗？是否是我的gateway服务加入了 -Dcsp.sentinel.app.type=1 ，成为一个网关的服务，那么他的操作是不具有持久化的

[kyiln]

你好。我在使用过的过程中，你描述的 五大规则的确可以。非常棒。但是现在我想同时在网关层（gateway）做流控，会抛出异常：CommandNotFoundException: /gateway/getApiDefinitions
。增加了五大规则的持久化，不应该影响网关使用异常

[cdfive]

加入了 -Dcsp.sentinel.app.type=1 ，成为一个网关的服务
抛出异常：CommandNotFoundException: /gateway/getApiDefinitions

记得没有修改网关规则处理，理论上不影响使用，只是规则是存在内存里的。
这个异常提示是缺少相关command的处理包，看看应用端是否有加sentinel-gateway-adapter相关的依赖，比如使用SpringCloudGateway的sentinel-spring-cloud-gateway-adapter, Zuul1的sentinel-zuul-adapter, Zuul2的sentinel-zuul2-adapter。网关command的处理是在它们的父依赖包sentinel-api-gateway-adapter-common下。

网关规则的持久化暂时还没实现，因为跟其它规则有些不同，有API分组和网关流控规则，还不太确定之前的抽象能否满足，还有集群流控规则也还没实现，社区看看一起讨论设计下哈，如果项目中有使用问题请多反馈和讨论。

[sczyh30]

Could you please resolve conflicts?

[cdfive]

@sczyh30 Sorry for late. Merged from master and the conflicts have been resolved, please check.

[codecov-io]

Codecov Report

Merging #1453 into master will decrease coverage by 0.22%.
The diff coverage is 87.03%.

@@             Coverage Diff              @@
##             master    #1453      +/-   ##
============================================
- Coverage     44.58%   44.36%   -0.23%     
+ Complexity     1804     1750      -54     
============================================
  Files           382      376       -6     
  Lines         10912    10752     -160     
  Branches       1467     1426      -41     
============================================
- Hits           4865     4770      -95     
+ Misses         5451     5414      -37     
+ Partials        596      568      -28     

Impacted Files	Coverage Δ	Complexity Δ
...a/csp/sentinel/slots/system/SystemRuleManager.java	70.39% <83.33%> (+5.39%)	21.00 <2.00> (+2.00)
...alibaba/csp/sentinel/slots/block/AbstractRule.java	70.73% <100.00%> (+12.11%)	22.00 <8.00> (+8.00)
...ource/spring/cloud/config/SentinelRuleLocator.java	36.71% <0.00%> (-0.98%)	5.00% <0.00%> (ø%)
...nterceptor/AbstractSentinelInterceptorSupport.java
...n/cdi/interceptor/SentinelResourceInterceptor.java
...ployment/SentinelJaxRsQuarkusAdapterProcessor.java
...tion/cdi/interceptor/ResourceMetadataRegistry.java
...inel/annotation/cdi/interceptor/MethodWrapper.java
...ent/SentinelAnnotationQuarkusAdapterProcessor.java
...a/com/alibaba/csp/sentinel/node/StatisticNode.java	51.11% <0.00%> (+1.11%)	26.00% <0.00%> (+1.00%)
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b91305a...058eb26. Read the comment docs.

[huyoufu]

ParamFlowRuleEntity 这个类中代码
public ParamFlowRuleEntity(ParamFlowRule rule) {
AssertUtil.notNull(rule, "Authority rule should not be null");
this.rule = rule;
}
是不是不小心写错单词了

[cdfive]

ParamFlowRuleEntity(ParamFlowRule rule)

确实写错了，该段代码在PR已注释掉没有使用，上面的ParamFlowRuleEntity fromAuthorityRule(String app, String ip, Integer port, ParamFlowRule rule)也是拼写错误。
@huyoufu 感谢指出问题，找时间我去把这里修正完善

[cdfive]

Master code has been merged and conflicts fixed, PTAL. @sczyh30 @jasonjoo2010 @linlinisme

[sczyh30]

Also cc @LearningGp