v0.21.0-beta8

What's Changed

• cherry-pick #7146 fix: blueprint gets disabled when disabling dora for project by @github-actions in #7147
• cherry-pick #7144 fix: missed plugin opsgenie doc by @github-actions in #7145
• cherry-pick #7148 fix: unable to trigger blueprints with webhooks only by @github-actions in #7149
• cherry-pick #7161 fix: jira epic add time after by @github-actions in #7162
• cherry-pick #7163 fix: some jira test connection error by @github-actions in #7164
• cherry-pick #7165 fix: jira server test connection bug by @github-actions in #7166
• cherry-pick #7169 fix: limit layout content width to 1280 by @github-actions in #7170
• Morelog by @d4x1 in #7173
• cherry-pick #7175 fix(zentao): update error message when testing connection fails by @github-actions in #7176

Full Changelog: v0.21.0-beta7...v0.21.0-beta8

v0.21.0-beta7

What's Changed

• fix(ui): Add /health url with basicAuth disabled (#7009) by @sayeedhussain in #7038
• cherry-pick #7039 fix(zentao): fix PatchConnection by @github-actions in #7040
• cherry-pick #7032 fix updated_at field in cicd_tasks and cicd_pipelines by @github-actions in #7041
• cherry-pick #7042 fix(tapd): sanitize connection's resp by @github-actions in #7043
• cherry-pick #7033 fix: cicd deployment dora delete when data nil by @github-actions in #7044
• cherry-pick #7045 fix(security): fix cwe-276, (parts of)cwe-22 by @github-actions in #7047
• cherry-pick #7051 fix(zentao): fix parsing errors for date related fields on zentao v18.10 to v0.21 by @d4x1 in #7055
• cherry-pick #7030 feat: add docs about v0.21 by @github-actions in #7057
• chore: adjust the doc link to v0.21 by @mintsweet in #7058
• fix: pr cycle time doesn't respect time interval and is not null replace of != null by @abeizn in #7056
• cherry pick #7060 fix(azuredevops): fix environment field in cicd_tasks and cicd_pipelines to v0.21 by @d4x1 in #7061
• cherry-pick #7062 fix(azuredevops): remove unneccessary collectors when re-transformati… by @github-actions in #7063
• cherry-pick #7064 fix(azuredevops): fix environment field by @github-actions in #7065
• cherry-pick #7068 Fetch AzureDevops Builds by finishTimeDescending by @github-actions in #7069
• cherry-pick #7067 fix: jenkins should be using fullName instead of scopeId when generating plan by @github-actions in #7071
• cherry-pick #7070 fix: empty dora data when run the tasks by @github-actions in #7072
• cherry-pick #7066 fix: jenkins task name error by @github-actions in #7077
• cherry-pick #7080 feat(plugins): add WRAP_RESPONSE_ERROR in config to avoid SSRF when testing connections by @github-actions in #7081
• cherry-pick #7085 fix: show badge in connection name by @github-actions in #7087
• cherry-pick #7095 fix: gitlab delate data by raw data table by @github-actions in #7102
• cherry-pick #7093 fix: sonarqube connection delete on no scope config by @github-actions in #7103
• cherry-pick #7104 chore: upgrade miller-columns-select to 1.3.1 by @mintsweet in #7109
• cherry-pick #7106 fix: modify deployable_commit_title from varchar255 to text by @github-actions in #7110
• cherry-pick #7108 fix(azuredevops): fix some bugs by @github-actions in #7111
• cherry-pick #7112 fix: sonarqube test connection can not effective by @github-actions in #7113
• cherry-pick #7119 fix: cronjob won't work for a new instance by @github-actions in #7120
• cherry-pick #7122 fix(dora): remove calculated started_date when converting pipelines to deployments by @github-actions in #7123
• cherry-pick #7125 chore: upgrade miller-columns-select to 1.3.2 by @mintsweet in #7126
• cherry-pick #7130 fix(dora): add started_date when generating cicd_deployment_commits by @github-actions in #7131
• cherry-pick #7127 fix: missed path prefix by @github-actions in #7133
• cherry-pick #7134 fix: project path error by @github-actions in #7136
• cherry pick #7137 fix(plugins): fix wrong resp when testing connections, fix a typo in bitbucket to v0.21. by @d4x1 in #7138
• cherry-pick(customize): issue_repo_commit.host column ignores port, consistan… by @leric in #7140
• [cp][customize] Import csv issues API handle issues in multiple boar… by @leric in #7142
• fix: add flag for data convert to dora #7143

Full Changelog: v0.21.0-beta6...v0.21.0-beta7

v0.20.0

New Features

• [Website] Add the full-text search to the website: #605
• [Data Schema] Add deployment table in domain layer: #5475
• [Data Schema] Update transformation rules for status and result fields in tables cicd_tasks \ cicd_pipelines \ cicd_deployment_commits : #6398
• [Config UI] Add transform-box-styled UI to "Select Data Scope": #5640
• [Config UI] Add the Pipeline list back: #3673
• [Jira] Support extracting array data from Jira's raw layer: #5868
• [Jira] Search boards for adding data scope: #5089
• [GitHub/GitLab] Collect deployments from GitLab/GitHub: #5949
• [GitLab] Allow selecting GitLab projects owned by others that have no groups: #5914
• [GitExtractor] Make commit files collection optional: #5819

Bug Fixes

• [GitHub] Support using the branch_name of a GitHub run to indicate the environment: #6128
• [GitHub] Settings for GitHub App configuration: #5723
• [Azure Devops] Fix 400 error when cloning Azure repos#6246
• [Azure Devops] Add missing fields in _tool_azuredevops_gitrepositoryconfigs and _tool_azuredevops_gitrepositories : #6240
• [Bamboo] Fix cicd_deployment_commits.id, remove PlanResultKey from the primary key list from BambooDeployBuild#6375
• [Bamboo] Keep plan name the same in both /search-remote-scopes and /remote-scopes: #6284
• [BitBucket] Fix wrong create_date in pull_request_comments: #6367
• [BitBucket] normalise cicd_deployment_commits.environment field: #6333
• [BitBucket] Add build number to pipeline's name: #6330
• [BitBucket] lots of fixes about result\status and pipeline's name fields : #6321
• [BitBucket] Make cicd_tasks.type empty#6297
• [TAPD] Fix JSON unmarshal error when decoding bug and story status : #6396 #6379
• [TAPD] Update workspace's id and parent_id's type: #6239
• [PagerDuty] Search remote scope by keyword: #6255
• [Zentao] Fix type conversion when setting scopes: #6286
• [Gitextractor] Update close function: #6306
• [Config UI] Fix the enableGraphql option in the GitHub connection: #6391
• [Framework] Remove deprecated position fields in table pull_request_comments: #6393

Refactors

• [Framwork] Break up Blueprint config in the database : #5360
• [Framwork] Adjust the API about scope: #6198
• [Framework] Remove Temporal support: #6176
• [Framework] Convert bp.settings into multiple tables: #6006
• [Gitextractor]: Move "clone a repo" from "PrePareTaskData" to a new subtask called "CloneGitRepo": #5992
• [Developer] Deprecate plugin code generator: #5922
• [Tap] Remove all Singer Spec / tap related tables and code: #5923
• [Framework] Unify Field types for JSON : #3729

Breaking Changes

• [Framework] Allow users to configure the sync modes for plugins that support incremental sync: #5841
• [Framework] Add API Keys Management: #5791 (Webhooks will be effected, see #6472)
• [Framwork] Unify connection/scope/scopeConfig helpers : #5519

How to Upgrade

Please follow the steps below to upgrade Apache Incubator DevLake:

For Docker Compose and Kubernetes (k8s) Users:

1. Update your docker-compose.yml or k8s-deploy.yaml file with the following image tags:

• apache/devlake:v0.20.0-beta1
• apache/devlake-dashboard:v0.20.0-beta1
• apache/devlake-config-ui:v0.20.0-beta1

2. Set the value for "ENCRYPTION_SECRET". Please refer to the documentation at Docker Compose Setup for detailed instructions based on your installation method.

For Helm Users:

1. Follow the instructions provided in the documentation at Helm Setup.
2. Ensure that you have set the value for "ENCRYPTION_SECRET".

Upgrade Notes:

For detailed upgrade instructions and further information, please refer to the documentation in Upgrade Guide.

v0.21.0-beta6

What's Changed

• cherry-pick #6993 refactor(middlewares): add CheckAuthorizationHeader by @github-actions in #6994
• cherry-pick #6995 refactor: optimize bp cronjob scheduling by @github-actions in #6996
• feat: updates backend to listen on both IPv4 and IPv6 addresses by @wanisfahmyDE in #6992
• cherry pick #7001 fix(azuredevops): fix some fields' value to v0.21 by @d4x1 in #7003
• cherry-pick #7004 refactor(azuredevops): update mysqlclien to v2.2.4, make it compatibl… by @github-actions in #7012
• cherry-pick #7013 [AzureDevops ] fix: some fields' values by @github-actions in #7015
• cherry-pick #7016 [jira test connection api]fix: invalid url detection not working correctly by @github-actions in #7017
• cherry-pick #7018 fix(gitextractor): cover errors that may leak sensitive information by @github-actions in #7019
• cherry-pick #7022 fix(azuredevops): fix unexpected environment field in cicd_tasks and … by @github-actions in #7023
• cherry-pick #7024 fix: unset scopeConfigId for scope not working on azdo/bamboo by @github-actions in #7025
• cp: cherrypick PR #6965 to release-0.21 by @leric in #7028
• cherry-pick #7011 feat: missed circleci transformation (#7011) by @mintsweet in #7029
• cherry-pick #7031 fix: jira test connection error message for wrong user/pass not working by @github-actions in #7034
• fix: cfr dashboard value by @abeizn in #7035
• cherry-pick #7036 fix(dockerfile): try to fix build errors by @github-actions in #7037

Full Changelog: v0.21.0-beta5...v0.21.0-beta6

v0.21.0-beta5

What's Changed

• feat: extends DNS variable assignment to additionally handle IPv6 (#6976) by @wanisfahmyDE in #6979
• cherry-pick #6986 fix(zentao): update the type of PriOrder by @github-actions in #6987

Full Changelog: v0.21.0-beta4...v0.21.0-beta5

v0.21.0-beta4

What's Changed

• chore: adds support for IPv6 in nginx (#6955) by @wanisfahmyDE in #6960
• cherry-pick #6966 fix: /grafana should be off by @github-actions in #6967
• cherry-pick #6970 fix(migration): fix rename operations in tapd plugin by @github-actions in #6972

Full Changelog: v0.21.0-beta3...v0.21.0-beta4

v0.21.0-beta3

What's Changed

• cherry-pick #6945 fix: update cfr metric by @github-actions in #6946
• cherry pick #6947 feat(dashboard): revert changes that try to remove wanings to v0.21 by @d4x1 in #6948

Full Changelog: v0.21.0-beta2...v0.21.0-beta3

v0.21.0-beta2

What's Changed

• cherry-pick #6873 fix: github deployment status active and inactive should be treated as success by @github-actions in #6874
• cherry-pick #6876 fix: github run api page<35 limit by @github-actions in #6878
• cherry-pick #6879 feat: support hide custom type toast in operator by @github-actions in #6880
• cherry-pick #6882 fix: some bugs by @github-actions in #6883
• cherry-pick #6886 fix: adjust the style for page header by @github-actions in #6887
• cherry-pick #6895 fix: the layout error by @github-actions in #6896
• cherry-pick #6897 fix: icon missed in data scope select by @github-actions in #6898
• cherry-pick #6901 fix: triggering blueprint concurrently might lead to deadlock by @github-actions in #6904
• cherry-pick #6902 cherry-pick #6901 fix: triggering blueprint concurrently might lead to deadlock by @github-actions in #6903
• cherry-pick #6900 fix: asfheader format by @github-actions in #6906
• cherry-pick #6909 fix: adjust the layout for page by @github-actions in #6910
• cherry-pick #6913 fix: adjust the margin for page layout by @github-actions in #6914
• fix: copywriting by @mintsweet in #6917
• cherry pick #6881 #6871 update dashboards, remove warnings to release v0.21 by @d4x1 in #6919
• cherry-pick #6893 feat: upgrade dora benchmarks to 2023 version by @github-actions in #6920
• cherry-pick #6922 chore: upgrade antd to 5.14.0 by @github-actions in #6924
• cherry-pick #6925 fix: default value error in jira transformation by @github-actions in #6926
• cherry-pick #6928 fix: missed redirect path prefix in connection detail by @github-actions in #6929
• cherry-pick #6932 fix: rerun task deadlock by @github-actions in #6933
• cherry-pick #6935 fix: new dshelper does not set raw data origin properly by @github-actions in #6936
• cherry-pick #6937 feat: add default name for scope config by @github-actions in #6938
• cherry-pick #6939 fix: rerun pipeline deadlock by @github-actions in #6940
• cherry-pick #6931 fix: deployment dora dashboard by @github-actions in #6942

Full Changelog: v0.21.0-beta1...v0.21.0-beta2

v0.20.0-beta10

What's Changed

• cherry-pick #6873 fix: github deployment status active and inactive should be treated as success to v0.20 by @klesh in #6875
• cherry-pick #6876 fix: github run api page<35 limit by @github-actions in #6877

Full Changelog: v0.20.0-beta9...v0.20.0-beta10

v0.21.0-beta1

Breaking Changes

• fix: change the type of duration_sec from int to float #6465 - d4x1
• refactor: gitlab adopts new remote api helper #6546 - klesh
• fix: trigger bp api should reject disabled bps #6706 - klesh

New Features

• feat(config-ui): support theme setting #6446 - mintsweet
• feat(config-ui): simplify the display of the github scope config #6480 - mintsweet
• feat(schema): add original_status and original_result fields to all related devops tables #6476 - d4x1
• feat: add skip collector at advanced mode #6454 - abeizn
• feat(config-ui): improve connection management usability #6519 - mintsweet
• feat(config-ui): support new plugin circleci #6537 - mintsweet
• feat(bamboo): return simple error message when license has expired #6511 - d4x1
• feat(config-ui): dis-associate the scope config with a data scope #6527 - mintsweet
• feat(api): add sanitizer to remove sensitive fields #6538 - d4x1
• feat(zentao): check db url when testing connections #6258 - d4x1
• feat(config-ui): not returning the tokens or passwords on APIs #6611 - mintsweet
• feat: allow deployment webhook to push deployments to multiple repos … #6535 - abeizn
• feat(db): add date relation fields to devops tables #6499 - d4x1
• feat(plugins): secret key and token are not empty now #6704 - d4x1
• feat: Opsgenie plugin implementation #6154 - sandesvitor
• feat: add filter jql checker #6789 - abeizn
• feat(pagerduty): search remote scope by keyword #6255 - d4x1
• feat(gitextractor): add progress info when cloning git repos #6195 - d4x1
• feat: add fullsync flag to DirectRun #6819 - klesh
• feat: forbid starting tx via the dalgorm.Exec #6685 - klesh
• feat(plugins): support partical partial update when editing connections #6790 - d4x1
• feat(config-ui): add new page not found #6811 - mintsweet

Bug Fixes

• fix(config-ui): show error when initializing connection error #6471 - mintsweet
• fix(webhook): add migration script when webhooks have no API keys #6472 - d4x1
• fix: Version API response in coming as empty string in v0.20.0-beta1 #6474 - klesh
• fix: upgrade grafana to fix some known vulnerabilities #6459 - klesh
• fix: jenkins stages durationMillis<0 #6462 - abeizn
• fix: modify _tool_jira_issue_relationships primary key #6253 - abeizn
• fix(db): add missing primary keys #6505 - d4x1
• fix(gitlab): fix incremental deployment collector #6254 - d4x1
• fix: collect all data when timeafter is nil #6663 - abeizn
• fix: make some plugin time filed default null #6813 - abeizn
• fix: gitlab Deployable.Commit.Message can be bigger than 255 #6588 - abeizn
• fix: github issue data too long for column type #6560 - abeizn
• fix: update certifi version to address CVE-2023-37920 #6509 - antoinecaputo
• fix: panic during github_graphql task when grabbing deployments #6513 - abeizn
• fix(db): upgrade gorm to v1.25.5 #6633 - d4x1
• fix: github grapahql increment collector #6601 - abeizn
• fix: option time after for plugins #6585 - abeizn
• fix: gitextractor can not run #6660 - abeizn
• fix cert update issue #6622 - ZhangNing10
• fix: convert rules index out of range #6670 - abeizn
• fix(zentao): fix panic when checking db connections #6649 - d4x1
• fix: duration_sec has wrong value #6688 - abeizn
• fix: add collect sonarqube mode by file #6507 - abeizn
• fix: split the collector into collector+extractor #6564 - abeizn
• fix: zentao collect bug with priOrder #6776 - matrixji
• fix: add jira component #6725 - abeizn
• fix: type trans rules #6679 - abeizn
• fix(api): proxy api pass through x-headers #6788 - leric
• fix: gitLab collection failed with error message Incorrect datetime v… #6769 - abeizn
• fix: gitlab incidents' type is nil #6732 - abeizn
• fix: issue created on Sunday are not being considered #6682 - abeizn
• fix: wrong value in table dora_benchmarks #6729 - Startrekzky
• fix: engineering overview work done dashboard #6742 - abeizn
• fix: gitlab incident #6748 - abeizn
• fix: mysql busy buffer, fails to fully parse all all pull request met… #6781 - abeizn
• fix: SonarQube dashboard #6739 - Startrekzky
• fix: dora benchmark metric value #6737 - abeizn
• fix: syntax error near unexpected token & #6458 - abeizn
• fix(pipeline): fix potiential panic #6484 - d4x1
• fix(config-ui): missed key of header links #6504 - mintsweet
• fix(bamboo): update tool layer table _tool_bamboo_plan_build_commits's schema #6488 - d4x1
• fix(config-ui): adjust the style for layout #6506 - mintsweet
• fix(config-ui): adjust the zentao search to local #6817 - mintsweet
• fix(config-ui): missed style for github check icon #6821 - mintsweet
• fix(azure): fix 400 error when cloning azure repos #6246 - d4x1
• fix(configui): tagsLimit should be a number #6261 - d4x1
• fix(config-ui): zentao error local search #6816 - mintsweet
• fix(config-ui): adjust doc link to stable #6517 - mintsweet
• fix: github token expired may cause 500 error #6561 - klesh
• fix: incorrect datetime value: '0000-00-00' for column latest_updated… #6541 - abeizn
• fix(config-ui): add label UTC to sync frequency #6525 - mintsweet
• fix(fetch): fix wrong use of db.Fetch #6666 - d4x1
• fix(config-ui): github token is valid condition error #6823 - mintsweet
• fix: incorrect datetime value: '0000-00-00' for column latest_updated #6587 - abeizn
• fix: bp trigger swag #6657 - abeizn
• [fix(zentao): set correct value for 'nextPageToke...