Merge remote-tracking branch 'upstream/master'

.github/pull_request_template.md

@@ -27,9 +27,7 @@ For details, see [tips for choosing the affected versions](https://github.com/pi
 - [ ] v7.5 (TiDB 7.5 versions)
 - [ ] v7.4 (TiDB 7.4 versions)
 - [ ] v7.3 (TiDB 7.3 versions)
-- [ ] v7.2 (TiDB 7.2 versions)
 - [ ] v7.1 (TiDB 7.1 versions)
-- [ ] v7.0 (TiDB 7.0 versions)
 - [ ] v6.5 (TiDB 6.5 versions)
 - [ ] v6.1 (TiDB 6.1 versions)
 - [ ] v5.4 (TiDB 5.4 versions)

.lycheeignore

@@ -13,4 +13,5 @@ file://.*https:/%7BnodeIP%7D:%7BnodePort%7D/dashboard
 file://.*?http:/\$%7BPD_IP%7D:\$%7BPD_PORT%7D/dashboard.*
 http://\{grafana-ip\}:3000
 http://\{pd-ip\}:2379/dashboard
-http://localhost:\d+/
+http://localhost:\d+/
+https://github\.com/\$user/(docs|docs-cn)

TOC.md

@@ -556,6 +556,7 @@
       - [DDL Replication](/ticdc/ticdc-ddl.md)
       - [Bidirectional Replication](/ticdc/ticdc-bidirectional-replication.md)
       - [Data Integrity Validation for Single-Row Data](/ticdc/ticdc-integrity-check.md)
+      - [Data Consistency Validation for TiDB Upstream/Downstream Clusters](/ticdc/ticdc-upstream-downstream-check.md)
     - Monitor and Alert
       - [Monitoring Metrics Summary](/ticdc/ticdc-summary-monitor.md)
       - [Monitoring Metrics Details](/ticdc/monitor-ticdc.md)
@@ -609,7 +610,6 @@
     - [Overview](/sync-diff-inspector/sync-diff-inspector-overview.md)
     - [Data Check for Tables with Different Schema/Table Names](/sync-diff-inspector/route-diff.md)
     - [Data Check in the Sharding Scenario](/sync-diff-inspector/shard-diff.md)
-    - [Data Check for TiDB Upstream/Downstream Clusters](/sync-diff-inspector/upstream-downstream-diff.md)
     - [Data Check in the DM Replication Scenario](/sync-diff-inspector/dm-diff.md)
 - Reference
   - Cluster Architecture

deploy-monitoring-services.md

@@ -226,7 +226,7 @@ To import a Grafana dashboard for the PD server, the TiKV server, and the TiDB s
 
 2. In the sidebar menu, click **Dashboards** -> **Import** to open the **Import Dashboard** window.
 
-3. Click **Upload .json File** to upload a JSON file (Download TiDB Grafana configuration files from [pingcap/tidb](https://github.com/pingcap/tidb/tree/master/metrics/grafana), [tikv/tikv](https://github.com/tikv/tikv/tree/master/metrics/grafana), and [tikv/pd](https://github.com/tikv/pd/tree/master/metrics/grafana)).
+3. Click **Upload .json File** to upload a JSON file (Download TiDB Grafana configuration files from [pingcap/tidb](https://github.com/pingcap/tidb/tree/master/pkg/metrics/grafana), [tikv/tikv](https://github.com/tikv/tikv/tree/master/metrics/grafana), and [tikv/pd](https://github.com/tikv/pd/tree/master/metrics/grafana)).
 
     > **Note:**
     >

functions-and-operators/precision-math.md

@@ -51,7 +51,7 @@ DECIMAL columns do not store a leading `+` character or `-` character or leading
 
 DECIMAL columns do not permit values larger than the range implied by the column definition. For example, a `DECIMAL(3,0)` column supports a range of `-999` to `999`. A `DECIMAL(M,D)` column permits at most `M - D` digits to the left of the decimal point.
 
-For more information about the internal format of the DECIMAL values, see [`mydecimal.go`](https://github.com/pingcap/tidb/blob/master/types/mydecimal.go) in TiDB souce code.
+For more information about the internal format of the DECIMAL values, see [`mydecimal.go`](https://github.com/pingcap/tidb/blob/master/pkg/types/mydecimal.go) in TiDB souce code.
 
 ## Expression handling
 

glossary.md

@@ -70,6 +70,12 @@ The in-memory pessimistic lock is a new feature introduced in TiDB v6.0.0. When
 
 Leader/Follower/Learner each corresponds to a role in a Raft group of [peers](#regionpeerraft-group). The leader services all client requests and replicates data to the followers. If the group leader fails, one of the followers will be elected as the new leader. Learners are non-voting followers that only serves in the process of replica addition.
 
+## M
+
+### MPP
+
+Starting from v5.0, TiDB introduces Massively Parallel Processing (MPP) architecture through TiFlash nodes, which shares the execution workloads of large join queries among TiFlash nodes. When the MPP mode is enabled, TiDB, based on cost, determines whether to use the MPP framework to perform the calculation. In the MPP mode, the join keys are redistributed through the Exchange operation while being calculated, which distributes the calculation pressure to each TiFlash node and speeds up the calculation. For more information, see [Use TiFlash MPP Mode](/tiflash/use-tiflash-mpp-mode.md).
+
 ## O
 
 ### Old value

hybrid-deployment-topology.md

@@ -23,7 +23,7 @@ The deployment machine has multiple CPU processors with sufficient memory. To im
 
 ### Topology templates
 
-- [The simple template for the hybrid deployment](https://github.com/pingcap/docs-cn/blob/master/config-templates/simple-multi-instance.yaml)
+- [The simple template for the hybrid deployment](https://github.com/pingcap/docs/blob/master/config-templates/simple-multi-instance.yaml)
 - [The complex template for the hybrid deployment](https://github.com/pingcap/docs/blob/master/config-templates/complex-multi-instance.yaml)
 
 For detailed descriptions of the configuration items in the above TiDB cluster topology file, see [Topology Configuration File for Deploying TiDB Using TiUP](/tiup/tiup-cluster-topology-reference.md).

maintain-tidb-using-tiup.md

@@ -113,7 +113,7 @@ When the cluster is in operation, if you need to modify the parameters of a comp
 
     **Use `.` to represent the hierarchy of the configuration items**.
 
-    For more information on the configuration parameters of components, refer to [TiDB `config.toml.example`](https://github.com/pingcap/tidb/blob/master/config/config.toml.example), [TiKV `config.toml.example`](https://github.com/tikv/tikv/blob/master/etc/config-template.toml), and [PD `config.toml.example`](https://github.com/tikv/pd/blob/master/conf/config.toml).
+    For more information on the configuration parameters of components, refer to [TiDB `config.toml.example`](https://github.com/pingcap/tidb/blob/master/pkg/config/config.toml.example), [TiKV `config.toml.example`](https://github.com/tikv/tikv/blob/master/etc/config-template.toml), and [PD `config.toml.example`](https://github.com/tikv/pd/blob/master/conf/config.toml).
 
 3. Rolling update the configuration and restart the corresponding components by running the `reload` command:
 
@@ -125,7 +125,7 @@ When the cluster is in operation, if you need to modify the parameters of a comp
 
 ### Example
 
-If you want to set the transaction size limit parameter (`txn-total-size-limit` in the [performance](https://github.com/pingcap/tidb/blob/master/config/config.toml.example) module) to `1G` in tidb-server, edit the configuration as follows:
+If you want to set the transaction size limit parameter (`txn-total-size-limit` in the [performance](https://github.com/pingcap/tidb/blob/master/pkg/config/config.toml.example) module) to `1G` in tidb-server, edit the configuration as follows:
 
 ```
 server_configs:

performance-tuning-methods.md

@@ -489,6 +489,6 @@ For the `Store` thread, `Commit Log Duration` is obviously higher than `Apply Lo
 
 ## If my TiDB version is earlier than v6.1.0, what should I do to use the Performance Overview dashboard?
 
-Starting from v6.1.0, Grafana has a built-in Performance Overview dashboard by default. This dashboard is compatible with TiDB v4.x and v5.x versions. If your TiDB is earlier than v6.1.0, you need to manually import [`performance_overview.json`](https://github.com/pingcap/tidb/blob/master/metrics/grafana/performance_overview.json), as shown in the following figure:
+Starting from v6.1.0, Grafana has a built-in Performance Overview dashboard by default. This dashboard is compatible with TiDB v4.x and v5.x versions. If your TiDB is earlier than v6.1.0, you need to manually import [`performance_overview.json`](https://github.com/pingcap/tidb/blob/master/pkg/metrics/grafana/performance_overview.json), as shown in the following figure:
 
 ![Store](/media/performance/import_dashboard.png)

production-deployment-using-tiup.md

@@ -291,7 +291,7 @@ The following examples cover seven common scenarios. You need to modify the conf
 
 For more configuration description, see the following configuration examples:
 
-- [TiDB `config.toml.example`](https://github.com/pingcap/tidb/blob/master/config/config.toml.example)
+- [TiDB `config.toml.example`](https://github.com/pingcap/tidb/blob/master/pkg/config/config.toml.example)
 - [TiKV `config.toml.example`](https://github.com/tikv/tikv/blob/master/etc/config-template.toml)
 - [PD `config.toml.example`](https://github.com/pingcap/pd/blob/master/conf/config.toml)
 - [TiFlash `config.toml.example`](https://github.com/pingcap/tiflash/blob/master/etc/config-template.toml)

releases/release-6.3.0.md

@@ -198,7 +198,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows:
 
 * TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#6977](https://github.com/pingcap/tiflow/issues/6977) @[asddongmen](https://github.com/asddongmen)
 
-    In the scenarios of data replication for disaster recovery, TiCDC supports [periodically maintaining a downstream data snapshot](/sync-diff-inspector/upstream-downstream-diff.md#data-check-for-tidb-upstream-and-downstream-clusters) so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better support the scenarios where reads and writes are separate, and help you lower the cost.
+    In the scenarios of data replication for disaster recovery, TiCDC supports [periodically maintaining a downstream data snapshot](/ticdc/ticdc-upstream-downstream-check.md) so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better support the scenarios where reads and writes are separate, and help you lower the cost.
 
 * TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand)
 

releases/release-7.4.0.md

@@ -278,9 +278,13 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with-
 
 ### Behavior changes
 
-* Starting with v7.4.0, TiDB is compatible with core features of MySQL 8.0, and `version()` returns the version prefixed with `8.0.11`.
+- Starting with v7.4.0, TiDB is compatible with essential features of MySQL 8.0, and `version()` returns the version prefixed with `8.0.11`.
 
-* After TiFlash is upgraded to v7.4.0 from an earlier version, in-place downgrading to the original version is not supported. This is because, starting from v7.4, TiFlash optimizes the data compaction logic of PageStorage V3 to reduce the read and write amplification generated during data compaction, which leads to changes to some of the underlying storage file names.
+- After TiFlash is upgraded to v7.4.0 from an earlier version, in-place downgrading to the original version is not supported. This is because, starting from v7.4, TiFlash optimizes the data compaction logic of PageStorage V3 to reduce the read and write amplification generated during data compaction, which leads to changes to some of the underlying storage file names.
+
+- A [`TIDB_PARSE_TSO_LOGICAL()`](/functions-and-operators/tidb-functions.md#tidb-specific-functions) function is added to allow the extraction of the logical part of the TSO timestamp.
+
+- The [`information_schema.CHECK_CONSTRAINTS`](/information-schema/information-schema-check-constraints.md) table is added for improved compatibility with MySQL 8.0.
 
 ### System variables
 

schedule-replicas-by-topology-labels.md

@@ -131,7 +131,12 @@ The `location-level` configuration is an array of strings, which needs to corres
 
 When using TiUP to deploy a cluster, you can configure the TiKV location in the [initialization configuration file](/production-deployment-using-tiup.md#step-3-initialize-cluster-topology-file). TiUP will generate the corresponding configuration files for TiKV, PD, and TiFlash during deployment.
 
-In the following example, a two-layer topology of `zone/host` is defined. The TiKV nodes of the cluster are distributed among three zones, z1, z2, and z3, with each zone having four hosts, h1, h2, h3, and h4. In z1, four TiKV instances are deployed on two hosts, `tikv-1` and `tikv-2` on h1, and `tikv-3` and `tikv-4` on h2. Two TiFlash instances are deployed on the other two hosts, `tiflash-1` on h3 and `tiflash-2` on h4. In z2 and z3, two TiKV instances are deployed on two hosts, and two TiFlash instances are deployed on the other two hosts. In the following example, `tikv-n` represents the IP address of the `n`th TiKV node, and `tiflash-n` represents the IP address of the `n`th TiFlash node.
+In the following example, a two-layer topology of `zone/host` is defined. The TiKV nodes and TiFlash nodes of the cluster are distributed among three zones, z1, z2, and z3. 
+
+- In each zone, there are two hosts that have TiKV instances deployed. In z1, each host has two TiKV instances deployed. In z2 and z3, each host has a separate TiKV instance deployed. 
+- In each zone, there are two hosts that have TiFlash instances deployed, and each host has a separate TiFlash instance deployed. 
+
+In the following example, `tikv-host-machine-n` represents the IP address of the `n`th TiKV node, and `tiflash-host-machine-n` represents the IP address of the `n`th TiFlash node.
 
 ```
 server_configs:
@@ -140,86 +145,89 @@ server_configs:
 
 tikv_servers:
 # z1
-  - host: tikv-1
+  # machine-1 on z1
+  - host: tikv-host-machine-1
     port：20160
     config:
       server.labels:
         zone: z1
-        host: h1
-   - host: tikv-1
-     port：20161
+        host: tikv-host-machine-1
+  - host: tikv-host-machine-1
+    port：20161
     config:
       server.labels:
         zone: z1
-        host: h1
-  - host: tikv-2
+        host: tikv-host-machine-1
+  # machine-2 on z1
+  - host: tikv-host-machine-2
     port：20160
     config:
       server.labels:
         zone: z1
-        host: h2
-  - host: tikv-2
+        host: tikv-host-machine-2
+  - host: tikv-host-machine-2
     port：20161
     config:
       server.labels:
         zone: z1
-        host: h2
+        host: tikv-host-machine-2
 # z2
-  - host: tikv-5
+  - host: tikv-host-machine-3
     config:
       server.labels:
         zone: z2
-        host: h1
-   - host: tikv-6
+        host: tikv-host-machine-3
+  - host: tikv-host-machine-4
     config:
       server.labels:
         zone: z2
-        host: h2
+        host: tikv-host-machine-4
 # z3
-  - host: tikv-7
+  - host: tikv-host-machine-5
     config:
       server.labels:
         zone: z3
-        host: h1
-  - host: tikv-8
+        host: tikv-host-machine-5
+  - host: tikv-host-machine-6
     config:
       server.labels:
         zone: z3
-        host: h2s
+        host: tikv-host-machine-6
+
 tiflash_servers:
 # z1
-  - host: tiflash-1
+  - host: tiflash-host-machine-1
     learner_config:
       server.labels:
         zone: z1
-        host: h3
-   - host: tiflash-2
+        host: tiflash-host-machine-1
+  - host: tiflash-host-machine-2
     learner_config:
       server.labels:
         zone: z1
-        host: h4
+        host: tiflash-host-machine-2
 # z2
-  - host: tiflash-3
+  - host: tiflash-host-machine-3
     learner_config:
       server.labels:
         zone: z2
-        host: h3
-   - host: tiflash-4
+        host: tiflash-host-machine-3
+  - host: tiflash-host-machine-4
     learner_config:
       server.labels:
         zone: z2
-        host: h4
+        host: tiflash-host-machine-4
 # z3
-  - host: tiflash-5
+  - host: tiflash-host-machine-5
     learner_config:
       server.labels:
         zone: z3
-        host: h3
-  - host: tiflash-6
+        host: tiflash-host-machine-5
+  - host: tiflash-host-machine-6
     learner_config:
       server.labels:
         zone: z3
-        host: h4
+        host: tiflash-host-machine-6
 ```
 
 For details, see [Geo-distributed Deployment topology](/geo-distributed-deployment-topology.md).
@@ -234,13 +242,13 @@ PD schedules replicas according to the label layer to make sure that different r
 
 Take the topology in the previous section as an example.
 
-Assume that the number of cluster replicas is 3 (`max-replicas=3`). Because there are 3 zones in total, PD ensures that the 3 replicas of each Region are respectively placed in z1, z2, and z3. In this way, the TiDB cluster is still available when one data center fails.
+Assume that the number of cluster replicas is 3 (`max-replicas=3`). Because there are 3 zones in total, PD ensures that the 3 replicas of each Region are respectively placed in z1, z2, and z3. In this way, the TiDB cluster is still available when one zone fails.
 
 Then, assume that the number of cluster replicas is 5 (`max-replicas=5`). Because there are only 3 zones in total, PD cannot guarantee the isolation of each replica at the zone level. In this situation, the PD scheduler will ensure replica isolation at the host level. In other words, multiple replicas of a Region might be distributed in the same zone but not on the same host.
 
 In the case of the 5-replica configuration, if z3 fails or is isolated as a whole, and cannot be recovered after a period of time (controlled by `max-store-down-time`), PD will make up the 5 replicas through scheduling. At this time, only 4 hosts are available. This means that host-level isolation cannot be guaranteed and that multiple replicas might be scheduled to the same host. But if the `isolation-level` value is set to `zone` instead of being left empty, this specifies the minimum physical isolation requirements for Region replicas. That is to say, PD will ensure that replicas of the same Region are scattered among different zones. PD will not perform corresponding scheduling even if following this isolation restriction does not meet the requirement of `max-replicas` for multiple replicas.
 
-For example, a TiKV cluster is distributed across three data zones z1, z2, and z3. Each Region has three replicas as required, and PD distributes the three replicas of the same Region to these three data zones respectively. If a power outage occurs in z1 and cannot be recovered after a period of time (controlled by [`max-store-down-time`](/pd-configuration-file.md#max-store-down-time) and 30 minutes by default), PD determines that the Region replicas on z1 are no longer available. However, because `isolation-level` is set to `zone`, PD needs to strictly guarantee that different replicas of the same Region will not be scheduled on the same data zone. Because both z2 and z3 already have replicas, PD will not perform any scheduling under the minimum isolation level restriction of `isolation-level`, even if there are only two replicas at this moment.
+If the `isolation-level` setting is set to `zone`, this specifies the minimum isolation requirement for Region replicas at the physical level. In this case, PD will always guarantee that replicas of the same Region are distributed across different zones. Even if following this isolation restriction would not meet the multi-replica requirements of `max-replicas`, PD will not schedule accordingly. Taking a TiKV cluster distributed across three data zones (z1, z2, and z3) as an example, if each Region requires three replicas, PD distributes the three replicas of the same Region to these three data zones respectively. If a power outage occurs in z1 and cannot be recovered after a period of time (30 minutes by default, controlled by [`max-store-down-time`](/pd-configuration-file.md#max-store-down-time)), PD determines that the Region replicas in z1 are no longer available. However, because `isolation-level` is set to `zone`, PD needs to strictly guarantee that different replicas of the same Region will not be scheduled to the same data zone. Because both z2 and z3 already have replicas, PD will not perform any scheduling under the minimum isolation level restriction of `isolation-level`, even if there are only two replicas at this moment.
 
 Similarly, when `isolation-level` is set to `rack`, the minimum isolation level applies to different racks in the same data center. With this configuration, the isolation at the zone layer is guaranteed first if possible. When the isolation at the zone level cannot be guaranteed, PD tries to avoid scheduling different replicas to the same rack in the same zone. The scheduling works similarly when `isolation-level` is set to `host` where PD first guarantees the isolation level of rack, and then the level of host.