Skip to content

Commit

Permalink
Implemented PR suggestion
Browse files Browse the repository at this point in the history
  • Loading branch information
dlmarion committed Oct 2, 2024
1 parent 40c6217 commit d2b9591
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion _posts/blog/2024-08-30-accumulo4-preview.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ author: Dave Marion

## Background

In version 2.1, we introduced two new optional and experimental features, [External Compactions](https://accumulo.apache.org/blog/2021/07/08/external-compactions.html) and [ScanServers](https://github.com/apache/accumulo/pull/2665). The ExternalCompactions feature included two new server processes, the CompactionCoordinator and the Compactor. Using these new processes and their related configurations allows the user to perform major compactions on Tablets external to the TabletServer process. The configuration in 2.1 allows the user to define different “queues” for the external compactions and to assign a “queue” to the Compactor process when it’s started. This provides the user with some capability to define the resources for different classes of compactions (see the referenced blog post for examples). ExternalCompactions may provide lower latency for major compactions because major compactions that run in the TabletServer may queue up with all of the major compaction threads are busy.
In version 2.1, we introduced two new optional and experimental features, [External Compactions](https://accumulo.apache.org/blog/2021/07/08/external-compactions.html) and [ScanServers](https://github.com/apache/accumulo/pull/2665). The ExternalCompactions feature included two new server processes, the CompactionCoordinator and the Compactor. Using these new processes and their related configurations allows the user to perform major compactions on Tablets external to the TabletServer process. The configuration in 2.1 allows the user to define different “queues” for the external compactions and to assign a “queue” to the Compactor process when it’s started. This provides the user with some capability to define the resources for different classes of compactions (see the referenced blog post for examples). ExternalCompactions may provide lower latency for major compactions because major compactions that run in the TabletServer may queue up when all of the major compaction threads are busy.

The ScanServers feature included one new server process, the ScanServer, which allows users to execute scans against a Tablet external to the TabletServer. Because the ScanServer does not have access to the in-memory mutations within the TabletServer, we introduced a consistency level setting on the Scanner and BatchScanner where scans with the “immediate” consistency setting (default) would be sent to the TabletServer only and scans with the “eventual” consistency setting would be sent to a ScanServer. ScanServers can provide better allocation of resources against the current workload because many ScanServers can be used to scan the same Tablet, and a single ScanServer can be used to scan different versions of the Tablet. Immediate consistency scans are sent to the hosting TabletServer where they could possibly queue up, where eventual consistency scans can be serviced by many ScanServers at the cost of not seeing the most recent data (this time delta is configurable). ScanServer processes can be started with a group name which can be used in the client configuration such that eventual scans of a particular type can be sent to a specific group of ScanServer processes.

Expand Down

0 comments on commit d2b9591

Please sign in to comment.