Apache Helix 2.0 Overview

Apache Helix has been stable for a long time, but there are several areas where we can make significant improvements. The proposal of new generation of Apache Helix with the following major goals:

1. Multi-language support
2. Simpler API and user experience
3. Improved data model, communication protocol, and system performance
4. More pluggabilities

To achieve these goals, I have outlined the following projects:

• Multi-language support: Develop a gateway service that accepts general state transition calls through gRPC.

• API simplification: Minimize both client and Admin APIs to streamline the process.

• Simplified data model and communication protocol: Instead of defining a fine-grained state model, provide users with the final state and allow the application to progressively update its own partition states. This will minimize round trips between the Helix controller and participants and reduce traffic to the storage (Zookeeper).

• Components "micro-servicing": Abstract some components to offer the capability to plug user own logic. For example, choose move one partition at a time as movement strategy instead of using per partition N -> N + 1 by default.

• User guide / configuration documentation: Document all the features supported by Helix and update the configurations / examples.