Skip to content

Latest commit

 

History

History
61 lines (39 loc) · 3.07 KB

0021-query-scaling-implementation-decisions.md

File metadata and controls

61 lines (39 loc) · 3.07 KB

21. Query Scaling Implementation Decisions

Date: 2023-08-18

Status

Accepted

Context

Tangerine team is working on scaling multi-cluster querying initiative also known by Explorer. It was designed under this RFC. During its implementation we took different decisions that drifted from the original direction.

This ADR records them with its motivation and other context information that might be relevant.

Decision

Collector

Watching

Collection uses watching instead of polling, as well as leveraging controller-runtime machinery to have a more efficient (reduced latency to receive updates), reliable (include fault-tolerant mechanism) and simpler approach (already abstracted high-level functionality available).

Authentication and Authorization

In order to watch clusters, we need to authenticate and authorise against the remote kubernetes api. Collector follows the same approach as Weave Gitops of leveraging impersonation to manages its security context. It impersonates a collector service account that indicates the resources that collector can watch in remote clusters.

Authorization (RBAC)

Authorization for Explorer was implemented by watching RBAC resources via Collector, and as indicated in the RFC, doing authorization at query response time, filtering each object. Whilst this is custom business logic, the authorization business logic is leveraged to Kubernetes rbac implementation to have, as close as possible, native Kubernetes semantics.

Indexing

Given the limitations that searching based on sql-semantics imposes, Explorer was extended with indexing component that incorporate the features that are expected in an indexing based search engine like full-text search.

Retention

Initially Explorer was though a single cache to speed up load time and search experience. Since then, new use cases has been requested that have required extend the original design. Retention is one of them and has its own ADR

Unstructured resources

As part of retention, but to support the ability to build up more flexible and targeted searching capability, explorer response includes, apart from the normalised object metadata (as stated in the original design), the resource representation in json. It enables UIs and other experiences to be created in the way that better suits the use case.

Consequences

  • Anyone is able to build up a more complete understanding on Explorer implementation and design decisions.