[META] Grouping similar Top N Queries by Latency and Resource Usage

[deshsidd]

Please describe the end goal of this project

In RFC #13357, we proposed to group Top N queries by latency and resource usage using Query shape. For Top N queries by latency, we can encounter scenarios where some (or most) of the Top N queries contain duplicate queries. To overcome such scenarios and to get a more detailed view of the Top N query patterns we are proposing to implement Top N query shapes by resource usage (latency, memory, disk, etc).

Supporting References

RFC: #13357

Issues

Following is the high level break up of tasks of the project. I will keep linking github issues and PRs as and when they are published.

• Create query shape representation class structure
• Parse real queries to POJO representation
• Apply normalizations on the query shape and implement hashcode for the query shape object
• Enhance or create a new processor to store point in time Top N query shape data by latency
• Enhance APIs and/or configuration to expose the Top N query shapes feature to the end user and provide the ability to enable/disable the feature
• Write Unit and integration tests

Related component

Search:Query Insights

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@deshsidd Thanks for creating this meta issue to track this effort.

[deshsidd]

resolved in opensearch-project/query-insights#66