[BUG] IndicesStatsResponse bug when specifying the Shard Level in the IndicesStatsRequest

[JmacCl]

What is the Bug

When sending a IndicesStatsRequest via a OpenSearchIndicesClient request, the following error is raised

org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'ShardStats.shards'

The error is primarily related only to when the Shard level is specified. For other levels, the error does not occur.

I believe the error is centered around how the ShardStats class creates objects for the statistics for each of the primary shards. The error is called around the function call of requireNonNull in the ApiTypeHelper class.

There is method to bypass this, in which before calling the response, one call call ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true); to ensure that a given response is returned

How to Run

OpenSearch version info
{
"name": -----,
"cluster_name": -----,
"cluster_uuid": "eRQAiiiiVRJ67bdt_J2IBXQ",
"version": {
"distribution": "opensearch",
"number": "2.15.0",
"build_type": "tar",
"build_hash": "61dbcd079goe5gn4o5be5762175414bc38bbcadf",
"build_date": "2024-06-20T03:26:49.193630411Z",
"build_snapshot": false,
"lucene_version": "9.10.0",
"minimum_wire_compatibility_version": "7.10.0",
"minimum_index_compatibility_version": "7.0.0"
},
"tagline": "The OpenSearch Project: https://opensearch.org/"
}

For failure

IndicesStatsRequest indicesStatsRequest=newIndicesStatsRequest.Builder().index(indices).level(Level.Shards).build();
// (openSearchClient is a OpenSearchClient class)
openSearchClient.indices().stats(indicesStatsRequest);

For 'dangerous' success

ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true);
IndicesStatsRequest indicesStatsRequest=newIndicesStatsRequest.Builder().index(indices).level(Level.Shards).build();
// (openSearchClient is a OpenSearchClient class)
openSearchClient.indices().stats(indicesStatsRequest);

What is the expected behaviour

Either the ShardStats are returned or are not required.

Further details

The API Documentation on the README contains no information for the field, nor of the ShardStats class.
Furthermore, for the relevant documentation page, https://opensearch.org/docs/latest/api-reference/index-apis/stats, there is no information on the shards field

[dblock]

@JmacCl could you please check whether this is correctly specified in https://github.com/opensearch-project/opensearch-api-specification (and if not maybe fix it)? @Xtansia is working on replacing all this hand-rolled code by a generator.

[LiamMacpherson]

Hi @dblock, I'd be happy to contribute the resolution to this issue since I've seen it too, and had to utilise the workaround James outlined.

From the api-spec https://github.com/opensearch-project/opensearch-api-specification/blob/main/spec/schemas/indices.stats.yaml#L95, ShardStats.shards is not an expected property and the docs here https://opensearch.org/docs/latest/api-reference/index-apis/stats do not have a "shards" as a property in the exemplar response.

[dblock]

Hi @dblock, I'd be happy to contribute the resolution to this issue since I've seen it too, and had to utilise the workaround James outlined.

From the api-spec https://github.com/opensearch-project/opensearch-api-specification/blob/main/spec/schemas/indices.stats.yaml#L95, ShardStats.shards is not an expected property and the docs here https://opensearch.org/docs/latest/api-reference/index-apis/stats do not have a "shards" as a property in the exemplar response.

Please and thank you. You'll need to dig up when it was added, etc. We plan to generate docs from the api spec, so up to you if you want to manually add that too now.