Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support plugins in client #1120

Open
dsinghal-nice opened this issue Aug 1, 2024 · 10 comments
Open

[FEATURE] Support plugins in client #1120

dsinghal-nice opened this issue Aug 1, 2024 · 10 comments
Labels
enhancement New feature or request

Comments

@dsinghal-nice
Copy link

What is the bug?

OpenSearchClient is not having any way to use _plugins/_sql?format=json API.

How can one reproduce the bug?

POST _plugins/_sql?format=json
{
"query": "SELECT * FROM books_test WHERE year > 1950"
}

Trying to execute such SQL based query using OpenSearchClient, but there's no way to do it.

@dsinghal-nice dsinghal-nice added bug Something isn't working untriaged labels Aug 1, 2024
@dblock
Copy link
Member

dblock commented Aug 1, 2024

We are working on adding plugins via a new code generator (first parts in #1052) from spec (https://github.com/opensearch-project/opensearch-api-specification). If you would like to help, a good place to start is to contribute the missing specs via opensearch-project/opensearch-api-specification#235. There are some PRs in progress, e.g. opensearch-project/opensearch-api-specification#456. @Xtansia is working on the generator parts.

@dblock
Copy link
Member

dblock commented Aug 1, 2024

Btw, you can use the generic client interface that should work for SQL, https://github.com/opensearch-project/opensearch-java/blob/main/guides/generic.md.

@dsinghal-nice
Copy link
Author

Thanks @dblock for the information, I will try to help for missing specs.

Just one question, When I use generic client to perform a SQL query search, I get the response Body in HitsMetadata format, is there a way to convert the Body to POJO, I checked the generic.md file as well, but the HitsMetadata class does not have _DESERIALIZER. Can you tell me if there's a way already available to convert below response in POJO or I need to write my own logic?

{
    "took": 25,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 0.0,
        "hits": [{
                "_index": "books_test",
                "_id": "2",
                "_score": 0.0,
                "_source": {
                    "id": "2",
                    "title": "Kafka on the shore Revised",
                    "year": 1963,
                    "author": "Philip Gabriel, Haruki Murakami"
                }
            }
        ]
    }
}

I tried below approach as mentioned in generic.md but the _DESERIALIZER is not present.

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

@dblock
Copy link
Member

dblock commented Aug 2, 2024

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

@reta
Copy link
Collaborator

reta commented Aug 2, 2024

We have the issue regarding that: #540

@reta
Copy link
Collaborator

reta commented Aug 2, 2024

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

This approach should work when POJO is properly specified

@dsinghal-nice
Copy link
Author

dsinghal-nice commented Aug 2, 2024

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

This approach should work when POJO is properly specified

@reta I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

@reta
Copy link
Collaborator

reta commented Aug 2, 2024

@reta I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

Correct, as per #540, the plugin specific clients are not yet available, so the _DESERIALIZER (or equivalents) need to be coded manually at the moment.

@Xtansia
Copy link
Collaborator

Xtansia commented Aug 5, 2024

I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

@dsinghal-nice It looks more like it returns the equivalent of SearchResponse, HitsMetadata is what's nested under hits. Both expose a createSearchResponseDeserializer or createHitsMetadataDeserializer respectively which is what you'd use instead of _DESERIALIZER as you need to provide the document type deserializer.

@dsinghal-nice
Copy link
Author

dsinghal-nice commented Aug 6, 2024

Hi @Xtansia, @reta, @dblock

Thanks for the help !!

Can you please help me with below questions as well ?

  1. While creating connection pool if the connection fails then does the library retries and how to configure in this case ?
  2. If connection lost while performing some operation like search or bulk insert, then how can I retry to initiate the connection back, does the library retries by itself ?
  3. How to create connection pooling from Java client in opensearch or how can I check how many active connections are live from my client ?
  4. How many concurrent request does the client supports ?

@Xtansia Xtansia removed the untriaged label Aug 7, 2024
@Xtansia Xtansia added enhancement New feature or request and removed bug Something isn't working labels Oct 21, 2024
@Xtansia Xtansia changed the title [BUG] Client not supporting Plugins [FEATURE] Support plugins in client Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants