Skip to content

Commit

Permalink
Address editorial feedback Full-text search section
Browse files Browse the repository at this point in the history
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
  • Loading branch information
vagimeli committed Dec 7, 2023
1 parent a8c3dd0 commit 6e4b17f
Showing 1 changed file with 36 additions and 34 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,28 @@
* SPDX-License-Identifier: Apache-2.0
*/

export const fullTextSearchFunction = `## Full Text Search
export const fullTextSearchFunction = `## Full-text search
---
The full text search based functions enable users to search the index for
documents by the full text search of the input query. The functions are built
on the top of the search queries of the OpenSearch engine, but in memory
execution within the plugin is not supported. These functions are able
to perform the global filter of a query, for example the condition
expression in a \`WHERE\` clause or in a \`HAVING\` clause. For more details
of the full text search based search, check out the design here: [Relevance
Based Search With SQL/PPL Query
Engine](https://github.com/opensearch-project/sql/issues/182)
Full-text search allows for searching by full-text queries. For details about full-text search in OpenSearch, see the [Full-text search](https://opensearch.org/docs/latest/search-plugins/sql/full-text/) documentation.
### MATCH
### Functions
**Description**
PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitte the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause.
#### MATCH
The \`match\` function maps user-defined criteria to OpenSearch queries, returning documents that match specific text, number, date, or Boolean values.
#### Function signature
The function signature is as follows:
\`match(field_expression, query_expression[, option=<option_value>]*)\`
The match function maps to the match query used in search engine, to
return the documents that match a provided text, number, date or boolean
value with a given field. Available parameters include:
#### Available
The available parameters are as follows:
- analyzer
- auto\_generate\_synonyms\_phrase
Expand All @@ -39,8 +39,9 @@ value with a given field. Available parameters include:
- zero\_terms\_query
- boost
Example with only \`field\` and \`query\` expressions, and all other
parameters are set default values:
**Example 1: Using specific expressions and default values**
The following example PPL query uses only the \`field\` and \`query\` expressions, with all other parameters set to their default values:
os> source=accounts | where match(address, 'Street') | fields lastname, address;
fetched rows / total rows = 2/2
Expand All @@ -51,8 +52,9 @@ parameters are set default values:
| Bates | 789 Madison Street |
+------------+--------------------+
Another example to show how to set custom values for the optional
parameters:
**Example 2: Setting custom values for optional parameters**
The following example PPL query sets custom values for the optional parameters:
os> source=accounts | where match(firstname, 'Hattie', operator='AND', boost=2.0) | fields lastname;
fetched rows / total rows = 1/1
Expand All @@ -64,19 +66,19 @@ parameters:
### Limitations
The full text search functions are available to execute only in OpenSearch DSL
but not in memory as of now, so the full text search might fail for
queries that are too complex to translate into DSL if the full text search
function is following after a complex PPL query. To make your queries
always work-able, it is recommended to place the full text search commands as
close to the search command as possible, to ensure the full text search
functions are eligible to push down. For example, a complex query like
\`search source = people | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | where match(employer, 'Open Search') | stats count() by city\`
could fail because it is difficult to translate to DSL, but it would be
better if we rewrite it to an equivalent query as
\`search source = people | where match(employer, 'Open Search') | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | stats count() by city\`
by moving the where command with full text search function to the second
command right after the search command, and the full text search would be
optimized and executed smoothly in OpenSearch DSL. See [Optimization](https://github.com/opensearch-project/sql/blob/22924b13d9cb46759c8d213a7ce903effe06ab47/docs/user/optimization/optimization.rst)
to get more details about the query engine optimization.
The full-text search functions can be executed only in [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/), not in-memory.
To ensure optimal performance and avoid translation issues with complex full-text searhes, place them as clauses within the search command.
**Example:
The following is an example complex query that could fail because it is difficult to translate to query DSL:
\`search source = people | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | where match(employer, 'Open Search') | stats count() by city\`
To optimize full-text search performance, rewrite the query by placing the \`WHERE\` clause with the full-text search function as the second command after the \`SEARCH\` command. This ensures the full-text search gets pushed down to query DSL. The following is an example query:
\`search source = people | where match(employer, 'Open Search') | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | stats count() by city\`
For details about query engine optimization, see the [Optimizations](https://github.com/opensearch-project/sql/blob/22924b13d9cb46759c8d213a7ce903effe06ab47/docs/user/optimization/optimization.rst) developer documentation on GitHub.
`;

0 comments on commit 6e4b17f

Please sign in to comment.