From cfdc58ea0edc280d774681740db722da352da12a Mon Sep 17 00:00:00 2001 From: "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Date: Thu, 18 Jan 2024 00:58:26 -0800 Subject: [PATCH] [DOC]Revise and edit PPL in-product documentation (#1233) (#1368) (cherry picked from commit 31b0a94f2619c76c01dc053d6f4861b562e2c0a3) Signed-off-by: Melissa Vagi Signed-off-by: github-actions[bot] Co-authored-by: github-actions[bot] Co-authored-by: Nathan Bower (cherry picked from commit 65f8eaa015b3a526d9fc946d8b5a1805b9f10793) --- auto_sync_commit_metadata.json | 4 +- .../common/helpers/ppl_docs/commands/dedup.ts | 55 +-- .../common/helpers/ppl_docs/commands/eval.ts | 41 +- .../helpers/ppl_docs/commands/fields.ts | 25 +- .../common/helpers/ppl_docs/commands/head.ts | 21 +- .../common/helpers/ppl_docs/commands/parse.ts | 70 +-- .../common/helpers/ppl_docs/commands/rare.ts | 26 +- .../helpers/ppl_docs/commands/rename.ts | 21 +- .../helpers/ppl_docs/commands/search.ts | 26 +- .../common/helpers/ppl_docs/commands/sort.ts | 47 +- .../common/helpers/ppl_docs/commands/stats.ts | 171 ++----- .../helpers/ppl_docs/commands/syntax.ts | 17 +- .../common/helpers/ppl_docs/commands/top.ts | 38 +- .../common/helpers/ppl_docs/commands/where.ts | 15 +- .../helpers/ppl_docs/functions/condition.ts | 90 ++-- .../helpers/ppl_docs/functions/datetime.ts | 463 +++++++----------- .../ppl_docs/functions/full_text_search.ts | 64 ++- .../common/helpers/ppl_docs/functions/math.ts | 318 +++++------- .../helpers/ppl_docs/functions/string.ts | 129 ++--- .../ppl_docs/language_structure/datatypes.ts | 328 +++++-------- .../language_structure/identifiers.ts | 82 +--- .../common/helpers/ppl_docs/overview.tsx | 35 +- 22 files changed, 758 insertions(+), 1328 deletions(-) diff --git a/auto_sync_commit_metadata.json b/auto_sync_commit_metadata.json index a91ec1207..89c292428 100644 --- a/auto_sync_commit_metadata.json +++ b/auto_sync_commit_metadata.json @@ -1,4 +1,4 @@ { - "last_github_commit": "67a5952bb6563ef62f3be857dbb4702c32d8d4da", - "last_gitfarm_commit": "2a49733406a265032c2f0e297b04253a8f715d17" + "last_github_commit": "65f8eaa015b3a526d9fc946d8b5a1805b9f10793", + "last_gitfarm_commit": "9bafd992c7fd41788f7bacb5e1fb82a1a8478edd" } \ No newline at end of file diff --git a/public/components/common/helpers/ppl_docs/commands/dedup.ts b/public/components/common/helpers/ppl_docs/commands/dedup.ts index 2de86921b..1acde2fb9 100644 --- a/public/components/common/helpers/ppl_docs/commands/dedup.ts +++ b/public/components/common/helpers/ppl_docs/commands/dedup.ts @@ -8,33 +8,21 @@ export const dedupCmd = `## dedup ### Description -Using \'dedup\' command to remove identical document defined by field from -the search result. +Use the \'dedup\' command to remove identical documents from the search results, based on the specified field. ### Syntax dedup \[int\] <field-list> \[keepempty=<bool>\] \[consecutive=<bool>\] -- int: optional. The \'dedup\' command retains multiple events for each - combination when you specify <int>. The number for <int> - must be greater than 0. If you do not specify a number, only the - first occurring event is kept. All other duplicates are removed from - the results. **Default:** 1 -- keepempty: optional. if true, keep the document if the any field in - the field-list has NULL value or field is MISSING. **Default:** - false. -- consecutive: optional. If set to true, removes only events with - duplicate combinations of values that are consecutive. **Default:** - false. -- field-list: mandatory. The comma-delimited field list. At least one - field is required. +- \`field-list\`: Required. The comma-delimited field list. At least one field is required. +- \`consecutive\`: Optional. If set to \`true\`, removes duplicate events, where the duplicate events have consecutive timestamps. Default is \`false\`. +- \`int\`: Optional. The \'dedup\' command retains multiple events for each combination when you specify \`<int>\`. The number for \`<int>\` must be greater than 0. If you do not specify a number, only the first occurring event is kept. All other duplicates are removed from the results. Default is \`1\`. +- \`keepempty\`: Optional. If set to \`true\`, keeps the document if any field in the \`field-list\` is null or missing. Default is \`false\`. -### Example 1: Dedup by one field +#### Example 1: Dedup by one field -The example show dedup the document with gender field. - -PPL query: +The following example PPL query shows how to use \`dedup\` to remove duplicate documents based on the \`gender\` field: os> source=accounts | dedup gender | fields account_number, gender; fetched rows / total rows = 2/2 @@ -45,12 +33,9 @@ PPL query: | 13 | F | +------------------+----------+ -### Example 2: Keep 2 duplicates documents - -The example show dedup the document with gender field keep 2 -duplication. +#### Example 2: Keep two duplicate documents -PPL query: +The following example PPL query shows how to use \`dedup\` to remove duplicate documents based on the \`gender\` field while keeping two duplicates: os> source=accounts | dedup 2 gender | fields account_number, gender; fetched rows / total rows = 3/3 @@ -62,11 +47,9 @@ PPL query: | 13 | F | +------------------+----------+ -### Example 3: Keep or Ignore the empty field by default - -The example show dedup the document by keep null value field. +#### Example 3: Keep or ignore empty fields by default -PPL query: +The following example PPL query shows how to use \`dedup\` to remove duplicate documents while keeping documents with null values in the specified field: os> source=accounts | dedup email keepempty=true | fields account_number, email; fetched rows / total rows = 4/4 @@ -79,9 +62,7 @@ PPL query: | 18 | daleadams@boink.com | +------------------+-----------------------+ -The example show dedup the document by ignore the empty value field. - -PPL query: +The following example PPL query shows how to use \`dedup\` to remove duplicate documents while ignoring documents with empty values in the specified field: os> source=accounts | dedup email | fields account_number, email; fetched rows / total rows = 3/3 @@ -93,11 +74,9 @@ PPL query: | 18 | daleadams@boink.com | +------------------+-----------------------+ -#### Example 4: Dedup in consecutive document - -The example show dedup the consecutive document. +#### Example 4: Remove duplicate consecutive documents -PPL query: +The following example PPL query shows how to use \`dedup\` to remove duplicate consecutive documents: os> source=accounts | dedup gender consecutive=true | fields account_number, gender; fetched rows / total rows = 3/3 @@ -109,6 +88,6 @@ PPL query: | 18 | M | +------------------+----------+ -#### Limitation -The \`dedup\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. -`; \ No newline at end of file +### Limitation +The \`dedup\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. +`; diff --git a/public/components/common/helpers/ppl_docs/commands/eval.ts b/public/components/common/helpers/ppl_docs/commands/eval.ts index 52a9429ed..f20a5ee35 100644 --- a/public/components/common/helpers/ppl_docs/commands/eval.ts +++ b/public/components/common/helpers/ppl_docs/commands/eval.ts @@ -8,26 +8,21 @@ export const evalCmd = `## eval ### Description -The \'eval\' command evaluate the expression and append the result to the -search result. +Use the \'eval\' command to evaluate the expression and append the result to the search result. ### Syntax eval <field>=<expression> \["," <field>=<expression> \]... -- field: mandatory. If the field name not exist, a new field is added. - If the field name already exists, it will be overrided. -- expression: mandatory. Any expression support by the system. +- \`field\`: Required. If the field name does not exist, a new field is created. If the field name exists, the value of the existing field is replaced. +- \`expression\`: Required. Any expression that is supported by the system. -### Example 1: Create the new field +#### Example 1: Create new fields -The example show to create new field doubleAge for each document. The -new doubleAge is the evaluation result of age multiply by 2. +The following example PPL query shows how to use \`eval\` to create a new field for each document. In this example, the new field is \`doubleAge\`. -PPL query: - - os> source=accounts | eval doubleAge = age * 2 | fields age, doubleAge ; + os> source=accounts | eval doubleAge = age * 2 | fields age, doubleAge; fetched rows / total rows = 4/4 +-------+-------------+ | age | doubleAge | @@ -38,13 +33,11 @@ PPL query: | 33 | 66 | +-------+-------------+ -### Example 2: Override the existing field - -The example show to override the exist age field with age plus 1. +#### Example 2: Override existing fields -PPL query: +The following example PPL query shows how to use \`eval\` to override an existing field. In this example, the existing field \`age\` is overridden by the \`age\` field plus 1. - os> source=accounts | eval age = age + 1 | fields age ; + os> source=accounts | eval age = age + 1 | fields age; fetched rows / total rows = 4/4 +-------+ | age | @@ -55,15 +48,11 @@ PPL query: | 34 | +-------+ -### Example 3: Create the new field with field defined in eval - -The example show to create a new field ddAge with field defined in eval -command. The new field ddAge is the evaluation result of doubleAge -multiply by 2, the doubleAge is defined in the eval command. +#### Example 3: Create new fields based on the fields defined in the \`eval\` expression -PPL query: +The following example PPL query shows how to use \`eval\` to create a new field based on the fields defined in the \`eval\` expression. In this example, the new field \`ddAge\` is the evaluation result of the \`doubleAge\` field multiplied by 2. \`doubleAge\` is defined in the \`eval\` command. - os> source=accounts | eval doubleAge = age * 2, ddAge = doubleAge * 2 | fields age, doubleAge, ddAge ; + os> source=accounts | eval doubleAge = age * 2, ddAge = doubleAge * 2 | fields age, doubleAge, ddAge; fetched rows / total rows = 4/4 +-------+-------------+---------+ | age | doubleAge | ddAge | @@ -74,6 +63,6 @@ PPL query: | 33 | 66 | 132 | +-------+-------------+---------+ -#### Limitation -The \`eval\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. -`; \ No newline at end of file +### Limitation +The \`eval\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. +`; diff --git a/public/components/common/helpers/ppl_docs/commands/fields.ts b/public/components/common/helpers/ppl_docs/commands/fields.ts index 61a6954fd..2110349b5 100644 --- a/public/components/common/helpers/ppl_docs/commands/fields.ts +++ b/public/components/common/helpers/ppl_docs/commands/fields.ts @@ -7,23 +7,18 @@ export const fieldsCmd = `## fields --- ### Description -Using \`fields\` command to keep or remove fields from the search result. +Use the \`fields\` command to specify the fields that should be included in or excluded from the search results. ### Syntax fields \[+\|-\] <field-list> -- index: optional. if the plus (+) is used, only the fields specified - in the field list will be keep. if the minus (-) is used, all the - fields specified in the field list will be removed. **Default** + -- field list: mandatory. comma-delimited keep or remove fields. +- \`field-list\`: Required. Comma-separated list of fields to keep or remove. +- \`index\`: Optional. If the plus sign \`+\` is used, only the fields specified in the field list will be included. If the minus \`-\` is used, all the fields specified in the field list will be excluded. Default is \`+\`. -### Example 1: Select specified fields from result +#### Example 1: Select specified fields from the search result -The example show fetch account\_number, firstname and lastname fields -from search results. - -PPL query: +The following example PPL query shows how to retrieve the \`account\_number\`, \`firstname\`, and \`lastname\` fields from the search results: os> source=accounts | fields account_number, firstname, lastname; fetched rows / total rows = 4/4 @@ -36,13 +31,11 @@ PPL query: | 18 | Dale | Adams | +------------------+-------------+------------+ -### Example 2: Remove specified fields from result - -The example show fetch remove account\_number field from search results. +#### Example 2: Remove specified fields from the search results -PPL query: +The following example PPL query shows how to remove the \`account\_number\` field from the search results: - os> source=accounts | fields account_number, firstname, lastname | fields - account_number ; + os> source=accounts | fields account_number, firstname, lastname | fields - account_number; fetched rows / total rows = 4/4 +-------------+------------+ | firstname | lastname | @@ -52,4 +45,4 @@ PPL query: | Nanette | Bates | | Dale | Adams | +-------------+------------+ -`; \ No newline at end of file +`; diff --git a/public/components/common/helpers/ppl_docs/commands/head.ts b/public/components/common/helpers/ppl_docs/commands/head.ts index 78ffe7b94..468da72d3 100644 --- a/public/components/common/helpers/ppl_docs/commands/head.ts +++ b/public/components/common/helpers/ppl_docs/commands/head.ts @@ -8,20 +8,17 @@ export const headCmd = `## head ### Description -The \`head\` command returns the first N number of specified results in -search order. +Use the \`head\` command to return the first N number of lines from a search result. ### Syntax head \[N\] -- N: optional. number of results to return. **Default:** 10 +- \`N\`: Optional. The number of results you want to return. Default is 10. -### Example 1: Get first 10 results +#### Example 1: Get the first 10 results -The example show first 10 results from accounts index. - -PPL query: +The following example PPL query shows how to use \`head\` to return the first 10 search results: os> source=accounts | fields firstname, age | head; fetched rows / total rows = 10/10 @@ -40,11 +37,9 @@ PPL query: | Fulton | 23 | +---------------+-----------+ -### Example 2: Get first N results - -The example show first N results from accounts index. +#### Example 2: Get the first N results -PPL query: +The following example PPL query shows how to use \`head\` to get a specified number of search results. In this example, N is equal to 3: os> source=accounts | fields firstname, age | head 3; fetched rows / total rows = 3/3 @@ -57,5 +52,5 @@ PPL query: +---------------+-----------+ #### Limitation -The \`head\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. -`; \ No newline at end of file +The \`head\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. +`; diff --git a/public/components/common/helpers/ppl_docs/commands/parse.ts b/public/components/common/helpers/ppl_docs/commands/parse.ts index ee752c1ea..28501cf85 100644 --- a/public/components/common/helpers/ppl_docs/commands/parse.ts +++ b/public/components/common/helpers/ppl_docs/commands/parse.ts @@ -7,33 +7,24 @@ export const parseCmd = `## parse --- ### Description -The \`parse\` command parses a text field using a regular expression and append -the result to the search result. +Use the \`parse\` command to extract information from a text field using a regular expression and add it to the search result. ### Syntax parse <field> <regular-expression> -- field: mandatory. The field must be a text field. -- regular-expression: mandatory. The regular expression used to extract new -fields from given text field. If a new field name already exists, it will -replace the original field. +- \`field\`: Required. Must be a text field. +- \`regular-expression\`: Required. The regular expression used to extract new fields from a text field. It replaces the original field if a new field name exists. -### Regular Expression +### Regular expression -The regular expression is used to match the whole text field of each document -with Java regex engine. Each named capture group in the expression will become -a new \`STRING\` field. +Use the Java regular expression engine to match the entire text field of each document. Each named capture group in the expression will be converted to a new \`string\` field. -### Example 1: Create the new field +#### Example 1: Create a new field -The example shows how to create new field \`host\` for each document. \`host\` -will be the host name after \`@\` in \`email\` field. Parsing a null field will -return an empty string. +The following example PPL query shows how to create new field \`host\` for each document. \`host\` becomes the hostname after the @ symbol in the \`email\` field. Parsing a null field returns an empty string. -PPL query: - - os> source=accounts | parse email '.+@(?.+)' | fields email, host ; + os> source=accounts | parse email '.+@(?.+)' | fields email, host; fetched rows / total rows = 4/4 +-----------------------+------------+ | email | host | @@ -44,14 +35,11 @@ PPL query: | daleadams@boink.com | boink.com | +-----------------------+------------+ -### Example 2: Override the existing field - -The example shows how to override the existing \`address\` field with street -number removed. +#### Example 2: Override an existing field -PPL query: +The following example PPL query shows how to override the existing \`address\` field while excluding the street number: - os> source=accounts | parse address '\\d+ (?
.+)' | fields address ; + os> source=accounts | parse address '\\d+ (?
.+)' | fields address; fetched rows / total rows = 4/4 +------------------+ | address | @@ -62,14 +50,11 @@ PPL query: | Hutchinson Court | +------------------+ -### Example 3: Filter and sort by casted parsed field +#### Example 3: Filter and sort by casted-parsed field -The example shows how to sort street numbers that are higher than 500 in -\`address\` field. +The following example PPL query shows how to sort street numbers that are greater than 500 in the \`address\` field: -PPL query: - - os> source=accounts | parse address '(?\d+) (?.+)' | where cast(streetNumber as int) > 500 | sort num(streetNumber) | fields streetNumber, street ; + os> source=accounts | parse address '(?\d+) (?.+)' | where cast(streetNumber as int) > 500 | sort num(streetNumber) | fields streetNumber, street; fetched rows / total rows = 3/3 +----------------+----------------+ | streetNumber | street | @@ -81,30 +66,21 @@ PPL query: ### Limitation -There are a few limitations with parse command: - -- Fields defined by parse cannot be parsed again. - - The following command will not work: - - source=accounts | parse address '\\d+ (?.+)' | parse street '\\w+ (?\\w+)' ; - -- Fields defined by parse cannot be overridden with other commands. +The following limitations apply: - \`where\` will not match any documents since \`street\` cannot be overridden: +- Parsed fields cannot be parsed again. For example, the following command is not valid: - source=accounts | parse address '\\d+ (?.+)' | eval street='1' | where street='1' ; + source=accounts | parse address '\\d+ (?.+)' | parse street '\\w+ (?\\w+)'; -- The text field used by parse cannot be overridden. +- Other commands cannot overwrite fields created by parsing. For example, in the following query, \`where\` does not match any documents because \`street\` cannot be overridden: - \`street\` will not be successfully parsed since \`address\` is overridden: + source=accounts | parse address '\\d+ (?.+)' | eval street='1' | where street='1'; - source=accounts | parse address '\\d+ (?.+)' | eval address='1' ; +- The text field that is parsed cannot be overridden. For example, in the following query, \`street\` is not successfully parsed because \`address\` is overridden: -- Fields defined by parse cannot be filtered/sorted after using them in -\`stats\` command. + source=accounts | parse address '\\d+ (?.+)' | eval address='1'; - \`where\` in the following command will not work: +- Fields created by parsing cannot be filtered or sorted after using them in the \`stats\` command. For example, in the following query, \`where\` is not valid: - source=accounts | parse email '.+@(?.+)' | stats avg(age) by host | where host=pyrami.com ; + source=accounts | parse email '.+@(?.+)' | stats avg(age) by host | where host=pyrami.com; `; diff --git a/public/components/common/helpers/ppl_docs/commands/rare.ts b/public/components/common/helpers/ppl_docs/commands/rare.ts index 37e504f33..61f5016f4 100644 --- a/public/components/common/helpers/ppl_docs/commands/rare.ts +++ b/public/components/common/helpers/ppl_docs/commands/rare.ts @@ -8,24 +8,18 @@ export const rareCmd = `## rare ### Description -Using \`rare\` command to find the least common tuple of values of all -fields in the field list. - -**Note**: A maximum of 10 results is returned for each distinct tuple of -values of the group-by fields. +Use the \`rare\` command to find the least common tuple of values across all fields in the \`field-list\` field. A maximum of 10 results is returned for each distinct tuple of group-by field values. ### Syntax rare <field-list> \[by-clause\] -- field-list: mandatory. comma-delimited list of field names. -- by-clause: optional. one or more fields to group the results by. - -### Example 1: Find the least common values in a field +- \`field-list\`: Required. A comma-separated list of field names. +- \`by-clause\`: Optional. One or more fields to group by. -The example finds least common gender of all the accounts. +#### Example 1: Find a field's least common values -PPL query: +The following example PPL query shows how to find a least common value in the \`gender\` field: os> source=accounts | rare gender; fetched rows / total rows = 2/2 @@ -37,11 +31,9 @@ PPL query: | M | +------------+ -### Example 2: Find the least common values organized by gender - -The example finds least common age of all the accounts group by gender. +#### Example 2: Find least common values in group-by fields -PPL query: +The following example PPL query shows how to find a least common value in the \`age\` field that is grouped by \`gender\`: os> source=accounts | rare age by gender; fetched rows / total rows = 20/20 @@ -71,5 +63,5 @@ PPL query: +----------+----------+ #### Limitation -The \`rare\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. -`; \ No newline at end of file +The \`rare\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. +`; diff --git a/public/components/common/helpers/ppl_docs/commands/rename.ts b/public/components/common/helpers/ppl_docs/commands/rename.ts index 46238384b..dc333b991 100644 --- a/public/components/common/helpers/ppl_docs/commands/rename.ts +++ b/public/components/common/helpers/ppl_docs/commands/rename.ts @@ -7,22 +7,19 @@ export const renameCmd = `## rename --- ### Description -Using \`rename\` command to rename one or more fields in the search -result. +Use the \`rename\` command to rename one or more fields in the search result. ### Syntax rename <source-field> AS <target-field>\["," <source-field> AS <target-field>\]... -- source-field: mandatory. The name of the field you want to rename. -- field list: mandatory. The name you want to rename to. +- \`source-field\`: Required. The field to rename. +- \`target-field\`: Required. The new field. -### Example 1: Rename one field +#### Example 1: Rename one field -The example show rename one field. - -PPL query: +The following example PPL query renames a field: os> source=accounts | rename account_number as an | fields an; fetched rows / total rows = 4/4 @@ -35,11 +32,9 @@ PPL query: | 18 | +------+ -### Example 2: Rename multiple fields - -The example show rename multiple fields. +#### Example 2: Rename two or more fields -PPL query: +The following example PPL query renames two or more fields: os> source=accounts | rename account_number as an, employer as emp | fields an, emp; fetched rows / total rows = 4/4 @@ -53,5 +48,5 @@ PPL query: +------+---------+ #### Limitation -The \`rename\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. +The \`rename\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. `; \ No newline at end of file diff --git a/public/components/common/helpers/ppl_docs/commands/search.ts b/public/components/common/helpers/ppl_docs/commands/search.ts index b54d5a739..2c2e361dd 100644 --- a/public/components/common/helpers/ppl_docs/commands/search.ts +++ b/public/components/common/helpers/ppl_docs/commands/search.ts @@ -8,24 +8,20 @@ export const searchCmd = `## search ### Description -Using \`search\` command to retrieve document from the index. \`search\` -command could be only used as the first command in the PPL query. +Use the \`search\` command to retrieve a document from the index. The \`search\` +command can only be used as the first command in a PPL query. ### Syntax search source=<index> \[boolean-expression\] -- search: search keywords, which could be ignore. -- index: mandatory. search command must specify which index to query - from. -- bool-expression: optional. any expression which could be evaluated - to boolean value. +- \`search\`: Search keywords, which can be ignored. +- \`index\`: Required. Search commands must specify the index to query. +- \`bool-expression\`: Optional. Any expression that can be evaluated to a Boolean value. -### Example 1: Fetch all the data +#### Example 1: Fetch all data from an index -The example show fetch all the document from accounts index. - -PPL query: +The following example PPL query shows how to fetch all documents from the \`accounts\` index: os> source=accounts; fetched rows / total rows = 4/4 @@ -38,11 +34,9 @@ PPL query: | 18 | Dale | 467 Hutchinson Court | 4180 | M | Orick | null | MD | 33 | daleadams@boink.com | Adams | +----------------+-----------+----------------------+---------+--------+--------+----------+-------+-----+-----------------------+----------+ -### Example 2: Fetch data with condition - -The example show fetch all the document from accounts index with . +#### Example 2: Fetch data with a condition -PPL query: +The following example PPL query shows how to fetch all documents from the \`accounts\` index by using the \`or\` condition. os> source=accounts account_number=1 or gender="F"; fetched rows / total rows = 2/2 @@ -52,4 +46,4 @@ PPL query: | 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | amberduke@pyrami.com | Duke | | 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates | +------------------+-------------+--------------------+-----------+----------+--------+------------+---------+-------+----------------------+------------+ -`; \ No newline at end of file +`; diff --git a/public/components/common/helpers/ppl_docs/commands/sort.ts b/public/components/common/helpers/ppl_docs/commands/sort.ts index e4c1f9404..d31e84a96 100644 --- a/public/components/common/helpers/ppl_docs/commands/sort.ts +++ b/public/components/common/helpers/ppl_docs/commands/sort.ts @@ -7,25 +7,18 @@ export const sortCmd = `## sort --- ### Description -Using \`sort\` command to sorts all the search result by the specified -fields. +Use the \`sort\` command to sort search results by a specified field. ### Syntax sort <\[+\|-\] sort-field>... -- \[+\|-\]: optional. The plus \[+\] stands for ascending order and - NULL/MISSING first and a minus \[-\] stands for descending order and - NULL/MISSING last. **Default:** ascending order and NULL/MISSING - first. -- sort-field: mandatory. The field used to sort. +- \`sort-field\`: Required. The field to sort by. +- \[+\|-\]: Optional. The symbols used to indicate the sorting order. A plus sign \[+\] indicates ascending order, with null and missing values first. A minus sign \[-\] indicates descending order, with null and missing last. Default is \[+\], with null and missing first. -### Example 1: Sort by one field +#### Example 1: Sort by one field -The example show sort all the document with age field in ascending -order. - -PPL query: +The following example PPL query shows how to sort documents by one field in ascending order: os> source=accounts | sort age | fields account_number, age; fetched rows / total rows = 4/4 @@ -38,12 +31,9 @@ PPL query: | 6 | 36 | +------------------+-------+ -### Example 2: Sort by one field return all the result - -The example show sort all the document with age field in ascending -order. +#### Example 2: Sort by one field and return all results -PPL query: +The following example PPL query shows how to sort by one field and return all results in ascending order: os> source=accounts | sort age | fields account_number, age; fetched rows / total rows = 4/4 @@ -56,12 +46,9 @@ PPL query: | 6 | 36 | +------------------+-------+ -### Example 3: Sort by one field in descending order - -The example show sort all the document with age field in descending -order. +#### Example 3: Sort by one field in descending order -PPL query: +The following example PPL query shows how to sort by one field in descending order: os> source=accounts | sort - age | fields account_number, age; fetched rows / total rows = 4/4 @@ -74,12 +61,9 @@ PPL query: | 13 | 28 | +------------------+-------+ -### Example 4: Sort by multiple field +#### Example 4: Sort multiple fields in both ascending and descending order -The example show sort all the document with gender field in ascending -order and age field in descending. - -PPL query: +The following example PPL query shows how to sort by multiple fields in both ascending and descending order. In this example, the \`gender\` field is in ascending order and the \`age\` field is in descending order. os> source=accounts | sort + gender, - age | fields account_number, gender, age; fetched rows / total rows = 4/4 @@ -92,12 +76,9 @@ PPL query: | 1 | M | 32 | +------------------+----------+-------+ -### Example 5: Sort by field include null value - -The example show sort employer field by default option (ascending order -and null first), the result show that null value is in the first row. +#### Example 5: Sort by field, including null values -PPL query: +The following example PPL query shows how to sort by the \`employer\` field using the default order (\[+\] with null and missing first): os> source=accounts | sort employer | fields employer; fetched rows / total rows = 4/4 @@ -109,4 +90,4 @@ PPL query: | Pyrami | | Quility | +------------+ -`; \ No newline at end of file +`; diff --git a/public/components/common/helpers/ppl_docs/commands/stats.ts b/public/components/common/helpers/ppl_docs/commands/stats.ts index a3533193b..7bbf9c2ab 100644 --- a/public/components/common/helpers/ppl_docs/commands/stats.ts +++ b/public/components/common/helpers/ppl_docs/commands/stats.ts @@ -8,10 +8,9 @@ export const statsCmd = `## stats ### Description -Using \`stats\` command to calculate the aggregation from search result. +Use the \`stats\` command to calculate the aggregation from the search results. -The following table catalogs the aggregation functions and also -indicates how the NULL/MISSING values is handled: +The following table catalogs the aggregation functions and defines how the null and missing values are handled. | | | | |----------|-------------|-------------| @@ -26,23 +25,14 @@ indicates how the NULL/MISSING values is handled: stats <aggregation>... \[by-clause\]... -- aggregation: mandatory. A aggregation function. The argument of - aggregation must be field. -- by-clause: optional. The one or more fields to group the results by. - **Default**: If no <by-clause> is specified, the stats command - returns only one row, which is the aggregation over the entire - result set. +- \`aggregation\`: Required. The aggregation function that must be applied to the field. +- \`by-clause\`: Optional. One or more fields to group by. Default: If \`<by-clause>\` is not specified, the \`stats\` command returns one row, which is the aggregation for the entire result set. -### **Aggregation Functions** +### Aggregation functions --- -### **COUNT** +#### COUNT -**Description** - -Usage: Returns a count of the number of expr in the rows retrieved by a -SELECT statement. - -Example: +The \`count\` function returns the number of rows in the result set. The following is an example PPL query: os> source=accounts | stats count(); fetched rows / total rows = 1/1 @@ -52,13 +42,9 @@ Example: | 4 | +-----------+ -### **SUM** - -**Description** - -Usage: SUM(expr). Returns the sum of expr. +#### SUM -Example: +The \`SUM(expr)\` function returns the sum of the values in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats sum(age) by gender; fetched rows / total rows = 2/2 @@ -69,13 +55,9 @@ Example: | 101 | M | +------------+----------+ -### **AVG** +#### AVG -**Description** - -Usage: AVG(expr). Returns the average value of expr. - -Example: +The \`AVG(expr)\` function returns the average of the values in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats avg(age) by gender; fetched rows / total rows = 2/2 @@ -86,13 +68,9 @@ Example: | 33.666666666666664 | M | +--------------------+----------+ -### **MAX** - -**Description** +#### MAX -Usage: MAX(expr). Returns the maximum value of expr. - -Example: +The \`MAX(expr)\` function returns the largest value in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats max(age); fetched rows / total rows = 1/1 @@ -102,13 +80,9 @@ Example: | 36 | +------------+ -### **MIN** - -**Description** - -Usage: MIN(expr). Returns the minimum value of expr. +#### MIN -Example: +The \`MIN(expr)\` function returns the smallest value in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats min(age); fetched rows / total rows = 1/1 @@ -118,13 +92,9 @@ Example: | 28 | +------------+ -### **VAR\_SAMP** +#### VAR\_SAMP -**Description** - -Usage: VAR\_SAMP(expr). Returns the sample variance of expr. - -Example: +The \`VAR\_SAMP(expr)\` function returns the sample variance of a selection of data in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats var_samp(age); fetched rows / total rows = 1/1 @@ -134,13 +104,9 @@ Example: | 10.916666666666666 | +--------------------+ -### **VAR\_POP** - -**Description** +#### VAR\_POP -Usage: VAR\_POP(expr). Returns the population standard variance of expr. - -Example: +The \`VAR\_POP(expr)\` function returns the population variance of a selection of data in the expression \`expr\`. See the following example. os> source=accounts | stats var_pop(age); fetched rows / total rows = 1/1 @@ -150,13 +116,9 @@ Example: | 8.1875 | +----------------+ -### **STDDEV\_SAMP** - -**Description** +#### STDDEV\_SAMP -Usage: STDDEV\_SAMP(expr). Return the sample standard deviation of expr. - -Example: +The \`STDDEV\_SAMP(expr)\` function returns the sample standard deviation of a set of values in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats stddev_samp(age); fetched rows / total rows = 1/1 @@ -166,14 +128,9 @@ Example: | 3.304037933599835 | +--------------------+ -### **STDDEV\_POP** - -**Description** +#### STDDEV\_POP -Usage: STDDEV\_POP(expr). Return the population standard deviation of -expr. - -Example: +The \`STDDEV\_POP(expr)\` function returns the population standard deviation of a set of values in the expression \`expr\`. The following is an example PPL query: os> source=accounts | stats stddev_pop(age); fetched rows / total rows = 1/1 @@ -183,23 +140,13 @@ Example: | 2.8613807855648994 | +--------------------+ -### **By Clause** +### By clause -The by clause could be the fields and expressions like scalar functions -and aggregation functions. Besides, the span clause can also be used in -the by clause to split specific field into buckets in the same interval, -the stats then does the aggregation by these span buckets. +The \`by\` clause can contain fields, expressions, scalar functions, or aggregation functions. The \`span\` clause can be used in the \`by\` clause to split specific fields into buckets of the same interval. The \`stats\` command then performs the aggregation on these buckets. -The span syntax is \`span(field_expr, interval_expr)\`, the unit of the -interval expression is the natural unit by default. If the field is a -date and time type field, and the interval is in date/time units, you -will need to specify the unit in the interval expression. For example, -to split the field \`age\` into buckets by 10 years, it looks like -\`span(age, 10)\`. And here is another example of time span, the span to -split a \`timestamp\` field into hourly intervals, it looks like -\`span(timestamp, 1h)\`. +The span syntax is \`span(field_expr, interval_expr)\`. By default, the interval expression in the \`span\` clause is interpreted in natural units. If the field is a date and time type field and the interval is in date and time units, you must specify the unit in the interval expression. For example, to split the \`age\` field into buckets of 10 years, you would use \`span(age, 10y). To split a timestamp field into hourly intervals, you would use \`span(timestamp, 1h)\`. -Available time unit: +The following table lists the available time units. | Span Interval Units | |----------------------------| @@ -213,11 +160,13 @@ Available time unit: | quarter (q) | | year (y) | -### Example 1: Calculate the count of events +### PPL queries using the stats command + +The following example PPL queries show ways you can use the \`stats\` command in your queries. -The example show calculate the count of events in the accounts. +#### Example 1: Calculate event counts -PPL query: +The following example PPL query calculates event counts: os> source=accounts | stats count(); fetched rows / total rows = 1/1 @@ -227,11 +176,9 @@ PPL query: | 4 | +-----------+ -### Example 2: Calculate the average of a field - -The example show calculate the average age of all the accounts. +#### Example 2: Calculate a field's average -PPL query: +The following example PPL query calculates the average age: os> source=accounts | stats avg(age); fetched rows / total rows = 1/1 @@ -241,12 +188,9 @@ PPL query: | 32.25 | +------------+ -### Example 3: Calculate the average of a field by group +#### Example 3: Calculate the average of a field by group -The example show calculate the average age of all the accounts group by -gender. - -PPL query: +The following example PPL query calculates the average age grouped by gender: os> source=accounts | stats avg(age) by gender; fetched rows / total rows = 2/2 @@ -257,14 +201,11 @@ PPL query: | 33.666666666666664 | M | +--------------------+----------+ -### Example 4: Calculate the average, sum and count of a field by group - -The example show calculate the average age, sum age and count of events -of all the accounts group by gender. +#### Example 4: Calculate the average, sum, and count of a field by group -PPL query: +The following example PPL query calculates the average age, sum age, and count of events by gender. - os> source=accounts | stats avg(age), sum(age), count() by gender; + os> source=accounts | stats avg(age), sum(age), count() grouped by gender; fetched rows / total rows = 2/2 +--------------------+------------+-----------+----------+ | avg(age) | sum(age) | count() | gender | @@ -273,11 +214,9 @@ PPL query: | 33.666666666666664 | 101 | 3 | M | +--------------------+------------+-----------+----------+ -### Example 5: Calculate the maximum of a field - -The example calculates the max age of all the accounts. +#### Example 5: Calculate a field's maximum -PPL query: +The following example PPL query calculates the maximum age: os> source=accounts | stats max(age); fetched rows / total rows = 1/1 @@ -287,12 +226,9 @@ PPL query: | 36 | +------------+ -### Example 6: Calculate the maximum and minimum of a field by group +#### Example 6: Calculate a field's min/max by group -The example calculates the max and min age values of all the accounts -group by gender. - -PPL query: +The following example PPL query calculates the min/max age grouped by gender: os> source=accounts | stats max(age), min(age) by gender; fetched rows / total rows = 2/2 @@ -303,14 +239,11 @@ PPL query: | 36 | 32 | M | +------------+------------+----------+ -### Example 7: Calculate the distinct count of a field +#### Example 7: Calculate a field's distinct count -To get the count of distinct values of a field, you can use -\`DISTINCT_COUNT\` (or \`DC\`) function instead of \`COUNT\`. The example -calculates both the count and the distinct count of gender field of all -the accounts. +To count the number of distinct values in a field, you can use the \`DISTINCT_COUNT\` or \`DC\` function instead of the \`COUNT\` funtion. -PPL query: +The following PPL query calculates both the count and distinct count of the \`gender\` field for all accounts. os> source=accounts | stats count(gender), distinct_count(gender); fetched rows / total rows = 1/1 @@ -320,11 +253,9 @@ PPL query: | 4 | 2 | +-----------------+--------------------------+ -### Example 8: Calculate the count by a span - -The example gets the count of age by the interval of 10 years. +#### Example 8: Calculate count by span -PPL query: +The following PPL query calculates age by span of 10 years. os> source=accounts | stats count(age) by span(age, 10) as age_span fetched rows / total rows = 2/2 @@ -335,11 +266,9 @@ PPL query: | 3 | 30 | +--------------+------------+ -### Example 9: Calculate the count by a gender and span - -The example gets the count of age by the interval of 10 years and group by gender. +#### Example 9: Calculate count by gender and span -PPL query:: +The following PPL query calculates age by span of 10 years and groups by gender. os> source=accounts | stats count() as cnt by span(age, 5) as age_span, gender fetched rows / total rows = 3/3 diff --git a/public/components/common/helpers/ppl_docs/commands/syntax.ts b/public/components/common/helpers/ppl_docs/commands/syntax.ts index b579ac517..7ba0a1e8f 100644 --- a/public/components/common/helpers/ppl_docs/commands/syntax.ts +++ b/public/components/common/helpers/ppl_docs/commands/syntax.ts @@ -5,12 +5,10 @@ export const syntaxCmd = `## Syntax --- -### Command Order +### Command order -The PPL query started with \`search\` command to reference a table search -from. All the following command could be in any order. In the following -example, \`search\` command refer the accounts index as the source, then -using fields and where command to do the further processing. +The PPL query starts with a \`search\` command to reference a table to search. +Commands can be in any order. For example, in the following query, the \`search\` command references the \`accounts\` index as the source and then uses fields and a \`where\` command to perform further processing. \`\`\` search source=accounts @@ -18,10 +16,7 @@ search source=accounts | fields firstname, lastname \`\`\` -### Required arguments +### Required and optional arguments -Required arguments are shown in angle brackets \< \>. - -### Optional arguments - -Optional arguments are enclosed in square brackets \[ \].`; \ No newline at end of file +Required arguments are enclosed in angle brackets \< \>, and optional arguments are enclosed in square brackets \[ \]`. +; diff --git a/public/components/common/helpers/ppl_docs/commands/top.ts b/public/components/common/helpers/ppl_docs/commands/top.ts index aa7db2958..0cf6b7c8b 100644 --- a/public/components/common/helpers/ppl_docs/commands/top.ts +++ b/public/components/common/helpers/ppl_docs/commands/top.ts @@ -7,38 +7,20 @@ export const topCmd = `## top --- ### Description -Using \`top\` command to find the most common tuple of values of all +Use the \`top\` command to find the most common tuple of values for all fields in the field list. ### Syntax top \[N\] <field-list> \[by-clause\] -- N: number of results to return. **Default**: 10 -- field-list: mandatory. comma-delimited list of field names. -- by-clause: optional. one or more fields to group the results by. +- \`N\`: The number of results you want to return. Default is 10. +- \`field-list\`: Required. The comma-delimited field list. +- \`by-clause\`: Optional. One or more fields to group by. -### Example 1: Find the most common values in a field +#### Example 1: Find the most common values in a field -The example finds most common gender of all the accounts. - -PPL query: - - os> source=accounts | top gender; - fetched rows / total rows = 2/2 - +------------+ - | gender | - |------------| - | M | - |------------| - | F | - +------------+ - -### Example 2: Find the most common values in a field - -The example finds most common gender of all the accounts. - -PPL query: +The following example PPL query finds the most common gender. os> source=accounts | top 1 gender; fetched rows / total rows = 1/1 @@ -48,11 +30,9 @@ PPL query: | M | +------------+ -### Example 3: Find the most common values organized by gender - -The example finds most common age of all the accounts group by gender. +#### Example 2: Find the most common values grouped by gender -PPL query: +The following example PPL query finds the most common age grouped by gender. os> source=accounts | top 1 age by gender; fetched rows / total rows = 2/2 @@ -64,5 +44,5 @@ PPL query: +----------+----------+ #### Limitation -The \`top\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node. +The \`top\` command is not rewritten to [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/). It is only run on the coordinating node. `; diff --git a/public/components/common/helpers/ppl_docs/commands/where.ts b/public/components/common/helpers/ppl_docs/commands/where.ts index 9adbcb739..b92d65b01 100644 --- a/public/components/common/helpers/ppl_docs/commands/where.ts +++ b/public/components/common/helpers/ppl_docs/commands/where.ts @@ -8,22 +8,17 @@ export const whereCmd = `## where ### Description -The \`where\` command bool-expression to filter the search result. The -\`where\` command only return the result when bool-expression evaluated to -true. +Use the \`where\` command to filter search results. The \`where\` command only returns the result when the \`bool-expression\` is set to \`true\`. ### Syntax -where <boolean-expression> +\`where <boolean-expression>\` -- bool-expression: optional. any expression which could be evaluated - to boolean value. +- \`bool-expression\`: Optional. Any expression that can be evaluated to a Boolean expression. -### Example 1: Filter result set with condition +#### Example 1: Filter the result set with a condition -The example show fetch all the document from accounts index with . - -PPL query: +The following example PPL query fetches all documents from the \`accounts\` index using an \`or\ condition. os> source=accounts | where account_number=1 or gender="F" | fields account_number, gender; fetched rows / total rows = 2/2 diff --git a/public/components/common/helpers/ppl_docs/functions/condition.ts b/public/components/common/helpers/ppl_docs/functions/condition.ts index 85e3ed074..4824b5aa2 100644 --- a/public/components/common/helpers/ppl_docs/functions/condition.ts +++ b/public/components/common/helpers/ppl_docs/functions/condition.ts @@ -6,15 +6,21 @@ export const conditionFunction = `## Condition --- -### Description +### Condition functions -Usage: isnull(field) return true if field is null. +PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitate the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause. -Argument type: all the supported data type. +The following sections describe the condition PPL functions. -Return type: BOOLEAN +### ISNULL -Example: +The \`isnull(field)\` function checks a specific field and returns \`true\` if the field contains no data, that is, it's null. + +**Argument type:** All supported data types + +**Return type:** \`BOOLEAN\` + +#### Example os> source=accounts | eval result = isnull(employer) | fields result, employer, firstname fetched rows / total rows = 4/4 @@ -27,17 +33,15 @@ Example: | True | null | Dale | +----------+------------+-------------+ -### **ISNOTNULL** +### ISNOTNULL -**Description** +The \`isnotnull(field)\` function is the opposite of \`isnull(field)\`. Instead of checking for null values, it checks a specific field and returns \`true\` if the field contains data, that is, it is not null. -Usage: isnotnull(field) return true if field is not null. +**Argument type:** All supported data types -Argument type: all the supported data type. +**Return type:** \`BOOLEAN\` -Return type: BOOLEAN - -Example: +#### Example os> source=accounts | where not isnotnull(employer) | fields account_number, employer fetched rows / total rows = 1/1 @@ -47,14 +51,11 @@ Example: | 18 | null | +------------------+------------+ -### **EXISTS** +### EXISTS -[Because OpenSearch doesn't differentiate null and -missing](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). -so we can't provide function like ismissing/isnotmissing to test field -exist or not. But you can still use isnull/isnotnull for such purpose. +OpenSearch does not differentiate between null and missing. Thus, a function such as \`ismissing\` or \`isnotmissing\` cannot be used to test if a field exists or not. The \`isnull\` or \`isnotnull\` functions can be used for this purpose. -Example, the account 13 doesn't have email field: +#### Example os> source=accounts | where isnull(email) | fields account_number, email fetched rows / total rows = 1/1 @@ -64,18 +65,15 @@ Example, the account 13 doesn't have email field: | 13 | null | +------------------+---------+ -### **IFNULL** +### IFNULL -**Description** +The \`ifnull(field1, field2)\` function returns the value in the first field if it is not null; otherwise, it returns the value in the second field. -Usage: ifnull(field1, field2) return field2 if field1 is null. +**Argument type:** All supported data types (Note that the semantic check will fail if the parameters are different types.) -Argument type: all the supported data type, (NOTE : if two parameters -has different type, you will fail semantic check.) +**Return type:** Any -Return type: any - -Example: +#### Example os> source=accounts | eval result = ifnull(employer, 'default') | fields result, employer, firstname fetched rows / total rows = 4/4 @@ -88,20 +86,15 @@ Example: | default | null | Dale | +----------+------------+-------------+ -### **NULLIF** - -**Description** +### NULLIF -Usage: nullif(field1, field2) return null if two parameters are same, -otherwiser return field1. +The \`nullif(field1, field2)\` function returns \`null\` if the values in both fields are identical. If the values differ, the function returns the value in the first field (field1). -Argument type: all the supported data type, (NOTE : if two parameters -has different type, if two parameters has different type, you will fail -semantic check) +**Argument type:** All supported data types (Note that the semantic check will fail if the parameters are different types.) -Return type: any +**Return type:** Any -Example: +#### Example os> source=accounts | eval result = nullif(employer, 'Pyrami') | fields result, employer, firstname fetched rows / total rows = 4/4 @@ -114,18 +107,15 @@ Example: | null | null | Dale | +----------+------------+-------------+ -### **ISNULL** +### ISNULL -**Description** +The \`isnull(field1, field2)\` function checks for null values and returns \`null\` if the values in both fields are identical. If the values differ, the function returns the value in the first field (field1). -Usage: isnull(field1, field2) return null if two parameters are same, -otherwise return field1. +**Argument type:** All supported data types -Argument type: all the supported data type +**Return type:** Any -Return type: any - -Example: +#### Example os> source=accounts | eval result = isnull(employer) | fields result, employer, firstname fetched rows / total rows = 4/4 @@ -138,17 +128,13 @@ Example: | True | null | Dale | +----------+------------+-------------+ -### **IF** - -**Description** +### IF -Usage: if(condition, expr1, expr2) return expr1 if condition is true, -otherwiser return expr2. +The \`if(condition, expr1, expr2)\` function returns \`expr1\` if \`condition\` is \`true\`, and \`expr2\` otherwise. -Argument type: all the supported data type, (NOTE : if expr1 and expr2 -are different type, you will fail semantic check +**Argument type:** All supported data types (Note that the semantic check will fail if \`expr1\` and \`expr2\` have different types.) -Return type: any +**Return type:** Any Example: diff --git a/public/components/common/helpers/ppl_docs/functions/datetime.ts b/public/components/common/helpers/ppl_docs/functions/datetime.ts index a593b1c56..5b2ca67f5 100644 --- a/public/components/common/helpers/ppl_docs/functions/datetime.ts +++ b/public/components/common/helpers/ppl_docs/functions/datetime.ts @@ -6,27 +6,23 @@ export const datetimeFunction = `## Datetime --- -### **ADDDATE** +### Datetime functions -Description +PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitate the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause. -Usage: adddate(date, INTERVAL expr unit)/ adddate(date, expr) adds the -time interval of second argument to date; adddate(date, days) adds the -second argument as integer number of days to date. +The following sections describe the \`datetime\` PPL functions. -Argument type: DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG +### ADDDATE -Return type map: +The \`adddate\` function add a time interval to a date. It supports two forms: adding a specified interval using \`INTERVAL\` keyword or adding an integer number of days directly. -(DATE/DATETIME/TIMESTAMP/STRING, INTERVAL) -> DATETIME +**Argument type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG\` -(DATE, LONG) -> DATE +**Return type:** \`(DATE/DATETIME/TIMESTAMP/STRING, INTERVAL) -> DATETIME\`, \`(DATE, LONG) -> DATE\`, \`(DATETIME/TIMESTAMP/STRING, LONG) -> DATETIME\` -(DATETIME/TIMESTAMP/STRING, LONG) -> DATETIME +**Synonyms**: \`[DATE\_ADD](#date_add)\` -Synonyms: [DATE\_ADD](#date_add) - -Example: +#### Example os> source=people | eval \`ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR)\` = ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR), \`ADDDATE(DATE('2020-08-26'), 1)\` = ADDDATE(DATE('2020-08-26'), 1), \`ADDDATE(TIMESTAMP('2020-08-26 01:01:01'), 1)\` = ADDDATE(TIMESTAMP('2020-08-26 01:01:01'), 1) | fields \`ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR)\`, \`ADDDATE(DATE('2020-08-26'), 1)\`, \`ADDDATE(TIMESTAMP('2020-08-26 01:01:01'), 1)\` fetched rows / total rows = 1/1 @@ -36,19 +32,15 @@ Example: | 2020-08-26 01:00:00 | 2020-08-27 | 2020-08-27 01:01:01 | +------------------------------------------------+----------------------------------+------------------------------------------------+ -### **DATE** - -**Description** +#### DATE -Usage: date(expr) constructs a date type with the input string expr as a -date. If the argument is of date/datetime/timestamp, it extracts the -date value part from the expression. +The \`date(expr)\` function converts strings to date types and extracts the date portion from existing date, datetime, and timestamp values. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: DATE +**Return type:** \`DATE\` -Example: +#### Example >od source=people | eval \`DATE('2020-08-26')\` = DATE('2020-08-26'), \`DATE(TIMESTAMP('2020-08-26 13:49:00'))\` = DATE(TIMESTAMP('2020-08-26 13:49:00')) | fields \`DATE('2020-08-26')\`, \`DATE(TIMESTAMP('2020-08-26 13:49:00'))\` fetched rows / total rows = 1/1 @@ -58,26 +50,18 @@ Example: | DATE '2020-08-26' | DATE '2020-08-26' | +----------------------+------------------------------------------+ -### **DATE\_ADD** - -**Description** - -Usage: date\_add(date, INTERVAL expr unit)/ date\_add(date, expr) adds -the time interval expr to date - -Argument type: DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG - -Return type map: +#### DATE\_ADD -DATE/DATETIME/TIMESTAMP/STRING, INTERVAL -> DATETIME +The \`date\_add(date, INTERVAL expr unit)\` or \`date\_add(date, expr)\` adds +the time interval specified by \`expr\` to a given \`date\`. It supports adding a specific interval and adding an integer number of days. -DATE, LONG -> DATE +**Argument type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG\` -DATETIME/TIMESTAMP/STRING, LONG -> DATETIME +**Return type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL ->\`, \`DATETIME\`, \`DATE, LONG -> DATE\`, \`DATETIME/TIMESTAMP/STRING, LONG -> DATETIME\` -Synonyms: [ADDDATE](#adddate) +**Synonyms:** \`[ADDDATE](#adddate)\` -Example: +#### Example os> source=people | eval \`DATE_ADD(DATE('2020-08-26'), INTERVAL 1 HOUR)\` = DATE_ADD(DATE('2020-08-26'), INTERVAL 1 HOUR), \`DATE_ADD(DATE('2020-08-26'), 1)\` = DATE_ADD(DATE('2020-08-26'), 1), \`DATE_ADD(TIMESTAMP('2020-08-26 01:01:01'), 1)\` = DATE_ADD(TIMESTAMP('2020-08-26 01:01:01'), 1) | fields \`DATE_ADD(DATE('2020-08-26'), INTERVAL 1 HOUR)\`, \`DATE_ADD(DATE('2020-08-26'), 1)\`, \`DATE_ADD(TIMESTAMP('2020-08-26 01:01:01'), 1)\` fetched rows / total rows = 1/1 @@ -87,56 +71,53 @@ Example: | 2020-08-26 01:00:00 | 2020-08-27 | 2020-08-27 01:01:01 | +-------------------------------------------------+-----------------------------------+-------------------------------------------------+ -### **DATE\_FORMAT** - -**Description** - -Usage: date\_format(date, format) formats the date argument using the -specifiers in the format argument. - -| Specifier | Description | -|-----------|--------------------------------------------------------------------------------------------------| -| %a | Abbreviated weekday name (Sun..Sat) | -| %b | Abbreviated month name (Jan..Dec) | -| %c | Month, numeric (0..12) | -| %D | Day of the month with English suffix (0th, 1st, 2nd, 3rd, …) | -| %d | Day of the month, numeric (00..31) | -| %e | Day of the month, numeric (0..31) | -| %f | Microseconds (000000..999999) | -| %H | Hour (00..23) | -| %h | Hour (01..12) | -| %I | Hour (01..12) | -| %i | Minutes, numeric (00..59) | -| %j | Day of year (001..366) | -| %k | Hour (0..23) | -| %l | Hour (1..12) | -| %M | Month name (January..December) | -| %m | Month, numeric (00..12) | -| %p | AM or PM | -| %r | Time, 12-hour (hh:mm:ss followed by AM or PM) | -| %S | Seconds (00..59) | -| %s | Seconds (00..59) | -| %T | Time, 24-hour (hh:mm:ss) | -| %U | Week (00..53), where Sunday is the first day of the week; WEEK() mode 0 | -| %u | Week (00..53), where Monday is the first day of the week; WEEK() mode 1 | -| %V | Week (01..53), where Sunday is the first day of the week; WEEK() mode 2; used with %X | -| %v | Week (01..53), where Monday is the first day of the week; WEEK() mode 3; used with %x | -| %W | Weekday name (Sunday..Saturday) | -| %w | Day of the week (0=Sunday..6=Saturday) | -| %X | Year for the week where Sunday is the first day of the week, numeric, four digits; used with %V | -| %x | Year for the week, where Monday is the first day of the week, numeric, four digits; used with %v | -| %Y | Year, numeric, four digits | -| %y | Year, numeric (two digits) | -| %% | A literal % character | -| %x | x, for any “x” not listed above | - -The following table describes the available specifier arguments. - -Argument type: STRING/DATE/DATETIME/TIMESTAMP, STRING - -Return type: STRING - -Example: +### DATE\_FORMAT + +The \`date\_format(date, format)\` function takes a date and a format string as arguments and returns the formatted date string according to the specified format. + +The following table lists the available specifier arguments. + +| Specifier | Description | +|-----------|-----------------------------------------------------------| +| %a | Abbreviated weekday name (Sun..Sat) | +| %b | Abbreviated month name (Jan..Dec) | +| %c | Month, numeric (0..12) | +| %D | Day of the month with English suffix (0th, 1st, 2nd, 3rd, …) | +| %d | Day of the month, numeric (00..31) | +| %e | Day of the month, numeric (0..31) | +| %f | Microseconds (000000..999999) | +| %H | Hour (00..23) | +| %h | Hour (01..12) | +| %I | Hour (01..12) | +| %i | Minutes, numeric (00..59) | +| %j | Day of year (001..366) | +| %k | Hour (0..23) | +| %l | Hour (1..12) | +| %M | Month name (January..December) | +| %m | Month, numeric (00..12) | +| %p | AM or PM | +| %r | Time, 12-hour (hh:mm:ss followed by AM or PM) | +| %S | Seconds (00..59) | +| %s | Seconds (00..59) | +| %T | Time, 24-hour (hh:mm:ss) | +| %U | Week (00..53), where Sunday is the first day of the week; WEEK() mode 0 | +| %u | Week (00..53), where Monday is the first day of the week; WEEK() mode 1 | +| %V | Week (01..53), where Sunday is the first day of the week; WEEK() mode 2; used with %X | +| %v | Week (01..53), where Monday is the first day of the week; WEEK() mode 3; used with %x | +| %W | Weekday name (Sunday..Saturday) | +| %w | Day of the week (0=Sunday..6=Saturday) | +| %X | Year for the week where Sunday is the first day of the week, numeric, four digits; used with %V | +| %x | Year for the week, where Monday is the first day of the week, numeric, four digits; used with %v | +| %Y | Year, numeric, four digits | +| %y | Year, numeric (two digits) | +| %% | A literal % character | +| %x | x, for any “x” not listed above | + +**Argument type:** STRING/DATE/DATETIME/TIMESTAMP, STRING + +**Return type:** STRING + +#### Example >od source=people | eval \`DATE_FORMAT('1998-01-31 13:14:15.012345', '%T.%f')\` = DATE_FORMAT('1998-01-31 13:14:15.012345', '%T.%f'), \`DATE_FORMAT(TIMESTAMP('1998-01-31 13:14:15.012345'), '%Y-%b-%D %r')\` = DATE_FORMAT(TIMESTAMP('1998-01-31 13:14:15.012345'), '%Y-%b-%D %r') | fields \`DATE_FORMAT('1998-01-31 13:14:15.012345', '%T.%f')\`, \`DATE_FORMAT(TIMESTAMP('1998-01-31 13:14:15.012345'), '%Y-%b-%D %r')\` fetched rows / total rows = 1/1 @@ -146,26 +127,20 @@ Example: | '13:14:15.012345' | '1998-Jan-31st 01:14:15 PM' | +-----------------------------------------------+----------------------------------------------------------------+ -### **DATE\_SUB** +### DATE\_SUB **Description** Usage: date\_sub(date, INTERVAL expr unit)/ date\_sub(date, expr) subtracts the time interval expr from date -Argument type: DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG - -Return type map: +Argument type: \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG\` -DATE/DATETIME/TIMESTAMP/STRING, INTERVAL -> DATETIME +**Return type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL -> DATETIME\`, \`DATE, LONG -> DATE\`, \`DATETIME/TIMESTAMP/STRING, LONG -> DATETIME\` -DATE, LONG -> DATE +**Synonyms:** \`[SUBDATE](#subdate)\` -DATETIME/TIMESTAMP/STRING, LONG -> DATETIME - -Synonyms: [SUBDATE](#subdate) - -Example: +#### Example os> source=people | eval \`DATE_SUB(DATE('2008-01-02'), INTERVAL 31 DAY)\` = DATE_SUB(DATE('2008-01-02'), INTERVAL 31 DAY), \`DATE_SUB(DATE('2020-08-26'), 1)\` = DATE_SUB(DATE('2020-08-26'), 1), \`DATE_SUB(TIMESTAMP('2020-08-26 01:01:01'), 1)\` = DATE_SUB(TIMESTAMP('2020-08-26 01:01:01'), 1) | fields \`DATE_SUB(DATE('2008-01-02'), INTERVAL 31 DAY)\`, \`DATE_SUB(DATE('2020-08-26'), 1)\`, \`DATE_SUB(TIMESTAMP('2020-08-26 01:01:01'), 1)\` fetched rows / total rows = 1/1 @@ -175,21 +150,17 @@ Example: | 2007-12-02 | 2020-08-25 | 2020-08-25 01:01:01 | +-------------------------------------------------+-----------------------------------+-------------------------------------------------+ -### **DAY** - -**Description** +### DAY -Usage: day(date) extracts the day of the month for date, in the range 1 -to 31. The dates with value 0 such as '0000-00-00' or '2008-00-00' are -invalid. +The \`day(date)\` function retrieves the day of the month (1-31) for a provided \`date\`. Note that dated with a value of 0, such as "0000-00-00" or "2008-00-00", are considered invalid. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Synonyms: DAYOFMONTH +**Synonyms:** \`DAYOFMONTH\` -Example: +#### Example os> source=people | eval \`DAY(DATE('2020-08-26'))\` = DAY(DATE('2020-08-26')) | fields \`DAY(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -199,18 +170,15 @@ Example: | 26 | +---------------------------+ -### **DAYNAME** +### DAYNAME -**Description** - -Usage: dayname(date) returns the name of the weekday for date, including -Monday, Tuesday, Wednesday, Thursday, Friday, Saturday and Sunday. +The \`dayname(date)\` function retrieves the full name of the weekday, for example, Monday, Tuesday, and so forth, for a given \`date\`. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`DAYNAME(DATE('2020-08-26'))\` = DAYNAME(DATE('2020-08-26')) | fields \`DAYNAME(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -220,21 +188,17 @@ Example: | Wednesday | +-------------------------------+ -### **DAYOFMONTH** - -**Description** +### DAYOFMONTH -Usage: dayofmonth(date) extracts the day of the month for date, in the -range 1 to 31. The dates with value 0 such as '0000-00-00' or -'2008-00-00' are invalid. +The \`dayofmonth(date)\` function retrieves the day of the month (1-31) for a provided \`date\`. Note that dated with a value of 0, such as "0000-00-00" or "2008-00-00", are considered invalid. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Synonyms: DAY +**Synonyms:** \`DAY\` -Example: +#### Example os> source=people | eval \`DAYOFMONTH(DATE('2020-08-26'))\` = DAYOFMONTH(DATE('2020-08-26')) | fields \`DAYOFMONTH(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -244,18 +208,15 @@ Example: | 26 | +----------------------------------+ -### **DAYOFWEEK** +### DAYOFWEEK -**Description** - -Usage: dayofweek(date) returns the weekday index for date (1 = Sunday, 2 -= Monday, …, 7 = Saturday). +The \`dayofweek(date)\` retrieves the numerical index (1-7) representing the weekday for a given \`date\`, where 1 corresponds to Sunday and 7 corresponds to Saturday. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`DAYOFWEEK(DATE('2020-08-26'))\` = DAYOFWEEK(DATE('2020-08-26')) | fields \`DAYOFWEEK(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -265,18 +226,15 @@ Example: | 4 | +---------------------------------+ -### **DAYOFYEAR** - -**Description** +### DAYOFYEAR -Usage: dayofyear(date) returns the day of the year for date, in the -range 1 to 366. +The \`dayofyear(date)\` function retrieves the day of the year for a given \`date\`, ranging from 1 to 366. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`DAYOFYEAR(DATE('2020-08-26'))\` = DAYOFYEAR(DATE('2020-08-26')) | fields \`DAYOFYEAR(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -286,17 +244,15 @@ Example: | 239 | +---------------------------------+ -### **FROM\_DAYS** +### FROM\_DAYS -**Description** - -Usage: from\_days(N) returns the date value given the day number N. +The \`from\_days(N)\` function retrieves the date value corresponding to the provided day number \`N\`. -Argument type: INTEGER/LONG +**Argument type:** \`INTEGER/LONG\` -Return type: DATE +**Return type:** \`DATE\` -Example: +#### Example os> source=people | eval \`FROM_DAYS(733687)\` = FROM_DAYS(733687) | fields \`FROM_DAYS(733687)\` fetched rows / total rows = 1/1 @@ -306,19 +262,15 @@ Example: | 2008-10-07 | +---------------------+ -### **HOUR** - -**Description** +### HOUR -Usage: hour(time) extracts the hour value for time. Different from the -time of day value, the time value has a large range and can be greater -than 23, so the return value of hour(time) can be also greater than 23. +The \`hour(time)\` function extracts the hour value from a given \`time\`. Unlike the typical time-of-day format wher hours range from 0 to 23, the \`time\` input can have a larger range. Therefore, the \`hour(time)\` function may return values exceeding 23. -Argument type: STRING/TIME/DATETIME/TIMESTAMP +**Argument type:** \`STRING/TIME/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`HOUR(TIME('01:02:03'))\` = HOUR(TIME('01:02:03')) | fields \`HOUR(TIME('01:02:03'))\` fetched rows / total rows = 1/1 @@ -328,26 +280,19 @@ Example: | 1 | +--------------------------+ -### **MAKETIME** - -**Description** - -Specifications: - -1. MAKETIME(INTEGER, INTEGER, INTEGER) -> DATE +### MAKETIME -### **MICROSECOND** +**Function signature:** \`MAKETIME(INTEGER, INTEGER, INTEGER) -> DATE\` -**Description** +### MICROSECOND -Usage: microsecond(expr) returns the microseconds from the time or -datetime expression expr as a number in the range from 0 to 999999. +The \`microsecond(expr)\` function retrieves the microsecond portion (0-999999) from a given \`time\` or \`datetime\` expression. -Argument type: STRING/TIME/DATETIME/TIMESTAMP +**Argument type:** \`STRING/TIME/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`MICROSECOND(TIME('01:02:03.123456'))\` = MICROSECOND(TIME('01:02:03.123456')) | fields \`MICROSECOND(TIME('01:02:03.123456'))\` fetched rows / total rows = 1/1 @@ -357,17 +302,15 @@ Example: | 123456 | +----------------------------------------+ -### **MINUTE** - -**Description** +### MINUTE -Usage: minute(time) returns the minute for time, in the range 0 to 59. +The \`minute(time)\` extracts the minute value (0-59) from a given \`time\` expression. -Argument type: STRING/TIME/DATETIME/TIMESTAMP +**Argument type:** \`STRING/TIME/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`MINUTE(TIME('01:02:03'))\` = MINUTE(TIME('01:02:03')) | fields \`MINUTE(TIME('01:02:03'))\` fetched rows / total rows = 1/1 @@ -377,19 +320,15 @@ Example: | 2 | +----------------------------+ -### **MONTH** - -**Description** +### MONTH -Usage: month(date) returns the month for date, in the range 1 to 12 for -January to December. The dates with value 0 such as '0000-00-00' or -'2008-00-00' are invalid. +The \`month(date)\` function extracts the month (1-12) from a valid \`date\` value. However, invalid dates containing 0 values for the month, such as "0000-00-00" or "2008-00-00" are considered invalid. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`MONTH(DATE('2020-08-26'))\` = MONTH(DATE('2020-08-26')) | fields \`MONTH(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -399,17 +338,15 @@ Example: | 8 | +-----------------------------+ -### **MONTHNAME** - -**Description** +### MONTHNAME -Usage: monthname(date) returns the full name of the month for date. +The \`monthname(date)\` function retrieves the full name of the month, for example, January, February, and so forth, for a given \`date\`. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`MONTHNAME(DATE('2020-08-26'))\` = MONTHNAME(DATE('2020-08-26')) | fields \`MONTHNAME(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -419,26 +356,19 @@ Example: | August | +---------------------------------+ -### **NOW** - -**Description** - -Specifications: +### NOW -1. NOW() -> DATE +**Function signature:** NOW() -> DATE -### **QUARTER** - -**Description** +### QUARTER -Usage: quarter(date) returns the quarter of the year for date, in the -range 1 to 4. +The \`quarter(date)\` function retrieves the quarter (1-4) for a given \`date\`. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`QUARTER(DATE('2020-08-26'))\` = QUARTER(DATE('2020-08-26')) | fields \`QUARTER(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 @@ -448,17 +378,15 @@ Example: | 3 | +-------------------------------+ -### **SECOND** +### SECOND -**Description** - -Usage: second(time) returns the second for time, in the range 0 to 59. +The \`second(time)\` function extracts the second value (0-59) from a given \`time\` expression. -Argument type: STRING/TIME/DATETIME/TIMESTAMP +**Argument type:** \`STRING/TIME/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`SECOND(TIME('01:02:03'))\` = SECOND(TIME('01:02:03')) | fields \`SECOND(TIME('01:02:03'))\` fetched rows / total rows = 1/1 @@ -468,26 +396,17 @@ Example: | 3 | +----------------------------+ -### **SUBDATE** - -**Description** - -Usage: subdate(date, INTERVAL expr unit)/ subdate(date, expr) subtracts -the time interval expr from date - -Argument type: DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG +### SUBDATE -Return type map: +The \`subdate(date, INTERVAL expr unit)\` or \`subdate(date, expr)\` function subtracts a time interval from a date. -DATE/DATETIME/TIMESTAMP/STRING, INTERVAL -> DATETIME +**Argument type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL/LONG\` -DATE, LONG -> DATE +**Return type:** \`DATE/DATETIME/TIMESTAMP/STRING, INTERVAL -> DATETIME\`, \`DATE, LONG -> DATE\`, \`DATETIME/TIMESTAMP/STRING, LONG -> DATETIME\` -DATETIME/TIMESTAMP/STRING, LONG -> DATETIME +**Synonyms:** \`[DATE\_SUB](#date_sub)\` -Synonyms: [DATE\_SUB](#date_sub) - -Example: +#### Example os> source=people | eval \`SUBDATE(DATE('2008-01-02'), INTERVAL 31 DAY)\` = SUBDATE(DATE('2008-01-02'), INTERVAL 31 DAY), \`SUBDATE(DATE('2020-08-26'), 1)\` = SUBDATE(DATE('2020-08-26'), 1), \`SUBDATE(TIMESTAMP('2020-08-26 01:01:01'), 1)\` = SUBDATE(TIMESTAMP('2020-08-26 01:01:01'), 1) | fields \`SUBDATE(DATE('2008-01-02'), INTERVAL 31 DAY)\`, \`SUBDATE(DATE('2020-08-26'), 1)\`, \`SUBDATE(TIMESTAMP('2020-08-26 01:01:01'), 1)\` fetched rows / total rows = 1/1 @@ -497,19 +416,15 @@ Example: | 2007-12-02 | 2020-08-25 | 2020-08-25 01:01:01 | +------------------------------------------------+----------------------------------+------------------------------------------------+ -### **TIME** - -**Description** +### TIME -Usage: time(expr) constructs a time type with the input string expr as a -time. If the argument is of date/datetime/time/timestamp, it extracts -the time value part from the expression. +The \`time(expr)\` function has dual functionality. If \`expr\` is a string, it contructs a \`time\` object from the provided time value format. Conversly, for input of the type \`date\`, \`datetime\`. \`time\`, or \`timestamp\`, it extracts and returns the pure time component from the given expression. -Argument type: STRING/DATE/DATETIME/TIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIME/TIMESTAMP\` -Return type: TIME +**Return type:** \`TIME\` -Example: +#### Example >od source=people | eval \`TIME('13:49:00')\` = TIME('13:49:00'), \`TIME(TIMESTAMP('2020-08-26 13:49:00'))\` = TIME(TIMESTAMP('2020-08-26 13:49:00')) | fields \`TIME('13:49:00')\`, \`TIME(TIMESTAMP('2020-08-26 13:49:00'))\` fetched rows / total rows = 1/1 @@ -519,18 +434,15 @@ Example: | TIME '13:49:00' | TIME '13:49:00' | +--------------------+------------------------------------------+ -### **TIME\_TO\_SEC** - -**Description** +### TIME\_TO\_SEC -Usage: time\_to\_sec(time) returns the time argument, converted to -seconds. +The \`time\_to\_sec(time)\` function transforms a given \`time\` value into its corresponding number of seconds. -Argument type: STRING/TIME/DATETIME/TIMESTAMP +**Argument type:** \`STRING/TIME/DATETIME/TIMESTAMP\` -Return type: LONG +**Return type:** \`LONG\` -Example: +#### Example os> source=people | eval \`TIME_TO_SEC(TIME('22:23:00'))\` = TIME_TO_SEC(TIME('22:23:00')) | fields \`TIME_TO_SEC(TIME('22:23:00'))\` fetched rows / total rows = 1/1 @@ -540,19 +452,15 @@ Example: | 80580 | +---------------------------------+ -### **TIMESTAMP** - -**Description** +### TIMESTAMP -Usage: timestamp(expr) construct a timestamp type with the input string -expr as an timestamp. If the argument is of date/datetime/timestamp -type, cast expr to timestamp type with default timezone UTC. +The \`timestamp(expr)\` function serves a dual purpose: it can both construct a timestamp object from a string representing a time value or act as a caster, converting exsiting date, datetime, or timestamp objects to a standardized timestamp type with the default UTC time zone. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: TIMESTAMP +**Return type:** \`TIMESTAMP\` -Example: +#### Example >od source=people | eval \`TIMESTAMP('2020-08-26 13:49:00')\` = TIMESTAMP('2020-08-26 13:49:00') | fields \`TIMESTAMP('2020-08-26 13:49:00')\` fetched rows / total rows = 1/1 @@ -562,18 +470,15 @@ Example: | TIMESTAMP '2020-08-26 13:49:00 | +------------------------------------+ -### **TO\_DAYS** - -**Description** +### TO\_DAYS -Usage: to\_days(date) returns the day number (the number of days since -year 0) of the given date. Returns NULL if date is invalid. +The \`to\_days(date)\` function calculates the number of days that have elapsed since the year 0 for a given \`date\`. If the provided date is invalid, it returns \`NULL\`. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: LONG +**Return type:** \`LONG\` -Example: +#### Example os> source=people | eval \`TO_DAYS(DATE('2008-10-07'))\` = TO_DAYS(DATE('2008-10-07')) | fields \`TO_DAYS(DATE('2008-10-07'))\` fetched rows / total rows = 1/1 @@ -583,12 +488,9 @@ Example: | 733687 | +-------------------------------+ -### **WEEK** - -**Description** +### WEEK -Usage: week(date\[, mode\]) returns the week number for date. If the -mode argument is omitted, the default mode 0 is used. +The \`week(date\[, mode\])\` function extracts the week number for a given \`date\`. If the mode argument is omitted, the default mode 0 is used. The following table lists the mode arguments. | Mode | First day of week | Range | Week 1 is the first week … | |------|-------------------|-------|-------------------------------| @@ -601,13 +503,11 @@ mode argument is omitted, the default mode 0 is used. | 6 | Sunday | 1-53 | with 4 or more days this year | | 7 | Monday | 1-53 | with a Monday in this year | -The following table describes how the mode argument works. - -Argument type: DATE/DATETIME/TIMESTAMP/STRING +**Argument type:** \`DATE/DATETIME/TIMESTAMP/STRING\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example >od source=people | eval \`WEEK(DATE('2008-02-20'))\` = WEEK(DATE('2008-02-20')), \`WEEK(DATE('2008-02-20'), 1)\` = WEEK(DATE('2008-02-20'), 1) | fields \`WEEK(DATE('2008-02-20'))\`, \`WEEK(DATE('2008-02-20'), 1)\` fetched rows / total rows = 1/1 @@ -617,18 +517,15 @@ Example: | 7 | 8 | +----------------------------+-------------------------------+ -### **YEAR** - -**Description** +### YEAR -Usage: year(date) returns the year for date, in the range 1000 to 9999, -or 0 for the “zero” date. +The \`year(date)\` function extracts the year component from a given \`date\` value. However, it only returns valid years within the range of 1000 to 9999. If the provided date is invalid or falls outside this range, the function returns 0. -Argument type: STRING/DATE/DATETIME/TIMESTAMP +**Argument type:** \`STRING/DATE/DATETIME/TIMESTAMP\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`YEAR(DATE('2020-08-26'))\` = YEAR(DATE('2020-08-26')) | fields \`YEAR(DATE('2020-08-26'))\` fetched rows / total rows = 1/1 diff --git a/public/components/common/helpers/ppl_docs/functions/full_text_search.ts b/public/components/common/helpers/ppl_docs/functions/full_text_search.ts index 4704df6ad..7f31bae65 100644 --- a/public/components/common/helpers/ppl_docs/functions/full_text_search.ts +++ b/public/components/common/helpers/ppl_docs/functions/full_text_search.ts @@ -3,28 +3,22 @@ * SPDX-License-Identifier: Apache-2.0 */ -export const fullTextSearchFunction = `## Full Text Search +export const fullTextSearchFunction = `## Full-text search --- -The full text search based functions enable users to search the index for -documents by the full text search of the input query. The functions are built -on the top of the search queries of the OpenSearch engine, but in memory -execution within the plugin is not supported. These functions are able -to perform the global filter of a query, for example the condition -expression in a \`WHERE\` clause or in a \`HAVING\` clause. For more details -of the full text search based search, check out the design here: [Relevance -Based Search With SQL/PPL Query -Engine](https://github.com/opensearch-project/sql/issues/182) +### Full-text search function + +PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitate the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause. + +Full-text search allows for searching by full-text queries. For details about full-text search in OpenSearch, see the [Full-text search](https://opensearch.org/docs/latest/search-plugins/sql/full-text/) documentation. ### MATCH -**Description** +The \`match\` function maps user-defined criteria to OpenSearch queries, returning documents that match specific text, number, date, or Boolean values. -\`match(field_expression, query_expression[, option=]*)\` +The function signature is \`match(field_expression, query_expression[, option=]*)\`. -The match function maps to the match query used in search engine, to -return the documents that match a provided text, number, date or boolean -value with a given field. Available parameters include: +The available parameters are: - analyzer - auto\_generate\_synonyms\_phrase @@ -39,8 +33,9 @@ value with a given field. Available parameters include: - zero\_terms\_query - boost -Example with only \`field\` and \`query\` expressions, and all other -parameters are set default values: +**Example 1: Using specific expressions and default values** + +The following example PPL query uses only the \`field\` and \`query\` expressions, with all other parameters set to their default values: os> source=accounts | where match(address, 'Street') | fields lastname, address; fetched rows / total rows = 2/2 @@ -51,8 +46,9 @@ parameters are set default values: | Bates | 789 Madison Street | +------------+--------------------+ -Another example to show how to set custom values for the optional -parameters: +**Example 2: Setting custom values for optional parameters** + +The following example PPL query sets custom values for the optional parameters: os> source=accounts | where match(firstname, 'Hattie', operator='AND', boost=2.0) | fields lastname; fetched rows / total rows = 1/1 @@ -64,19 +60,19 @@ parameters: ### Limitations -The full text search functions are available to execute only in OpenSearch DSL -but not in memory as of now, so the full text search might fail for -queries that are too complex to translate into DSL if the full text search -function is following after a complex PPL query. To make your queries -always work-able, it is recommended to place the full text search commands as -close to the search command as possible, to ensure the full text search -functions are eligible to push down. For example, a complex query like -\`search source = people | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | where match(employer, 'Open Search') | stats count() by city\` -could fail because it is difficult to translate to DSL, but it would be -better if we rewrite it to an equivalent query as -\`search source = people | where match(employer, 'Open Search') | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | stats count() by city\` -by moving the where command with full text search function to the second -command right after the search command, and the full text search would be -optimized and executed smoothly in OpenSearch DSL. See [Optimization](https://github.com/opensearch-project/sql/blob/22924b13d9cb46759c8d213a7ce903effe06ab47/docs/user/optimization/optimization.rst) -to get more details about the query engine optimization. +The full-text search functions can be executed only in [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/), not in-memory. + +To ensure optimal performance and avoid translation issues with complex full-text searhes, place them as clauses within the search command. + +#### Example + +The following is an example complex query that could fail because it is difficult to translate to query DSL: + + \`search source = people | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | where match(employer, 'Open Search') | stats count() by city\` + +To optimize full-text search performance, rewrite the query by placing the \`WHERE\` clause with the full-text search function as the second command after the \`SEARCH\` command. This ensures the full-text search gets pushed down to query DSL. The following is an example query: + + \`search source = people | where match(employer, 'Open Search') | rename firstname as name | dedup account_number | fields name, account_number, balance, employer | stats count() by city\` + +For details about query engine optimization, see the [Optimizations](https://github.com/opensearch-project/sql/blob/22924b13d9cb46759c8d213a7ce903effe06ab47/docs/user/optimization/optimization.rst) developer documentation on GitHub. `; diff --git a/public/components/common/helpers/ppl_docs/functions/math.ts b/public/components/common/helpers/ppl_docs/functions/math.ts index 33d63f17c..ed189c3eb 100644 --- a/public/components/common/helpers/ppl_docs/functions/math.ts +++ b/public/components/common/helpers/ppl_docs/functions/math.ts @@ -6,17 +6,21 @@ export const mathFunction = `## Math --- -### ABS +### Math functions + +PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitate the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause. -**Description** +The following sections describe the \`math\` PPL functions. -Usage: abs(x) calculate the abs x. +### ABS -Argument type: INTEGER/LONG/FLOAT/DOUBLE +The \`abs\` function is an absolute value function. -Return type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Example: +**Return type:** \`INTEGER/LONG/FLOAT/DOUBLE\` + +#### Example os> source=people | eval \`ABS(-1)\` = ABS(-1) | fields \`ABS(-1)\` fetched rows / total rows = 1/1 @@ -28,16 +32,13 @@ Example: ### ACOS -**Description** - -Usage: acos(x) calculate the arc cosine of x. Returns NULL if x is not -in the range -1 to 1. +The \`acos(x)\` function is an arc cosine function. The function expects values in the range of \`-1\` to \`1\` and returns \`NULL\` if the values aren't in that range. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`ACOS(0)\` = ACOS(0) | fields \`ACOS(0)\` fetched rows / total rows = 1/1 @@ -49,16 +50,13 @@ Example: ### ASIN -**Description** +The \`asin(x)\` function is an arc sine function. The function expects values in the range of \`-1\` to \`1\` and returns \`NULL\` if the values aren't in that range. -Usage: asin(x) calculate the arc sine of x. Returns NULL if x is not in -the range -1 to 1. +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`ASIN(0)\` = ASIN(0) | fields \`ASIN(0)\` fetched rows / total rows = 1/1 @@ -70,17 +68,13 @@ Example: ### ATAN -**Description** - -Usage: atan(x) calculates the arc tangent of x. atan(y, x) calculates -the arc tangent of y / x, except that the signs of both arguments are -used to determine the quadrant of the result. +The \`atan(x)\` function is an arc tangent function that returns an arc tangent of a value \`x\`. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`ATAN(2)\` = ATAN(2), \`ATAN(2, 3)\` = ATAN(2, 3) | fields \`ATAN(2)\`, \`ATAN(2, 3)\` fetched rows / total rows = 1/1 @@ -92,17 +86,13 @@ Example: ### ATAN2 -**Description** +The \`atan2(y, x)\` function is an arc tangent function that calculates the angle from a specified point to the coordinate origin as measured from the positive x-axis. -Usage: atan2(y, x) calculates the arc tangent of y / x, except that the -signs of both arguments are used to determine the quadrant of the -result. +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`ATAN2(2, 3)\` = ATAN2(2, 3) | fields \`ATAN2(2, 3)\` fetched rows / total rows = 1/1 @@ -114,16 +104,13 @@ Example: ### CEIL -**Description** - -Usage: ceil(x) return the smallest integer value this is greater or -equal to x. +The \`ceil(x)\` function returns the smallest integer value that is greater than or equal to the specified value. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`CEIL(2.75)\` = CEIL(2.75) | fields \`CEIL(2.75)\` fetched rows / total rows = 1/1 @@ -135,15 +122,13 @@ Example: ### CONV -**Description** +The \`CONV(x, a, b)\` function converts the number \`x\` from \`a\` base to \`b\` base. -Usage: CONV(x, a, b) converts the number x from a base to b base. +**Argument type:** \`x: STRING, a: INTEGER, b: INTEGER\` -Argument type: x: STRING, a: INTEGER, b: INTEGER +**Return type:** \`STRING\` -Return type: STRING - -Example: +#### Example os> source=people | eval \`CONV('12', 10, 16)\` = CONV('12', 10, 16), \`CONV('2C', 16, 10)\` = CONV('2C', 16, 10), \`CONV(12, 10, 2)\` = CONV(12, 10, 2), \`CONV(1111, 2, 10)\` = CONV(1111, 2, 10) | fields \`CONV('12', 10, 16)\`, \`CONV('2C', 16, 10)\`, \`CONV(12, 10, 2)\`, \`CONV(1111, 2, 10)\` fetched rows / total rows = 1/1 @@ -155,15 +140,13 @@ Example: ### COS -**Description** - -Usage: cos(x) calculate the cosine of x, where x is given in radians. +The \`cos(x)\` function is a cosine function, with \`x\` in radians. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`COS(0)\` = COS(0) | fields \`COS(0)\` fetched rows / total rows = 1/1 @@ -175,16 +158,13 @@ Example: ### COT -**Description** - -Usage: cot(x) calculate the cotangent of x. Returns out-of-range error -if x equals to 0. +The \`cot(x)\` function is a cotangent function. An out-of-range error is returned if \`x\` equals \`0\`. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`COT(1)\` = COT(1) | fields \`COT(1)\` fetched rows / total rows = 1/1 @@ -196,16 +176,13 @@ Example: ### CRC32 -**Description** +The \`crc32\` function calculates the cyclic redundancy check (CRC) value of a given string as a 32-bit unsigned value. -Usage: Calculates a cyclic redundancy check value and returns a 32-bit -unsigned value. +**Argument type:** \`STRING\` -Argument type: STRING +**Return type:** \`LONG\` -Return type: LONG - -Example: +#### Example os> source=people | eval \`CRC32('MySQL')\` = CRC32('MySQL') | fields \`CRC32('MySQL')\` fetched rows / total rows = 1/1 @@ -217,15 +194,13 @@ Example: ### DEGREES -**Description** - -Usage: degrees(x) converts x from radians to degrees. +The \`degrees(x)\` function converts \`x\` from radians to degrees. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`DEGREES(1.57)\` = DEGREES(1.57) | fields \`DEGREES(1.57)\` fetched rows / total rows = 1/1 @@ -237,13 +212,11 @@ Example: ### E -**Description** +The \`E()\` function returns Euler's number. -Usage: E() returns the Euler's number +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`E()\` = E() | fields \`E()\` fetched rows / total rows = 1/1 @@ -255,15 +228,13 @@ Example: ### EXP -**Description** - -Usage: exp(x) return e raised to the power of x. +The \`exp(x)\` function returns \`e\` raised to the power of \`x\`. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: INTEGER +**Return type:** \`INTEGER\` -Example: +#### Example os> source=people | eval \`EXP(2)\` = EXP(2) | fields \`EXP(2)\` fetched rows / total rows = 1/1 @@ -275,16 +246,13 @@ Example: ### FLOOR -**Description** +The \`floor(x)\` function returns the largest integer less than or equal to the specified value. -Usage: floor(x) return the largest integer value this is smaller or -equal to x. +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`INTEGER\` -Return type: INTEGER - -Example: +#### Example os> source=people | eval \`FLOOR(2.75)\` = FLOOR(2.75) | fields \`FLOOR(2.75)\` fetched rows / total rows = 1/1 @@ -296,15 +264,13 @@ Example: ### LN -**Description** - -Usage: ln(x) return the the natural logarithm of x. +The \`ln(x)\` function returns the natural logarithm of \`x\`. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`LN(2)\` = LN(2) | fields \`LN(2)\` fetched rows / total rows = 1/1 @@ -316,18 +282,13 @@ Example: ### LOG -**Description** +The \`log(x)\` function returns the natural logarithm of \`x\`. -Specifications: +**Argument type: \`INTEGER/LONG/FLOAT/DOUBLE\` -Usage: log(x) returns the natural logarithm of x that is the base e -logarithm of the x. log(B, x) is equivalent to log(x)/log(B). +**Return type:** \`DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE - -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`LOG(2)\` = LOG(2), \`LOG(2, 8)\` = LOG(2, 8) | fields \`LOG(2)\`, \`LOG(2, 8)\` fetched rows / total rows = 1/1 @@ -339,17 +300,13 @@ Example: ### LOG2 -**Description** - -Specifications: +The \`log2(x)\` function calculates the base-2 logarithm of \`x\`. -Usage: log2(x) is equivalent to log(x)/log(2). +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`LOG2(8)\` = LOG2(8) | fields \`LOG2(8)\` fetched rows / total rows = 1/1 @@ -361,17 +318,13 @@ Example: ### LOG10 -**Description** - -Specifications: +The \`log10(x)\` function calculates the base-10 logarithm of \`x\`. -Usage: log10(x) is equivalent to log(x)/log(10). +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`LOG10(100)\` = LOG10(100) | fields \`LOG10(100)\` fetched rows / total rows = 1/1 @@ -383,16 +336,13 @@ Example: ### MOD -**Description** - -Usage: MOD(n, m) calculates the remainder of the number n divided by m. +The \`MOD(n, m)\` function calculates the remainder of the number \`n\` divided by \`m\`. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: Wider type between types of n and m if m is nonzero value. -If m equals to 0, then returns NULL. +**Return type:** If \`m\` is a nonzero value, a type greater in size between types \`n\` and \`m\` is returned. If \`m\` equals \`0\`, \`NULL\` is returned. -Example: +#### Example os> source=people | eval \`MOD(3, 2)\` = MOD(3, 2), \`MOD(3.1, 2)\` = MOD(3.1, 2) | fields \`MOD(3, 2)\`, \`MOD(3.1, 2)\` fetched rows / total rows = 1/1 @@ -404,13 +354,11 @@ Example: ### PI -**Description** +The \`PI()\` function returns the constant pi. -Usage: PI() returns the constant pi +**Return type:** \`DOUBLE\` -Return type: DOUBLE - -Example: +#### Example os> source=people | eval \`PI()\` = PI() | fields \`PI()\` fetched rows / total rows = 1/1 @@ -422,18 +370,15 @@ Example: ### POW -**Description** - -Usage: POW(x, y) calculates the value of x raised to the power of y. Bad -inputs return NULL result. +The \`POW(x, y)\` function calculates the value of \`x\` raised to the power of \`y\`. Bad inputs return \`NULL\` results. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Synonyms: [POWER](#power) +**Synonyms:** \`[POWER](#power)\` -Example: +#### Example os> source=people | eval \`POW(3, 2)\` = POW(3, 2), \`POW(-3, 2)\` = POW(-3, 2), \`POW(3, -2)\` = POW(3, -2) | fields \`POW(3, 2)\`, \`POW(-3, 2)\`, \`POW(3, -2)\` fetched rows / total rows = 1/1 @@ -445,18 +390,15 @@ Example: ### POWER -**Description** +The \`POWER(x, y)\` function calculates the value of \`x\` raised to the power of \`y\`. Bad inputs return \`NULL\` results. -Usage: POWER(x, y) calculates the value of x raised to the power of y. -Bad inputs return NULL result. +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`DOUBLE\` -Return type: DOUBLE +**Synonyms:** \`[POW](#pow)\` -Synonyms: [POW](#pow) - -Example: +#### Example os> source=people | eval \`POWER(3, 2)\` = POWER(3, 2), \`POWER(-3, 2)\` = POWER(-3, 2), \`POWER(3, -2)\` = POWER(3, -2) | fields \`POWER(3, 2)\`, \`POWER(-3, 2)\`, \`POWER(3, -2)\` fetched rows / total rows = 1/1 @@ -468,15 +410,13 @@ Example: ### RADIANS -**Description** - -Usage: radians(x) converts x from degrees to radians. +The \`radians(x)\` function converts \`x\` from degrees to radians. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`RADIANS(90)\` = RADIANS(90) | fields \`RADIANS(90)\` fetched rows / total rows = 1/1 @@ -488,19 +428,13 @@ Example: ### RAND -**Description** - -Usage: RAND()/RAND(N) returns a random floating-point value in the range -0 <= value < 1.0. If integer N is specified, the seed is -initialized prior to execution. One implication of this behavior is with -identical argument N, rand(N) returns the same value each time, and thus -produces a repeatable sequence of column values. +The \`RAND()/RAND(N)\` function returns a random floating-point value in the range \`0 <= value < 1.0\`. If integer \`N\` is specified, the seed is initialized prior to execution. One implication of this behavior is that with identical argument \`N\`, \`rand(N)\` returns the same value each time and thus produces a repeatable sequence of column values. -Argument type: INTEGER +**Argument type:** \`INTEGER\` -Return type: FLOAT +**Return type:** \`FLOAT\` -Example: +#### Example os> source=people | eval \`RAND(3)\` = RAND(3) | fields \`RAND(3)\` fetched rows / total rows = 1/1 @@ -512,19 +446,13 @@ Example: ### ROUND -**Description** - -Usage: ROUND(x, d) rounds the argument x to d decimal places, d defaults -to 0 if not specified - -Argument type: INTEGER/LONG/FLOAT/DOUBLE +The \`ROUND(x, d)\` function rounds the argument \`x\` to \`d\` decimal places. \`d\` defaults to \`0\` if a value is not specified. -Return type map: +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -(INTEGER/LONG \[,INTEGER\]) -> LONG (FLOAT/DOUBLE \[,INTEGER\]) -> -LONG +**Return type:** \`(INTEGER/LONG \[,INTEGER\]) -> LONG (FLOAT/DOUBLE \[,INTEGER\]) ->LONG\` -Example: +#### Example os> source=people | eval \`ROUND(12.34)\` = ROUND(12.34), \`ROUND(12.34, 1)\` = ROUND(12.34, 1), \`ROUND(12.34, -1)\` = ROUND(12.34, -1), \`ROUND(12, 1)\` = ROUND(12, 1) | fields \`ROUND(12.34)\`, \`ROUND(12.34, 1)\`, \`ROUND(12.34, -1)\`, \`ROUND(12, 1)\` fetched rows / total rows = 1/1 @@ -536,16 +464,13 @@ Example: ### SIGN -**Description** +The \`sign\` function returns the sign of the argument as -1, 0, or 1, depending on whether the number is negative, zero, or positive. -Usage: Returns the sign of the argument as -1, 0, or 1, depending on -whether the number is negative, zero, or positive +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Return type:** \`INTEGER\` -Return type: INTEGER - -Example: +#### Example os> source=people | eval \`SIGN(1)\` = SIGN(1), \`SIGN(0)\` = SIGN(0), \`SIGN(-1.1)\` = SIGN(-1.1) | fields \`SIGN(1)\`, \`SIGN(0)\`, \`SIGN(-1.1)\` fetched rows / total rows = 1/1 @@ -557,15 +482,13 @@ Example: ### SIN -**Description** - -Usage: sin(x) calculate the sine of x, where x is given in radians. +The \`sin(x)\` function is a sine function, with \`x\` in radians. -Argument type: INTEGER/LONG/FLOAT/DOUBLE +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -Return type: DOUBLE +**Return type:** \`DOUBLE\` -Example: +#### Example os> source=people | eval \`SIN(0)\` = SIN(0) | fields \`SIN(0)\` fetched rows / total rows = 1/1 @@ -577,18 +500,13 @@ Example: ### SQRT -**Description** - -Usage: Calculates the square root of a non-negative number - -Argument type: INTEGER/LONG/FLOAT/DOUBLE +The \`sqrt\` function calculates the square root of a non-negative value \`x\`. -Return type map: +**Argument type:** \`INTEGER/LONG/FLOAT/DOUBLE\` -(Non-negative) INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE (Negative) -INTEGER/LONG/FLOAT/DOUBLE -> NULL +**Return type:** (Non-negative) \`INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE\` (Negative) \`INTEGER/LONG/FLOAT/DOUBLE -> NULL\` -Example: +#### Example os> source=people | eval \`SQRT(4)\` = SQRT(4), \`SQRT(4.41)\` = SQRT(4.41) | fields \`SQRT(4)\`, \`SQRT(4.41)\` fetched rows / total rows = 1/1 diff --git a/public/components/common/helpers/ppl_docs/functions/string.ts b/public/components/common/helpers/ppl_docs/functions/string.ts index 824727ae8..1667b6baf 100644 --- a/public/components/common/helpers/ppl_docs/functions/string.ts +++ b/public/components/common/helpers/ppl_docs/functions/string.ts @@ -6,18 +6,21 @@ export const stringFunction = `## String --- -### CONCAT +### String functions + +PPL functions use the search capabilities of the OpenSearch engine. However, these functions don't execute directly within the OpenSearch plugin's memory. Instead, they facilitate the global filtering of query results based on specific conditions, such as a \`WHERE\` or \`HAVING\` clause. -**Description** +The following sections describe the string functions. + +### CONCAT -Usage: CONCAT(str1, str2) returns str1 and str strings concatenated -together. +The \`CONCAT(str1, str2)\` function returns \`str1\` and \`str\` concatenated strings. -Argument type: STRING, STRING +**Argument type:** \`STRING, STRING\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`CONCAT('hello', 'world')\` = CONCAT('hello', 'world') | fields \`CONCAT('hello', 'world')\` fetched rows / total rows = 1/1 @@ -29,16 +32,13 @@ Example: ### CONCAT\_WS -**Description** +The \`CONCAT\_WS(sep, str1, str2)\` function concatenates two strings together, using \`sep\` as a separator between them. -Usage: CONCAT\_WS(sep, str1, str2) returns str1 concatenated with str2 -using sep as a separator between them. +**Argument type:** \`STRING, STRING, STRING\` -Argument type: STRING, STRING, STRING +**Return type:** \`STRING\` -Return type: STRING - -Example: +#### Example os> source=people | eval \`CONCAT_WS(',', 'hello', 'world')\` = CONCAT_WS(',', 'hello', 'world') | fields \`CONCAT_WS(',', 'hello', 'world')\` fetched rows / total rows = 1/1 @@ -50,19 +50,15 @@ Example: ### LENGTH -**Description** - -Specifications: +The \`length(str)\` function returns the length of a string, measured in number of bytes. -1. LENGTH(STRING) -> INTEGER +**Function signature:** \`LENGTH(STRING) -> INTEGER\` -Usage: length(str) returns length of string measured in bytes. +**Argument type:** \`STRING\` -Argument type: STRING +**Return type:** \`INTEGER\` -Return type: INTEGER - -Example: +#### Example os> source=people | eval \`LENGTH('helloworld')\` = LENGTH('helloworld') | fields \`LENGTH('helloworld')\` fetched rows / total rows = 1/1 @@ -74,18 +70,12 @@ Example: ### LIKE -**Description** - -Usage: like(string, PATTERN) return true if the string match the -PATTERN. +The \`like(string, PATTERN)\` function returns \`true\` if the string matches the \`PATTERN\` value. The following two wildcards are commonly used with the \`like\` operator: -There are two wildcards often used in conjunction with the LIKE -operator: +- \`%\`: A percent sign represents zero, one, or multiple characters. +- \`_\`: An underscore represents a single character. -- \`%\` - The percent sign represents zero, one, or multiple characters -- \`_\` - The underscore represents a single character - -Example: +#### Example os> source=people | eval \`LIKE('hello world', '_ello%')\` = LIKE('hello world', '_ello%') | fields \`LIKE('hello world', '_ello%')\` fetched rows / total rows = 1/1 @@ -97,15 +87,13 @@ Example: ### LOWER -**Description** - -Usage: lower(string) converts the string to lowercase. +The \`lower(string)\` function converts a string to lowercase. -Argument type: STRING +**Argument type:** \`STRING\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`LOWER('helloworld')\` = LOWER('helloworld'), \`LOWER('HELLOWORLD')\` = LOWER('HELLOWORLD') | fields \`LOWER('helloworld')\`, \`LOWER('HELLOWORLD')\` fetched rows / total rows = 1/1 @@ -117,15 +105,13 @@ Example: ### LTRIM -**Description** +The \`ltrim(str)\` function trims leading space characters from a string. -Usage: ltrim(str) trims leading space characters from the string. +**Argument type:** \`STRING\` -Argument type: STRING +**Return type:** \`STRING\` -Return type: STRING - -Example: +#### Example os> source=people | eval \`LTRIM(' hello')\` = LTRIM(' hello'), \`LTRIM('hello ')\` = LTRIM('hello ') | fields \`LTRIM(' hello')\`, \`LTRIM('hello ')\` fetched rows / total rows = 1/1 @@ -137,16 +123,13 @@ Example: ### RIGHT -**Description** - -Usage: right(str, len) returns the rightmost len characters from the -string str, or NULL if any argument is NULL. +The \`right(str, len)\` function returns the rightmost \`len\` characters from a \`str\` value. \`NULL\` is returned if any argument is null. -Argument type: STRING, INTEGER +**Argument type:** \`STRING, INTEGER\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`RIGHT('helloworld', 5)\` = RIGHT('helloworld', 5), \`RIGHT('HELLOWORLD', 0)\` = RIGHT('HELLOWORLD', 0) | fields \`RIGHT('helloworld', 5)\`, \`RIGHT('HELLOWORLD', 0)\` fetched rows / total rows = 1/1 @@ -158,15 +141,13 @@ Example: ### RTRIM -**Description** +The \`rtrim(str)\` function trims trailing space characters from a string. -Usage: rtrim(str) trims trailing space characters from the string. +**Argument type:** \`STRING\` -Argument type: STRING +**Return type:** \`STRING\` -Return type: STRING - -Example: +#### Example os> source=people | eval \`RTRIM(' hello')\` = RTRIM(' hello'), \`RTRIM('hello ')\` = RTRIM('hello ') | fields \`RTRIM(' hello')\`, \`RTRIM('hello ')\` fetched rows / total rows = 1/1 @@ -178,19 +159,15 @@ Example: ### SUBSTRING -**Description** - -Usage: substring(str, start) or substring(str, start, length) returns -substring using start and length. With no length, entire string from -start is returned. +The \`substring(str, start)\` or \`substring(str, start, length)\`function returns a substring of the input string \`str\`. If \`length\` is not specified, the function returns the entire string from the \`start\` index. -Argument type: STRING, INTEGER, INTEGER +**Argument type:** \`STRING, INTEGER, INTEGER\` -Return type: STRING +**Return type:** \`STRING\` -Synonyms: SUBSTR +**Synonyms:** \`SUBSTR\` -Example: +#### Example os> source=people | eval \`SUBSTRING('helloworld', 5)\` = SUBSTRING('helloworld', 5), \`SUBSTRING('helloworld', 5, 3)\` = SUBSTRING('helloworld', 5, 3) | fields \`SUBSTRING('helloworld', 5)\`, \`SUBSTRING('helloworld', 5, 3)\` fetched rows / total rows = 1/1 @@ -202,13 +179,13 @@ Example: ### TRIM -**Description** +The \`trim\` function removes leading and trailing white space from a string. -Argument Type: STRING +**Argument type:** \`STRING\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`TRIM(' hello')\` = TRIM(' hello'), \`TRIM('hello ')\` = TRIM('hello ') | fields \`TRIM(' hello')\`, \`TRIM('hello ')\` fetched rows / total rows = 1/1 @@ -220,15 +197,13 @@ Example: ### UPPER -**Description** - -Usage: upper(string) converts the string to uppercase. +The \`upper(string)\` function converts a string to uppercase. -Argument type: STRING +**Argument type:** \`STRING\` -Return type: STRING +**Return type:** \`STRING\` -Example: +#### Example os> source=people | eval \`UPPER('helloworld')\` = UPPER('helloworld'), \`UPPER('HELLOWORLD')\` = UPPER('HELLOWORLD') | fields \`UPPER('helloworld')\`, \`UPPER('HELLOWORLD')\` fetched rows / total rows = 1/1 diff --git a/public/components/common/helpers/ppl_docs/language_structure/datatypes.ts b/public/components/common/helpers/ppl_docs/language_structure/datatypes.ts index b01149143..7914bc628 100644 --- a/public/components/common/helpers/ppl_docs/language_structure/datatypes.ts +++ b/public/components/common/helpers/ppl_docs/language_structure/datatypes.ts @@ -5,226 +5,150 @@ export const pplDatatypes = `## Data Types --- -### **Overview** - -### PPL Data Types - -The PPL support the following data types. - -| PPL Data Type | -|---------------| -| boolean | -| byte | -| short | -| integer | -| long | -| float | -| double | -| string | -| text | -| timestamp | -| datetime | -| date | -| time | -| interval | -| ip | -| geo\_point | -| binary | -| struct | -| array | - -### Data Types Mapping - -The table below list the mapping between OpenSearch Data Type, PPL Data -Type and SQL Type. - -| OpenSearch Type | PPL Type | SQL Type | + +### Data types + +A data type defines a collection of data type values and a set of predefined operations for those values. PPL supports the following data types: + +.. hlist:: + :columns: 3 + + * array + * binary + * boolean + * byte + * date + * datetime + * double + * float + * geo\_point + * integer + * interval + * ip + * long + * short + * string + * struct + * text + * time + * timestamp + +### Data type mapping + +The following table is a reference guide for the mapping between an OpenSearch data type, a PPL data type, and a SQL data type. + +| OpenSearch type | PPL type | SQL type | |-----------------|-----------|-----------| +| binary | binary | VARBINARY | | boolean | boolean | BOOLEAN | | byte | byte | TINYINT | -| short | byte | SMALLINT | -| integer | integer | INTEGER | -| long | long | BIGINT | +| date | timestamp | TIMESTAMP | +| double | double | DOUBLE | | float | float | REAL | | half\_float | float | FLOAT | -| scaled\_float | float | DOUBLE | -| double | double | DOUBLE | -| keyword | string | VARCHAR | -| text | text | VARCHAR | -| date | timestamp | TIMESTAMP | +| integer | integer | INTEGER | | ip | ip | VARCHAR | -| date | timestamp | TIMESTAMP | -| binary | binary | VARBINARY | -| object | struct | STRUCT | +| keyword | string | VARCHAR | +| long | long | BIGINT | | nested | array | STRUCT | +| object | struct | STRUCT | +| scaled\_float | float | DOUBLE | +| short | byte | SMALLINT | +| text | text | VARCHAR | + +Some PPL types do not correspond to an OpenSearch type. To use functions that require date and time data types, data type conversion must be performed, as described in the following sections. -Notes: Not all the PPL Type has correspond OpenSearch Type. e.g. data -and time. To use function which required such data type, user should -explict convert the data type. +### Numeric data types -### **Numeric Data Types** +Numeric values ranging from -2147483648 to +2147483647 are recognized as +integers, with data type name \`INTEGER\`. For values that fall beyond the specified range, the \`LONG\` data type is assigned during parsing. -Numeric values ranged from -2147483648 to +2147483647 are recognized as -integer with type name \`INTEGER\`. For others outside the range, \`LONG\` -integer will be the data type after parsed. +### Date and time data types -### **Date and Time Data Types** +The data types \`date\` and \`time\` represent temporal values. The PPL plugin supports \`date\`, \`time\`, \`datetime\`, \`timestamp\`, and \`interval\`. By default, [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/) uses \`date\` for any date or time types. To integrate with PPL, each data type, excluding \`timestamp\`, contains temporal and time zone information. Use \`datetime\` functions to clarify the date and time types. Note that certain functions may have limitations on the input argument type. See the [Functions](functions.rst) section in this manual for more information. -The date and time data types are the types that represent temporal -values and PPL plugin supports types including DATE, TIME, DATETIME, -TIMESTAMP and INTERVAL. By default, the OpenSearch DSL uses date type as -the only date and time related type, which has contained all information -about an absolute time point. To integrate with PPL language, each of -the types other than timestamp is holding part of temporal or timezone -information, and the usage to explicitly clarify the date and time types -is reflected in the datetime functions (see [Functions](functions.rst) -for details), where some functions might have restrictions in the input -argument type. +#### Date -#### **Date** +The \`date\` data type represents the calendar date, regardless of time zone. A specific date value represents a 24-hour period, but this period differs across time zones and may be subject to variations due to daylight saving time adjustments. Additionally, the date alone does not contain time-specific information. The date values range from '1000-01-01' to '9999-12-31'. -Date represents the calendar date regardless of the time zone. A given -date value represents a 24-hour period, or say a day, but this period -varies in different timezones and might have flexible hours during -Daylight Savings Time programs. Besides, the date type does not contain -time information as well. The supported range is '1000-01-01' to -'9999-12-31'. +| Type | Syntax | Range | +|----------|--------------|------------------------------| +| \`date\` | 'yyyy-MM-dd' | '0001-01-01' to '9999-12-31' | -| Type | Syntax | Range | -|------|--------------|------------------------------| -| Date | 'yyyy-MM-dd' | '0001-01-01' to '9999-12-31' | +#### Time -#### **Time** +The \`time\` data type represents the time of day as displayed on a clock or watch, without specifying a particular time zone. It does not include any information about the calendar date. -Time represents the time on the clock or watch with no regard for which -timezone it might be related with. Time type data does not have date -information. +| Type | Syntax | Range | +|----------|--------------|------------------------------| +| \`time\` | 'hh:mm:ss\[.fraction\]' | '00:00:00.000000' to '23:59:59.999999' | -| Type | Syntax | Range | -|------|-------------------------|----------------------------------------| -| Time | 'hh:mm:ss\[.fraction\]' | '00:00:00.000000' to '23:59:59.999999' | +#### Datetime -#### **Datetime** +The \`datetime\` data type represents a combination of \`date\` and \`time\`. The \`datetime\` data type does not contain time zone information. For an absolute time point that contains both datetime and time zone information, see the [Timestamp](#timestamp) section. -Datetime type is the combination of date and time. The conversion rule -of date or time to datetime is described in [Conversion between date and -time types](#conversion-between-date-and-time-types). Datetime type does -not contain timezone information. For an absolute time point that -contains both date time and timezone information, see -[Timestamp](#timestamp). +See the [Conversion between date and time types](#conversion-between-date-and-time-types) section for information about the conversion rule for \`date\` or \`time\` to \`datetime\`. -| Type | Syntax | Range | -|----------|------------------------------------|--------------------------------------------------------------| -| Datetime | 'yyyy-MM-dd hh:mm:ss\[.fraction\]' | '0001-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999' | +| Type | Syntax | Range | +|----------|--------|-------| +| \`datetime\` | 'yyyy-MM-dd hh:mm:ss\[.fraction\]' | '0001-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999' | -#### **Timestamp** +#### Timestamp -A timestamp instance is an absolute instant independent of timezone or -convention. For example, for a given point of time, if we set the -timestamp of this time point into another timezone, the value should -also be different accordingly. Besides, the storage of timestamp type is -also different from the other types. The timestamp is converted from the -current timezone to UTC for storage, and is converted back to the set -timezone from UTC when retrieving. +The \`timestamp\` data type represents absolute points in time, unaffected by time zones or conventions. The \`timestamp\` data type differs from other data types in its storage and retrieval behavior. When a timestamp is sorted, it is converted from Coordinated Universal Time (UTC) to the specified time zone. Conversely, when a timestamp is retrieved, it is converted back to UTC before being displayed or used in calculations. This ensures that the timestamp values remain consistent and comparable across different time zones. -| Type | Syntax | Range | -|-----------|------------------------------------|------------------------------------------------------------------| -| Timestamp | 'yyyy-MM-dd hh:mm:ss\[.fraction\]' | '0001-01-01 00:00:01.000000' UTC to '9999-12-31 23:59:59.999999' | +| Type | Syntax | Range | +|-----------|--------|-------| +| Timestamp | 'yyyy-MM-dd hh:mm:ss\[.fraction\]' | '0001-01-01 00:00:01.000000' UTC to '9999-12-31 23:59:59.999999' | -#### **Interval** +#### Interval -Interval data type represents a temporal duration or a period. The -syntax is as follows: +The \`interval\` data type represents a span of time encompassing a specified duration or period. | Type | Syntax | |----------|--------------------| | Interval | INTERVAL expr unit | -The expr is any expression that can be iterated to a quantity value -eventually, see [Expressions](expressions.rst) for details. The unit -represents the unit for interpreting the quantity, including -MICROSECOND, SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER and -YEAR.The INTERVAL keyword and the unit specifier are not case sensitive. -Note that there are two classes of intervals. Year-week intervals can -store years, quarters, months and weeks. Day-time intervals can store -days, hours, minutes, seconds and microseconds. Year-week intervals are -comparable only with another year-week intervals. These two types of -intervals can only comparable with the same type of themselves. - -### **Conversion between date and time types** - -Basically the date and time types except interval can be converted to -each other, but might suffer some alteration of the value or some -information loss, for example extracting the time value from a datetime -value, or convert a date value to a datetime value and so forth. Here -lists the summary of the conversion rules that PPL plugin supports for -each of the types: - -#### **Conversion from DATE** - -- Since the date value does not have any time information, conversion - to [Time](#time) type is not useful, and will always return a zero - time value '00:00:00'. -- Conversion from date to datetime has a data fill-up due to the lack - of time information, and it attaches the time '00:00:00' to the - original date by default and forms a datetime instance. For example, - the result to covert date '2020-08-17' to datetime type is datetime - '2020-08-17 00:00:00'. -- Conversion to timestamp is to alternate both the time value and the - timezone information, and it attaches the zero time value '00:00:00' - and the session timezone (UTC by default) to the date. For example, - the result to covert date '2020-08-17' to datetime type with session - timezone UTC is datetime '2020-08-17 00:00:00' UTC. - -#### **Conversion from TIME** - -- Time value cannot be converted to any other date and time types - since it does not contain any date information, so it is not - meaningful to give no date info to a date/datetime/timestamp - instance. - -#### **Conversion from DATETIME** - -- Conversion from datetime to date is to extract the date part from - the datetime value. For example, the result to convert datetime - '2020-08-17 14:09:00' to date is date '2020-08-08'. -- Conversion to time is to extract the time part from the datetime - value. For example, the result to convert datetime '2020-08-17 - 14:09:00' to time is time '14:09:00'. -- Since the datetime type does not contain timezone information, the - conversion to timestamp needs to fill up the timezone part with the - session timezone. For example, the result to convert datetime - '2020-08-17 14:09:00' with system timezone of UTC, to timestamp is - timestamp '2020-08-17 14:09:00' UTC. - -#### **Conversion from TIMESTAMP** - -- Conversion from timestamp is much more straightforward. To convert - it to date is to extract the date value, and conversion to time is - to extract the time value. Conversion to datetime, it will extracts - the datetime value and leave the timezone information over. For - example, the result to convert datetime '2020-08-17 14:09:00' UTC to - date is date '2020-08-17', to time is '14:09:00' and to datetime is - datetime '2020-08-17 14:09:00'. - -### **String Data Types** - -A string is a sequence of characters enclosed in either single or double -quotes. For example, both 'text' and "text" will be treated as string -literal. - -### **Query Struct Data Types** - -In PPL, the Struct Data Types corresponding to the [Object field type in -OpenSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html). -The "." is used as the path selector when access the inner attribute of -the struct data. - -#### **Example: People** - -There are three fields in test index \`people\`: 1) deep nested object -field \`city\`; 2) object field of array value \`account\`; 3) nested field -\`projects\`: +The expression \`expr\` is configured to be repeatedly evaluated to produce a quantitative value. See the [Expressions](expressions.rst) section of this manual for more information. The unit represents the unit used to interpret the quantity, including MICROSECOND, SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, and YEAR. The INTERVAL keyword and the unit specifier are not case sensitive. + +Intervals consist of two classes: day-time and year-week. Day-time intervals store days, hours, minutes, seconds, and microseconds. Year-week intervals store years, quarters, months, and weeks. Each type can only be compared to the same type. + +### Date and time conversion + +Date and time types, excluding \`interval\`, can be mutually converted, with some alteration of the value or some information loss, for example, when extracting the \`time\` value from a \`datetime\` value or converting a \`date\` value to a \`datetime\` value. The PPL plugin supports the following conversion rules for each of the types. + +#### \`date\` conversion + +- Because \`date\` does not contain time information, conversion to \`time\` returns a zero time value \`00:00:00\`. +- Converting from \`date\` to \`datetime\` sets the time value to \`00:00:00\` if \`time\` is not provided, for example, \`2020-08-17\` converts to \`2020-08-17 00:00:00\`. +- Converting to \`timestamp\` sets \`time\` to \`00:00:00\` and time zone (UTC by default), for example, \`2020-08-17\` converts to \`2020-08-17 00:00:00 UTC\`. + +#### \`time\` conversion + +- A \`time\` value does not have any date information, so it cannot be converted to other date and time types. + +#### \`datetime\` conversion + +- Converting from \`datetime\` to \`date\` extracts the date component from the \`datetime\` value, for example, \`2020-08-17 14:09:00\` converts to \`2020-08-08\`. +- Converting to \`time\` extracts the time component from the \`datetime\` value, for example, \`2020-08-17 14:09:00\` converts to \`14:09:00\`. +- Because \`datetime\` does not contain time zone information, conversion to \`timestamp\` sets the time zone to the session's time zone, for example, \`2020-08-17 14:09:00\`, with the system time zone set to UTC, for example, \`2020-08-17 14:09:00 UTC\`. + +#### \`timestamp\` conversion + +- Converting from \`timestamp\` to \`date\ extracts the \`date\` and \`time\` values. Converting from \`timestamp\` to \`datetime\` extracts the \`datetime\` value and retains the time zone information. For example, \`2020-08-17 14:09:00 UTC\` converts \`date\` and \`time\` to \`2020-08-17\` and \`14:09:00\` and \`datetime\` to \`2020-08-17 14:09:00\`. + +### String data types + +A \`string\` data type is a series of characters enclosed within single or double quotation marks that serves as a data type for storing text data. + +### Query struct data type + +In PPL, the \`struct\` data type corresponds to the [Object field type in +OpenSearch](https://opensearch.org/docs/latest/field-types/supported-field-types/object-fields/). The \`"."\` is used as the path selector for accessing the inner attribute of the struct data. + +#### Example 1: Struct to store population data + +The following example struct stores population data in an index containing the following fields: deep-nested object field \`city\`, object field of array value \`account\`, and nested field \`projects\`. { "mappings": { @@ -262,10 +186,9 @@ field \`city\`; 2) object field of array value \`account\`; 3) nested field } } -#### **Example: Employees** +#### Example 2: Struct to store employee data -Here is the mapping for test index \`employees_nested\`. Note that field -\`projects\` is a nested field: +The following example struct stores employee data and includes a nested field: { "mappings": { @@ -365,13 +288,9 @@ Result set: ] } -#### **Example 1: Select struct inner attribute** +#### Example 3: Select a struct inner attribute -The example show fetch city (top level), city.name (second level), -city.location.latitude (deeper level) struct type data from people -results. - -PPL query: +The following example PPL query shows how to fetch \`city\` (top level), \`city.name\` (second level), and \`city.location.latitude\` (deeper level) struct data from the results: os> source=people | fields city, city.name, city.location.latitude; fetched rows / total rows = 1/1 @@ -381,11 +300,9 @@ PPL query: | {'name': 'Seattle', 'location': {'latitude': 10.5}} | Seattle | 10.5 | +-----------------------------------------------------+-------------+--------------------------+ -#### **Example 2: Group by struct inner attribute** - -The example show group by object field inner attribute. +#### Example 4: Group by a struct inner attribute -PPL query: +The following example PPL query shows how to group by a struct inner attribute: os> source=people | stats count() by city.name; fetched rows / total rows = 1/1 @@ -395,12 +312,9 @@ PPL query: | 1 | Seattle | +-----------+-------------+ -#### **Example 3: Selecting Field of Array Value** +#### Example 5: Select an object field of an array value -Select deeper level for object fields of array value which returns the -first element in the array. For example, because inner field -\`accounts.id\` has three values instead of a tuple in this document, the -first entry is returned.: +The following example PPL query shows how to select a deeper level for object fields of array values that return the first element in the array. In this example, the document's inner field \`accounts.id\` has three values instead of a tuple: os> source = people | fields accounts, accounts.id; fetched rows / total rows = 1/1 diff --git a/public/components/common/helpers/ppl_docs/language_structure/identifiers.ts b/public/components/common/helpers/ppl_docs/language_structure/identifiers.ts index 1a931f642..362887363 100644 --- a/public/components/common/helpers/ppl_docs/language_structure/identifiers.ts +++ b/public/components/common/helpers/ppl_docs/language_structure/identifiers.ts @@ -6,44 +6,26 @@ export const pplIdentifiers = `## Indentifiers --- -### **Introduction** +### Identifiers -Identifiers are used for naming your database objects, such as index -name, field name, customLabel etc. Basically there are two types of -identifiers: regular identifiers and delimited identifiers. +Identifiers are used for naming database objects, such as an index name, a field name, or a custom label. The two types of identifiers are _regular identifiers_ and _delimited identifiers_. -### **Regular Identifiers** +#### Regular identifiers -**Description** +A regular identifier is a string of characters that starts with an ASCII letter (lowercase or uppercase). The subsequent characters can be a combination of letters, digits, or underscores (\`_\`). A regular identifier cannot be a reversed keyword, and white space or other special characters are not allowed. -A regular identifier is a string of characters that must start with -ASCII letter (lower or upper case). The subsequent character can be a -combination of letter, digit, underscore (\`_\`). It cannot be a reversed -key word. And whitespace and other special characters are not allowed. +The following identifiers are supported by OpenSearch extensions: -For OpenSearch, the following identifiers are supported extensionally: +- **Identifiers prefixed by dot \`.\`:** This is called a hidden index. An example is \`.opensearch_dashboards\`. +- **Identifiers prefixed by the \`@\` symbol:** This is common in meta fields generated for data ingestion. +- **Identifiers with \`-\` in the middle:** This is common in index naming conventions with date information. +- **Identifiers with the \`*\` symbol:** This is common in wildcard matches in an index pattern. -1. Identifiers prefixed by dot \`.\`: this is called hidden index in - OpenSearch, for example \`.opensearch_dashboards\`. -2. Identifiers prefixed by at sign \`@\`: this is common for meta fields - generated in Logstash ingestion. -3. Identifiers with \`-\` in the middle: this is mostly the case for - index name with date information. -4. Identifiers with star \`*\` present: this is mostly an index pattern - for wildcard match. +Index names with a date suffix separated by dashes or dots, such as \`cwl-2020.01.11\` or \`logs-7.0-2020.01.11\`, are common in data ingestion. Identifiers used as index names do not need to be enclosed in quotation marks. Additionally, wildcards within date patterns are accepted, enabling data retrieval across indexes covering different date ranges. For example, you can use \`logs-2020.1*\` to search in indexes for October, November, and December 2020. -Index name with date suffix separated by dash or dots, such as -\`cwl-2020.01.11\` or \`logs-7.0-2020.01.11\`, is common for those created -by Logstash or FileBeat ingestion. So, this kind of identifier used as -index name is also supported without the need of being quoted for user -convenience. In this case, wildcard within date pattern is also allowed -to search for data across indices of different date range. For example, -you can use \`logs-2020.1*\` to search in indices for October, November -and December 2020. +#### Example 1: Index pattern without quotes -#### **Examples** - -Here are examples for using index pattern directly without quotes: +The following example PPL query uses an index pattern directly without quotes: os> source=accounts | fields account_number, firstname, lastname; fetched rows / total rows = 4/4 @@ -56,30 +38,21 @@ Here are examples for using index pattern directly without quotes: | 18 | Dale | Adams | +------------------+-------------+------------+ -### **Delimited Identifiers** - -**Description** +### Delimited identifiers -A delimited identifier is an identifier enclosed in back ticks \`\`. In -this case, the identifier enclosed is not necessarily a regular -identifier. In other words, it can contain any special character not -allowed by regular identifier. +A delimited identifier is an identifier enclosed in backticks \`\` that contains special characters not permitted in regular identifiers. This allows for the use of characters that would otherwise violate the naming rules for identifiers. -#### **Use Cases** +#### Use cases -Here are typical examples of the use of delimited identifiers: +Common use cases for delimited identifiers include the following: -1. Identifiers of reserved key word name -2. Identifiers with dot \`.\` present: similarly as \`-\` in index name to - include date information, it is required to be quoted so parser can - differentiate it from identifier with qualifiers. -3. Identifiers with other special character: OpenSearch has its own - rule which allows more special character, for example Unicode - character is supported in index name. +- Identifiers that coincide with reserved keywords. +- Identifiers that contain a dot \`.\` or a dash \`-\` need to be distinguished from regular identifiers with qualifiers. Enclosing such identifiers in backticks \`\` allows for the parser to differentiate them from qualified identifiers and enables date information within index names. +- Identifiers with special characters in index names. Note that OpenSearch permits the use of special characters, including Unicode characters. -#### **Examples** +#### Example 2: Index name enclosed in backticks -Here are examples for quoting an index name by back ticks: +The following example PPL query uses an index name enclosed in backticks \`\`: os> source=\`accounts\` | fields \`account_number\`; fetched rows / total rows = 4/4 @@ -92,16 +65,7 @@ Here are examples for quoting an index name by back ticks: | 18 | +------------------+ -### **Case Sensitivity** - -**Description** - -Identifiers are treated in case sensitive manner. So it must be exactly -same as what is stored in OpenSearch. - -### **Examples** +### Case sensitivity -For example, if you run \`source=Accounts\`, it will end up with an index -not found exception from our plugin because the actual index name is -under lower case. +Identifiers are case sensitive and must match what is stored in OpenSearch. For example, if you run \`source=Accounts\`, an error \`index not found\` occurs \` because the index name \`accounts\` is lowercase. `; diff --git a/public/components/common/helpers/ppl_docs/overview.tsx b/public/components/common/helpers/ppl_docs/overview.tsx index 0bb3bd71f..5c8d92e57 100644 --- a/public/components/common/helpers/ppl_docs/overview.tsx +++ b/public/components/common/helpers/ppl_docs/overview.tsx @@ -5,33 +5,20 @@ export const overview = `## Overview --- -Piped Processing Language (PPL), powered by OpenSearch, enables -OpenSearch users with exploration and discovery of, and finding search -patterns in data stored in OpenSearch, using a set of commands delimited -by pipes (\|). These are essentially read-only requests to process data -and return results. +Piped Processing Language (PPL) is a query language that processes data in a sequential, step-by-step manner. PPL uses a set of commands, connected by pipes (|), to process data and return results (note that the requests and results are read-only in OpenSearch). -Currently, OpenSearch users can query data using either Query DSL or -SQL. Query DSL is powerful and fast. However, it has a steep learning -curve, and was not designed as a human interface to easily create ad hoc -queries and explore user data. SQL allows users to extract and analyze -data in OpenSearch in a declarative manner. OpenSearch now makes its -search and query engine robust by introducing Piped Processing Language -(PPL). It enables users to extract insights from OpenSearch with a -sequence of commands delimited by pipes () syntax. It enables -developers, DevOps engineers, support engineers, site reliability -engineers (SREs), and IT managers to effectively discover and explore -log, monitoring and observability data stored in OpenSearch. +You can query data in OpenSearch using [PPL](https://opensearch.org/docs/latest/search-plugins/sql/ppl/index/), [query domain-specific language (DSL)](https://opensearch.org/docs/latest/query-dsl/index/), or [SQL](https://opensearch.org/docs/latest/search-plugins/sql/sql/index/). -We expand the capabilities of our Workbench, a comprehensive and -integrated visual query tool currently supporting only SQL, to run -on-demand PPL commands, and view and save results as text and JSON. We -also add a new interactive standalone command line tool, the PPL CLI, to -run on-demand PPL commands, and view and save results as text and JSON. +PPL is the primary language used for observability tasks in OpenSearch. Developers, DevOps engineers, support engineers, site reliability engineers, and IT managers find it useful for exploring and discovering log, monitoring, and observability data. For example, you can use PPL to: -The query start with search command and then flowing a set of command -delimited by pipe ( for example, the following query retrieve firstname -and lastname from accounts if age large than 18. +- Find all log messages that contain a specific error code. +- Identify trends in your data over time. +- Group similar data points. +- Calculate statistics for your data. + +PPL is available in OpenSearch Dashboards and as a standalone command-line tool. Within OpenSearch Dashboards, you can use [Query Workbench](https://opensearch.org/docs/latest/dashboards/query-workbench/) to run on-demand PPL commands and view and save the results as both text and JSON. The [PPL command line interface (CLI)](https://opensearch.org/docs/latest/search-plugins/sql/cli/) is a standalone Python application that you can launch with the \`opensearchsql\` command and then run on-demand PPL commands and view and save the results as both text and JSON. + +Here is an example of a PPL query. This query retrieves the first and last names of all accounts for which the age is greater than 18. \`\`\` source=accounts