Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC]Revise and edit PPL in-product documentation #1233

Merged
merged 116 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
bbab065
Revise and edit PPL in-product documentation
vagimeli Nov 8, 2023
990fde2
Revise and edit PPL in-product documentation
vagimeli Nov 9, 2023
4a99c6e
Revise and edit PPL in-product documentation
vagimeli Nov 9, 2023
baa5629
Revise and edit PPL in-product documentation
vagimeli Nov 9, 2023
97f7f8a
Revise and edit PPL in-product documentation
vagimeli Nov 9, 2023
8c32384
Revise and edit PPL in-product documentation
vagimeli Nov 10, 2023
b3efdc8
Revise and edit PPL in-product documentation
vagimeli Nov 10, 2023
08d12ba
Revise and edit PPL in-product documentation
vagimeli Nov 10, 2023
f7f41cf
Revise intro and examples to use date math index name use cases
vagimeli Nov 10, 2023
3d3934a
Revise intro and examples to use date math index name use cases
vagimeli Nov 10, 2023
ecddec5
Revise and edit PPL in-product documentation
vagimeli Nov 10, 2023
260552b
Revise data types section
vagimeli Nov 13, 2023
ab9f9cc
Revise data types section
vagimeli Nov 13, 2023
93b6b96
Revise functions section
vagimeli Nov 13, 2023
998966b
Revise math section
vagimeli Nov 15, 2023
61ef791
Revise string section
vagimeli Nov 15, 2023
c9e0938
Address tech review comments in Overview section
vagimeli Nov 15, 2023
825e7b7
Address tech review comments in Overview section
vagimeli Nov 17, 2023
10165a9
Address tech review comments in Overview section
vagimeli Nov 21, 2023
3be6b41
Address tech review comments in datatypes section
vagimeli Nov 28, 2023
f9795de
Update public/components/common/helpers/ppl_docs/commands/dedup.ts
vagimeli Dec 5, 2023
6bb86aa
Update public/components/common/helpers/ppl_docs/commands/dedup.ts
vagimeli Dec 5, 2023
80082e1
Update public/components/common/helpers/ppl_docs/commands/dedup.ts
vagimeli Dec 5, 2023
d0a4113
Update public/components/common/helpers/ppl_docs/overview.tsx
vagimeli Dec 5, 2023
1beebff
Update public/components/common/helpers/ppl_docs/commands/eval.ts
vagimeli Dec 5, 2023
edc130d
Update public/components/common/helpers/ppl_docs/overview.tsx
vagimeli Dec 5, 2023
f696c5f
Update public/components/common/helpers/ppl_docs/commands/eval.ts
vagimeli Dec 5, 2023
5829b40
Update public/components/common/helpers/ppl_docs/commands/fields.ts
vagimeli Dec 5, 2023
dd850ac
Update public/components/common/helpers/ppl_docs/commands/fields.ts
vagimeli Dec 5, 2023
6eac093
Update public/components/common/helpers/ppl_docs/commands/head.ts
vagimeli Dec 5, 2023
dd25b5f
Update public/components/common/helpers/ppl_docs/commands/head.ts
vagimeli Dec 5, 2023
3c921e1
Update public/components/common/helpers/ppl_docs/commands/head.ts
vagimeli Dec 5, 2023
62265bd
Update public/components/common/helpers/ppl_docs/commands/head.ts
vagimeli Dec 5, 2023
4fb67f2
Update public/components/common/helpers/ppl_docs/commands/parse.ts
vagimeli Dec 5, 2023
76b9bdf
Update public/components/common/helpers/ppl_docs/commands/parse.ts
vagimeli Dec 5, 2023
0a79b88
Update public/components/common/helpers/ppl_docs/commands/rare.ts
vagimeli Dec 5, 2023
117c8e5
Update public/components/common/helpers/ppl_docs/commands/rare.ts
vagimeli Dec 5, 2023
229768f
Update public/components/common/helpers/ppl_docs/commands/rename.ts
vagimeli Dec 5, 2023
300f2f0
Update public/components/common/helpers/ppl_docs/commands/search.ts
vagimeli Dec 5, 2023
d2bb13c
Update public/components/common/helpers/ppl_docs/commands/search.ts
vagimeli Dec 5, 2023
817f9da
Update public/components/common/helpers/ppl_docs/commands/sort.ts
vagimeli Dec 5, 2023
1823707
Update public/components/common/helpers/ppl_docs/commands/sort.ts
vagimeli Dec 5, 2023
a06a76b
Update public/components/common/helpers/ppl_docs/commands/stats.ts
vagimeli Dec 5, 2023
33aefbf
Update public/components/common/helpers/ppl_docs/commands/stats.ts
vagimeli Dec 5, 2023
4d2072a
Update public/components/common/helpers/ppl_docs/commands/stats.ts
vagimeli Dec 5, 2023
5e43822
Update public/components/common/helpers/ppl_docs/commands/stats.ts
vagimeli Dec 5, 2023
0de62b2
Apply suggestions from code review
vagimeli Dec 5, 2023
27f587c
Update public/components/common/helpers/ppl_docs/overview.tsx
vagimeli Dec 5, 2023
9baa4e7
Update public/components/common/helpers/ppl_docs/commands/stats.ts
vagimeli Dec 5, 2023
0a7ef66
Update public/components/common/helpers/ppl_docs/commands/syntax.ts
vagimeli Dec 5, 2023
65bb0b5
Update public/components/common/helpers/ppl_docs/commands/top.ts
vagimeli Dec 5, 2023
9e5c113
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
9ccbac8
Update public/components/common/helpers/ppl_docs/commands/where.ts
vagimeli Dec 5, 2023
673d856
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
be427f0
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
51fa19f
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
bb7788b
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
411e988
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
5beb994
Update public/components/common/helpers/ppl_docs/functions/string.ts
vagimeli Dec 5, 2023
81bfb14
Update public/components/common/helpers/ppl_docs/functions/math.ts
vagimeli Dec 5, 2023
ce00026
Update public/components/common/helpers/ppl_docs/functions/string.ts
vagimeli Dec 5, 2023
504897e
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
6eac9a7
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
e40a338
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
347a139
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
6c63ed8
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
85f57df
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
3422aca
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
d08b3f7
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
98ce7e8
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
7771ad4
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
6e4e937
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
1cf5226
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
03131d7
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
90cc089
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
76bef7a
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
ef9d7a3
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
61deecf
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
987240b
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
1d88511
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
2e0b318
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
57b8638
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
8553aad
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
233b313
Update public/components/common/helpers/ppl_docs/language_structure/d…
vagimeli Dec 5, 2023
3ded3ce
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
12c09ab
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
8b85a07
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
40e58f4
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
61dd71b
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
585b4ff
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
702e764
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 5, 2023
808f61e
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 6, 2023
91f6d44
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 6, 2023
0d25a85
Update public/components/common/helpers/ppl_docs/overview.tsx
vagimeli Dec 6, 2023
44f0d76
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 6, 2023
debf4c8
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 6, 2023
f188b18
Update public/components/common/helpers/ppl_docs/language_structure/i…
vagimeli Dec 6, 2023
a6d02d7
Address editorial review feedback
vagimeli Dec 6, 2023
e5017ce
Address editorial review feedback Overview section
vagimeli Dec 6, 2023
f79d4b0
Address editorial review feedback Data types section
vagimeli Dec 6, 2023
950e450
Address editorial feedback Identifiers section
vagimeli Dec 6, 2023
ae2b498
Address editorial feedback Overview section
vagimeli Dec 6, 2023
f84884f
Address editorial feedback String section
vagimeli Dec 6, 2023
a8c3dd0
Address editorial feedback Math section
vagimeli Dec 6, 2023
6e4b17f
Address editorial feedback Full-text search section
vagimeli Dec 7, 2023
3b628a9
rewrite datetime functions section for clarity and accuracy
vagimeli Dec 7, 2023
cffeac0
rewrite condition functions section for clarity and accuracy
vagimeli Dec 7, 2023
b59d8d9
rewrite condition functions section for clarity and accuracy
vagimeli Dec 7, 2023
2190fb9
rewrite condition functions section for clarity and accuracy
vagimeli Dec 7, 2023
ca1d7a0
rewrite condition functions section for clarity and accuracy
vagimeli Dec 7, 2023
c476024
Address editorial feedback where section
vagimeli Dec 7, 2023
a5f621a
Address editorial feedback top section
vagimeli Dec 7, 2023
e4ff466
Address editorial feedback syntax section
vagimeli Dec 8, 2023
ef6ae83
Address editorial feedback stats section
vagimeli Dec 8, 2023
7892a30
Address editorial feedback stats section
vagimeli Dec 8, 2023
65ac8c5
Address editorial feedback stats section
vagimeli Dec 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 16 additions & 37 deletions public/components/common/helpers/ppl_docs/commands/dedup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,33 +8,21 @@ export const dedupCmd = `## dedup

### Description

Using \'dedup\' command to remove identical document defined by field from
the search result.
Use the \'dedup\' command to remove identical documents from the search results, based on the specified field.

### Syntax

dedup \[int\] <field-list> \[keepempty=<bool>\]
\[consecutive=<bool>\]

- int: optional. The \'dedup\' command retains multiple events for each
combination when you specify <int>. The number for <int>
must be greater than 0. If you do not specify a number, only the
first occurring event is kept. All other duplicates are removed from
the results. **Default:** 1
- keepempty: optional. if true, keep the document if the any field in
the field-list has NULL value or field is MISSING. **Default:**
false.
- consecutive: optional. If set to true, removes only events with
duplicate combinations of values that are consecutive. **Default:**
false.
- field-list: mandatory. The comma-delimited field list. At least one
field is required.
- \`field-list\`: Required. The comma-delimited field list. At least one field is required.
- \`consecutive\`: Optional. If set to \`true\`, removes duplicate events, where the duplicate events have consecutive timestamps. **Default:** \`false\`
- \`int\`: Optional. The \'dedup\' command retains multiple events for each combination when you specify \`<int>\`. The number for \`<int>\` must be greater than 0. If you do not specify a number, only the first occurring event is kept. All other duplicates are removed from the results. Default is \`1\`.
- \`keepempty\`: Optional. If set to \`true\`, keep the document if any field in the \`field-list\` is null or missing. Default is \`false\`.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Example 1: Dedup by one field
#### Example 1: Dedup by one field

The example show dedup the document with gender field.

PPL query:
The following PPL query example shows how to use \`dedup\` to remove duplicate documents based on the \`gender\` field.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | dedup gender | fields account_number, gender;
fetched rows / total rows = 2/2
Expand All @@ -45,12 +33,9 @@ PPL query:
| 13 | F |
+------------------+----------+

### Example 2: Keep 2 duplicates documents

The example show dedup the document with gender field keep 2
duplication.
#### Example 2: Keep two duplicate documents

PPL query:
The following PPL query example shows how to use \`dedup\` to remove duplicate documents based on the \`gender\` field while keeping two duplicates.

os> source=accounts | dedup 2 gender | fields account_number, gender;
fetched rows / total rows = 3/3
Expand All @@ -62,11 +47,9 @@ PPL query:
| 13 | F |
+------------------+----------+

### Example 3: Keep or Ignore the empty field by default

The example show dedup the document by keep null value field.
#### Example 3: Keep or ignore empty fields by default

PPL query:
The following PPL query example shows how to use \`dedup\` to remove duplicate documents while keeping documents with null values in the specified field.

os> source=accounts | dedup email keepempty=true | fields account_number, email;
fetched rows / total rows = 4/4
Expand All @@ -79,9 +62,7 @@ PPL query:
| 18 | daleadams@boink.com |
+------------------+-----------------------+

The example show dedup the document by ignore the empty value field.

PPL query:
The following PPL query example shows how to use \`dedup\` to remove duplicate documents while ignoring documents with empty values in the specified field.

os> source=accounts | dedup email | fields account_number, email;
fetched rows / total rows = 3/3
Expand All @@ -93,11 +74,9 @@ PPL query:
| 18 | daleadams@boink.com |
+------------------+-----------------------+

#### Example 4: Dedup in consecutive document

The example show dedup the consecutive document.
#### Example 4: Remove duplicate consecutive documents

PPL query:
The following PPL query example shows how to use \`dedup\` to remove duplicate documents while removing the duplicate consecutive document.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | dedup gender consecutive=true | fields account_number, gender;
fetched rows / total rows = 3/3
Expand All @@ -109,6 +88,6 @@ PPL query:
| 18 | M |
+------------------+----------+

#### Limitation
The \`dedup\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node.
### Limitation
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
The \`dedup\` command is not rewritten to Query DSL. It's only run on the coordinating node.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
`;
33 changes: 11 additions & 22 deletions public/components/common/helpers/ppl_docs/commands/eval.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,19 @@ export const evalCmd = `## eval

### Description

The \'eval\' command evaluate the expression and append the result to the
search result.
Use the \'eval\' command to evaluate the expression and append the result to the search result.

### Syntax

eval <field>=<expression> \[","
<field>=<expression> \]...

- field: mandatory. If the field name not exist, a new field is added.
If the field name already exists, it will be overrided.
- expression: mandatory. Any expression support by the system.
- \`field\`: Required. If the field name does not exist, a new field is created. If the field name exists, the vlaue of the existing field is replaced.
- \`expression\`: Required. Any expression that is supported by the system.

### Example 1: Create the new field
#### Example 1: Create new fields

The example show to create new field doubleAge for each document. The
new doubleAge is the evaluation result of age multiply by 2.

PPL query:
The following PPL example shows how to use \`eval\` to create a new field for each document. In this example, the new field is \`doubleAge\`.

os> source=accounts | eval doubleAge = age * 2 | fields age, doubleAge ;
fetched rows / total rows = 4/4
Expand All @@ -38,11 +33,9 @@ PPL query:
| 33 | 66 |
+-------+-------------+

### Example 2: Override the existing field

The example show to override the exist age field with age plus 1.
#### Example 2: Override existing fields

PPL query:
The following PPL example shows how to use \`eval\` to override an existing field. In this example, the existing field \`age\` is overridden by the \`age\` field plus 1.

os> source=accounts | eval age = age + 1 | fields age ;
fetched rows / total rows = 4/4
Expand All @@ -55,13 +48,9 @@ PPL query:
| 34 |
+-------+

### Example 3: Create the new field with field defined in eval

The example show to create a new field ddAge with field defined in eval
command. The new field ddAge is the evaluation result of doubleAge
multiply by 2, the doubleAge is defined in the eval command.
#### Example 3: Create new fields based on the fields defined in the \`eval\` expression

PPL query:
The following PPL example shows how to use \`eval\` to create a new field based on the fields defined in the \`eval\` esperession. In this example, the new field \`ddAge\` is the evaluation result of the \`doubleAge\` field multiplied by 2. The doubleAge is defined in the eval command.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | eval doubleAge = age * 2, ddAge = doubleAge * 2 | fields age, doubleAge, ddAge ;
fetched rows / total rows = 4/4
Expand All @@ -74,6 +63,6 @@ PPL query:
| 33 | 66 | 132 |
+-------+-------------+---------+

#### Limitation
The \`eval\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node.
### Limitation
The \`eval\` command is not rewritten to Query DSL. It's only run on the coordinating node.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
`;
21 changes: 7 additions & 14 deletions public/components/common/helpers/ppl_docs/commands/fields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,18 @@ export const fieldsCmd = `## fields
---
### Description

Using \`fields\` command to keep or remove fields from the search result.
Use the \`fields\` command to specify the fields that should be included or excluded in the search results.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Syntax

fields \[+\|-\] <field-list>

- index: optional. if the plus (+) is used, only the fields specified
in the field list will be keep. if the minus (-) is used, all the
fields specified in the field list will be removed. **Default** +
- field list: mandatory. comma-delimited keep or remove fields.
- \`field-list\`: Required. Comma-separated list of fields to keep or remove.
- \`index\`: Optional. If the plus sign \`+\` is used, only the fields specified in the field list will be included. If the minus \`-\` is used, all the fields specified in the field list will be excluded. **Default** \`+\`
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Example 1: Select specified fields from result
#### Example 1: Select specified fields from the search result

The example show fetch account\_number, firstname and lastname fields
from search results.

PPL query:
The following PPL query example shows how to retrieve the \`account\_number\`,\`firstname\`, and \`lastname\` fields\` from the search results.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | fields account_number, firstname, lastname;
fetched rows / total rows = 4/4
Expand All @@ -36,11 +31,9 @@ PPL query:
| 18 | Dale | Adams |
+------------------+-------------+------------+

### Example 2: Remove specified fields from result

The example show fetch remove account\_number field from search results.
#### Example 2: Remove specified fields from the search results

PPL query:
The following PPL query example shows how to remove the \`account\_number\` field from the search results.

os> source=accounts | fields account_number, firstname, lastname | fields - account_number ;
fetched rows / total rows = 4/4
Expand Down
20 changes: 7 additions & 13 deletions public/components/common/helpers/ppl_docs/commands/head.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,17 @@ export const headCmd = `## head

### Description

The \`head\` command returns the first N number of specified results in
search order.
Use the \`head\` command to return the first N number of lines from a search result.

### Syntax

head \[N\]

- N: optional. number of results to return. **Default:** 10
- \`N\`: Optional. The number of results you want to see. Default is 10.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Example 1: Get first 10 results

The example show first 10 results from accounts index.

PPL query:
#### Example 1: Get first 10 results
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

The following PPL query example shows how to use \`head\` to see the first 10 search results.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
os> source=accounts | fields firstname, age | head;
fetched rows / total rows = 10/10
+---------------+-----------+
Expand All @@ -40,11 +36,9 @@ PPL query:
| Fulton | 23 |
+---------------+-----------+

### Example 2: Get first N results

The example show first N results from accounts index.
#### Example 2: Get first N results
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

PPL query:
The following PPL query example shows how to use \`head\` to get a speficied number of search results. In this example, N is 3.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | fields firstname, age | head 3;
fetched rows / total rows = 3/3
Expand All @@ -57,5 +51,5 @@ PPL query:
+---------------+-----------+

#### Limitation
The \`head\` command is not rewritten to OpenSearch DSL, it is only executed on the coordination node.
The \`head\` command is not rewritten to Query DSL. It's only run on the coordinating node.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
`;
58 changes: 17 additions & 41 deletions public/components/common/helpers/ppl_docs/commands/parse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,22 @@ export const parseCmd = `## parse
---
### Description

The \`parse\` command parses a text field using a regular expression and append
the result to the search result.
Use the \`parse\` command to extract information from a text field using a regular expression and add it to the search result.

### Syntax

parse <field> <regular-expression>

- field: mandatory. The field must be a text field.
- regular-expression: mandatory. The regular expression used to extract new
fields from given text field. If a new field name already exists, it will
replace the original field.
- \`field\`: Required. Must be a text field.
- \`regular-expression\`: Required. The regular expression to extract new fields from a text field. It replaces the original field if a new field name exists.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Regular Expression
### Regular expression

The regular expression is used to match the whole text field of each document
with Java regex engine. Each named capture group in the expression will become
a new \`STRING\` field.
Use the Java regular expression engine to match the entire text field of each document. Each named capture group in the expression will be converted to a new \`string\` field.

### Example 1: Create the new field
#### Example 1: Create a new field

The example shows how to create new field \`host\` for each document. \`host\`
will be the host name after \`@\` in \`email\` field. Parsing a null field will
return an empty string.

PPL query:
The following PPL query example shows how to create new field \`host\` for each document. \`host\` becomes the host name after the at symbol (\`@\`) in the \`email\` field. Parsing a null field returns an empty string.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

os> source=accounts | parse email '.+@(?<host>.+)' | fields email, host ;
fetched rows / total rows = 4/4
Expand All @@ -44,12 +35,9 @@ PPL query:
| daleadams@boink.com | boink.com |
+-----------------------+------------+

### Example 2: Override the existing field

The example shows how to override the existing \`address\` field with street
number removed.
#### Example 2: Override an existing field

PPL query:
The following PPL query example shows how to override the existing \`address\` field while excluding the street number.

os> source=accounts | parse address '\\d+ (?<address>.+)' | fields address ;
fetched rows / total rows = 4/4
Expand All @@ -62,12 +50,9 @@ PPL query:
| Hutchinson Court |
+------------------+

### Example 3: Filter and sort by casted parsed field
#### Example 3: Filter and sort by casted-parsed field

The example shows how to sort street numbers that are higher than 500 in
\`address\` field.

PPL query:
The following PPL query example shows how to sort street numbers that are greater than 500 in the \`address\` field.

os> source=accounts | parse address '(?<streetNumber>\d+) (?<street>.+)' | where cast(streetNumber as int) > 500 | sort num(streetNumber) | fields streetNumber, street ;
fetched rows / total rows = 3/3
Expand All @@ -81,30 +66,21 @@ PPL query:

### Limitation

There are a few limitations with parse command:

- Fields defined by parse cannot be parsed again.
The following limitations apply:

The following command will not work:
- Parsed fields cannpt be parsed again. For example, the following command isn't valid:

source=accounts | parse address '\\d+ (?<street>.+)' | parse street '\\w+ (?<road>\\w+)' ;

- Fields defined by parse cannot be overridden with other commands.

\`where\` will not match any documents since \`street\` cannot be overridden:
- Other commands cannot overwrite fields created by parsing. For example, in the following query, \`where\` doesn't match any documents because \`street\` cannot be overridden:

source=accounts | parse address '\\d+ (?<street>.+)' | eval street='1' | where street='1' ;

- The text field used by parse cannot be overridden.

\`street\` will not be successfully parsed since \`address\` is overridden:
- The text field that is parsed cannot be overridden. For example, in the following query, \`street\` is not successfully parsed because \`address\` is overridden:

source=accounts | parse address '\\d+ (?<street>.+)' | eval address='1' ;

- Fields defined by parse cannot be filtered/sorted after using them in
\`stats\` command.

\`where\` in the following command will not work:
- Fields created by parsing cannot be filtered or sorted after using them in the \`stats\` command. For example, in the following query, \`where\` isn't valid:

source=accounts | parse email '.+@(?<host>.+)' | stats avg(age) by host | where host=pyrami.com ;
`;
`;
Loading
Loading