[Security Solution] [Attack discovery] Alerts filtering #205070
Merged
+2,380
−77
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrew-goldstein
added
release_note:enhancement
v9.0.0
Team: SecuritySolution
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Tested locally, everything works great! Found a few things we might want to have a look into: 1. Seems like
|
e40pud
approved these changes
Dec 23, 2024
| ); | ||
|
|
||
| // search bar query: | ||
| const [query, setQuery] = // useState<Query>(getDefaultQuery()); |
There was a problem hiding this comment.
do we still need // useState<Query>(getDefaultQuery());?
There was a problem hiding this comment.
Resolved by commit 4e43e1a
| setApproximateFutureTime(null); | ||
|
|
||
| // call the internal API to generate attack discoveries: | ||
| const rawResponse = await http.fetch('/internal/elastic_assistant/attack_discovery', { |
There was a problem hiding this comment.
NIT: any reason not to use http.post here?
| const { theme } = useKibana().services; | ||
| const dateFormat = useDateFormat(); | ||
| const timeZone = useTimeZone(); |
There was a problem hiding this comment.
Just curious why we would want to use time zone. Might be wrong, but if I remember correctly we do not use it in other places where we deal with alerts in security solution.
There was a problem hiding this comment.
Resolved by commit fdc75d0
andrew-goldstein
force-pushed
the
attack_discovery_filtering
branch
from
d7ff8bb to
05d6d35
Compare
Great catch, thank you @e40pud! ✅ Resolved by commit 1dca994 In the commit above, users accumulate an "unsubmitted" query as they type in the search bar, but have not pressed the To match the behavior of Discover,
Also to match the behavior of Discover, we do NOT call
This timeline error does not appear to to be related to Attack discovery. It is reproducible on any other Security Solution page that includes Timeline, i.e. Alerts.
At the time of this writing, Kibana Tasks are only canceled on a timeout or Kibana shutdown. As a result, [Security solution] Attack discovery background task and persistence does not use tasks for cancellation; there is potential for race conditions like the one you observed. In the short term, we may be able to reduce or eliminate the issue you observed using Elasticsearch's document versioning feature. @jamesspi let's include this in the refactoring of background tasks, per our recent offline discussion. Thank you. |
|
/ci |
|
/ci |
e40pud
approved these changes
Dec 24, 2024
| // 2) The user clicks the refresh button | ||
| // | ||
| // Also to match the behavior of Discover, we must NOT call `setQuery` with | ||
| // the `unSubmittedQuery` query when the user clicks the `Save` button button. |
There was a problem hiding this comment.
Suggested change
| // the `unSubmittedQuery` query when the user clicks the `Save` button button. | |
| // the `unSubmittedQuery` query when the user clicks the `Save` button. |
There was a problem hiding this comment.
Resolved by commit 4f83f49
| // call the internal API to generate attack discoveries: | ||
| const rawResponse = await http.post('/internal/elastic_assistant/attack_discovery', { | ||
| body: JSON.stringify(bodyWithOverrides), | ||
| method: 'POST', |
There was a problem hiding this comment.
NIT: I believe we don't need method: 'POST' when we use http.post explicitly.
There was a problem hiding this comment.
Resolved by commit 4f83f49
 This PR enhances _Attack discovery_ by providing users additional control over which alerts are included as context to the large language model (LLM). Using the new resizeable _Attack discovery settings flyout_, users may: - Filter alerts via a search bar and filters - Control the time window (previously fixed to `Last 24 hrs`) ### Before (feature flag disabled) Previously, users could only set the number of alerts sent as context to the LLM via a modal:  ### After (feature flag enabled) The new Attack discovery settings flyout replaces the modal:  It has two tabs, _Alert summary_ and _Alerts preview_. ### Alert summary The _Alert summary_ Lens embeddable counts the selected field name via an ES|QL query:  The Alert summary query is an aggregation. It does NOT display the details of individual alerts. ### Alerts preview The _Alerts preview_ Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:  Users may resize the settings flyout to view all the fields in the Alerts preview. ### Feature flag Enable the `attackDiscoveryAlertFiltering` feature flag via the following setting in `kibana.dev.yml`: ```yaml xpack.securitySolution.enableExperimental: - 'attackDiscoveryAlertFiltering' ``` Enabling the feature flag: - Replaces the `Settings` modal with the `Attack discovery settings` flyout - Includes additional `start`, `end`, and `filters` parameters in requests to generate Attack discoveries - Enables new loading messages ### Details #### Loading messages The loading messages displayed when generating Attack discoveries were updated to render three types of date ranges: 1) The default date range (`Last 24 hours`), which displays the same message seen in previous versions:  2) Relative date ranges:  3) Absolute date ranges:  #### Filtering preferences Alert filtering preferences are stored in local storage. This PR adds the following new local storage keys: ``` elasticAssistantDefault.attackDiscovery.default.end elasticAssistantDefault.attackDiscovery.default.filters elasticAssistantDefault.attackDiscovery.default.query elasticAssistantDefault.attackDiscovery.default.start ``` Users may use the `Reset` button in the Attack discovery settings flyout to restore the above to their defaults. #### Known limitations The following known limitations in this PR may be mitigated in follow-up PRs: #### Table cell hover actions are disabled Table cell actions, i.e. `Filter for` and `Filter out` are disabled in the `Alert summary` and `Alerts preview` tables. The actions are disabled because custom cell hover actions registered in `x-pack/solutions/security/plugins/security_solution/public/app/actions/register.ts` do NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's _Dashboard_ app. ##### Default table sort indicators are hidden The `Alert summary` and `Alerts preview` tables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries. The tables _should_ display default sort indicators, as illustrated by the screenshots below:   The default indicators are hidden in this PR as a workaround for an error that occurs in `EuiDataGrid` when switching tabs when the column sort indicators are enabled: ``` TypeError: Cannot read properties of undefined (reading 'split') ``` To re-enable the sort indicators, `DEFAULT_ALERT_SUMMARY_SORT` and `DEFAULT_ALERTS_PREVIEW_SORT` must respectively be passed as the `sorting` prop to the `PreviewTab` in `x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx`, as illustrated by the following code: ```typescript <PreviewTab dataTestSubj={ALERT_SUMMARY_TEST_SUBJ} embeddableId={SUMMARY_TAB_EMBEDDABLE_ID} end={end} filters={filters} getLensAttributes={getAlertSummaryLensAttributes} getPreviewEsqlQuery={getAlertSummaryEsqlQuery} maxAlerts={maxAlerts} query={query} setTableStackBy0={setAlertSummaryStackBy0} start={start} sorting={DEFAULT_ALERT_SUMMARY_SORT} // <-- enables the sort indicator tableStackBy0={alertSummaryStackBy0} /> ``` ##### Selected date range not persisted The `start` and `end` date range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation.
…ar, and call `setQuery` with it when the user 1) selects a new time range, or 2) clicks the refresh button.
andrew-goldstein
force-pushed
the
attack_discovery_filtering
branch
from
a89113b to
4f83f49
Compare
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
Unknown metric groupsAPI count
History
|
|
Starting backport for target branches: 8.x |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Dec 24, 2024
## [Security Solution] [Attack discovery] Alerts filtering  This PR enhances _Attack discovery_ by providing users additional control over which alerts are included as context to the large language model (LLM). Using the new resizeable _Attack discovery settings flyout_, users may: - Filter alerts via a search bar and filters - Control the time window (previously fixed to `Last 24 hrs`) ### Before (feature flag disabled) Previously, users could only set the number of alerts sent as context to the LLM via a modal:  ### After (feature flag enabled) The new Attack discovery settings flyout replaces the modal:  It has two tabs, _Alert summary_ and _Alerts preview_. ### Alert summary The _Alert summary_ Lens embeddable counts the selected field name via an ES|QL query:  The Alert summary query is an aggregation. It does NOT display the details of individual alerts. ### Alerts preview The _Alerts preview_ Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:  Users may resize the settings flyout to view all the fields in the Alerts preview. ### Feature flag Enable the `attackDiscoveryAlertFiltering` feature flag via the following setting in `kibana.dev.yml`: ```yaml xpack.securitySolution.enableExperimental: - 'attackDiscoveryAlertFiltering' ``` Enabling the feature flag: - Replaces the `Settings` modal with the `Attack discovery settings` flyout - Includes additional `start`, `end`, and `filters` parameters in requests to generate Attack discoveries - Enables new loading messages ### Details #### Loading messages The loading messages displayed when generating Attack discoveries were updated to render three types of date ranges: 1) The default date range (`Last 24 hours`), which displays the same message seen in previous versions:  2) Relative date ranges:  3) Absolute date ranges:  #### Filtering preferences Alert filtering preferences are stored in local storage. This PR adds the following new local storage keys: ``` elasticAssistantDefault.attackDiscovery.default.end elasticAssistantDefault.attackDiscovery.default.filters elasticAssistantDefault.attackDiscovery.default.query elasticAssistantDefault.attackDiscovery.default.start ``` Users may use the `Reset` button in the Attack discovery settings flyout to restore the above to their defaults. #### Known limitations The following known limitations in this PR may be mitigated in follow-up PRs: #### Table cell hover actions are disabled Table cell actions, i.e. `Filter for` and `Filter out` are disabled in the `Alert summary` and `Alerts preview` tables. The actions are disabled because custom cell hover actions registered in `x-pack/solutions/security/plugins/security_solution/public/app/actions/register.ts` do NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's _Dashboard_ app. ##### Default table sort indicators are hidden The `Alert summary` and `Alerts preview` tables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries. The tables _should_ display default sort indicators, as illustrated by the screenshots below:   The default indicators are hidden in this PR as a workaround for an error that occurs in `EuiDataGrid` when switching tabs when the column sort indicators are enabled: ``` TypeError: Cannot read properties of undefined (reading 'split') ``` To re-enable the sort indicators, `DEFAULT_ALERT_SUMMARY_SORT` and `DEFAULT_ALERTS_PREVIEW_SORT` must respectively be passed as the `sorting` prop to the `PreviewTab` in `x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx`, as illustrated by the following code: ```typescript <PreviewTab dataTestSubj={ALERT_SUMMARY_TEST_SUBJ} embeddableId={SUMMARY_TAB_EMBEDDABLE_ID} end={end} filters={filters} getLensAttributes={getAlertSummaryLensAttributes} getPreviewEsqlQuery={getAlertSummaryEsqlQuery} maxAlerts={maxAlerts} query={query} setTableStackBy0={setAlertSummaryStackBy0} start={start} sorting={DEFAULT_ALERT_SUMMARY_SORT} // <-- enables the sort indicator tableStackBy0={alertSummaryStackBy0} /> ``` ##### Selected date range not persisted The `start` and `end` date range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation. (cherry picked from commit 681d40e)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Dec 24, 2024
… (#205137) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] [Attack discovery] Alerts filtering (#205070)](#205070) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Andrew Macri","email":"andrew.macri@elastic.co"},"sourceCommit":{"committedDate":"2024-12-24T10:49:10Z","message":"[Security Solution] [Attack discovery] Alerts filtering (#205070)\n\n## [Security Solution] [Attack discovery] Alerts filtering\r\n\r\n\r\n\r\nThis PR enhances _Attack discovery_ by providing users additional control over which alerts are included as context to the large language model (LLM).\r\n\r\nUsing the new resizeable _Attack discovery settings flyout_, users may:\r\n\r\n- Filter alerts via a search bar and filters\r\n- Control the time window (previously fixed to `Last 24 hrs`)\r\n\r\n### Before (feature flag disabled)\r\n\r\nPreviously, users could only set the number of alerts sent as context to the LLM via a modal:\r\n\r\n\r\n\r\n### After (feature flag enabled)\r\n\r\nThe new Attack discovery settings flyout replaces the modal:\r\n\r\n\r\n\r\nIt has two tabs, _Alert summary_ and _Alerts preview_.\r\n\r\n### Alert summary\r\n\r\nThe _Alert summary_ Lens embeddable counts the selected field name via an ES|QL query:\r\n\r\n\r\n\r\nThe Alert summary query is an aggregation. It does NOT display the details of individual alerts.\r\n\r\n### Alerts preview\r\n\r\nThe _Alerts preview_ Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:\r\n\r\n\r\n\r\nUsers may resize the settings flyout to view all the fields in the Alerts preview.\r\n\r\n### Feature flag\r\n\r\nEnable the `attackDiscoveryAlertFiltering` feature flag via the following setting in `kibana.dev.yml`:\r\n\r\n```yaml\r\nxpack.securitySolution.enableExperimental:\r\n - 'attackDiscoveryAlertFiltering'\r\n```\r\n\r\nEnabling the feature flag:\r\n\r\n- Replaces the `Settings` modal with the `Attack discovery settings` flyout\r\n- Includes additional `start`, `end`, and `filters` parameters in requests to generate Attack discoveries\r\n- Enables new loading messages\r\n\r\n### Details\r\n\r\n#### Loading messages\r\n\r\nThe loading messages displayed when generating Attack discoveries were updated to render three types of date ranges:\r\n\r\n1) The default date range (`Last 24 hours`), which displays the same message seen in previous versions:\r\n\r\n\r\n\r\n2) Relative date ranges:\r\n\r\n\r\n\r\n3) Absolute date ranges:\r\n\r\n\r\n\r\n#### Filtering preferences\r\n\r\nAlert filtering preferences are stored in local storage.\r\n\r\nThis PR adds the following new local storage keys:\r\n\r\n```\r\nelasticAssistantDefault.attackDiscovery.default.end\r\nelasticAssistantDefault.attackDiscovery.default.filters\r\nelasticAssistantDefault.attackDiscovery.default.query\r\nelasticAssistantDefault.attackDiscovery.default.start\r\n```\r\n\r\nUsers may use the `Reset` button in the Attack discovery settings flyout to restore the above to their defaults.\r\n\r\n#### Known limitations\r\n\r\nThe following known limitations in this PR may be mitigated in follow-up PRs:\r\n\r\n#### Table cell hover actions are disabled\r\n\r\nTable cell actions, i.e. `Filter for` and `Filter out` are disabled in the `Alert summary` and `Alerts preview` tables.\r\n\r\nThe actions are disabled because custom cell hover actions registered in `x-pack/solutions/security/plugins/security_solution/public/app/actions/register.ts` do NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's _Dashboard_ app.\r\n\r\n##### Default table sort indicators are hidden\r\n\r\nThe `Alert summary` and `Alerts preview` tables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries.\r\n\r\nThe tables _should_ display default sort indicators, as illustrated by the screenshots below:\r\n\r\n\r\n\r\n\r\n\r\nThe default indicators are hidden in this PR as a workaround for an error that occurs in `EuiDataGrid` when switching tabs when the column sort indicators are enabled:\r\n\r\n```\r\nTypeError: Cannot read properties of undefined (reading 'split')\r\n```\r\n\r\nTo re-enable the sort indicators, `DEFAULT_ALERT_SUMMARY_SORT` and `DEFAULT_ALERTS_PREVIEW_SORT` must respectively be passed as the `sorting` prop to the `PreviewTab` in `x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx`, as illustrated by the following code:\r\n\r\n```typescript\r\n <PreviewTab\r\n dataTestSubj={ALERT_SUMMARY_TEST_SUBJ}\r\n embeddableId={SUMMARY_TAB_EMBEDDABLE_ID}\r\n end={end}\r\n filters={filters}\r\n getLensAttributes={getAlertSummaryLensAttributes}\r\n getPreviewEsqlQuery={getAlertSummaryEsqlQuery}\r\n maxAlerts={maxAlerts}\r\n query={query}\r\n setTableStackBy0={setAlertSummaryStackBy0}\r\n start={start}\r\n sorting={DEFAULT_ALERT_SUMMARY_SORT} // <-- enables the sort indicator\r\n tableStackBy0={alertSummaryStackBy0}\r\n />\r\n```\r\n\r\n##### Selected date range not persisted\r\n\r\nThe `start` and `end` date range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation.","sha":"681d40eee64316bea85d04077f918bd7121988b9","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","Team: SecuritySolution","ci:cloud-deploy","ci:cloud-persist-deployment","Team:Security Generative AI","backport:version","v8.18.0"],"title":"[Security Solution] [Attack discovery] Alerts filtering","number":205070,"url":"https://github.com/elastic/kibana/pull/205070","mergeCommit":{"message":"[Security Solution] [Attack discovery] Alerts filtering (#205070)\n\n## [Security Solution] [Attack discovery] Alerts filtering\r\n\r\n\r\n\r\nThis PR enhances _Attack discovery_ by providing users additional control over which alerts are included as context to the large language model (LLM).\r\n\r\nUsing the new resizeable _Attack discovery settings flyout_, users may:\r\n\r\n- Filter alerts via a search bar and filters\r\n- Control the time window (previously fixed to `Last 24 hrs`)\r\n\r\n### Before (feature flag disabled)\r\n\r\nPreviously, users could only set the number of alerts sent as context to the LLM via a modal:\r\n\r\n\r\n\r\n### After (feature flag enabled)\r\n\r\nThe new Attack discovery settings flyout replaces the modal:\r\n\r\n\r\n\r\nIt has two tabs, _Alert summary_ and _Alerts preview_.\r\n\r\n### Alert summary\r\n\r\nThe _Alert summary_ Lens embeddable counts the selected field name via an ES|QL query:\r\n\r\n\r\n\r\nThe Alert summary query is an aggregation. It does NOT display the details of individual alerts.\r\n\r\n### Alerts preview\r\n\r\nThe _Alerts preview_ Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:\r\n\r\n\r\n\r\nUsers may resize the settings flyout to view all the fields in the Alerts preview.\r\n\r\n### Feature flag\r\n\r\nEnable the `attackDiscoveryAlertFiltering` feature flag via the following setting in `kibana.dev.yml`:\r\n\r\n```yaml\r\nxpack.securitySolution.enableExperimental:\r\n - 'attackDiscoveryAlertFiltering'\r\n```\r\n\r\nEnabling the feature flag:\r\n\r\n- Replaces the `Settings` modal with the `Attack discovery settings` flyout\r\n- Includes additional `start`, `end`, and `filters` parameters in requests to generate Attack discoveries\r\n- Enables new loading messages\r\n\r\n### Details\r\n\r\n#### Loading messages\r\n\r\nThe loading messages displayed when generating Attack discoveries were updated to render three types of date ranges:\r\n\r\n1) The default date range (`Last 24 hours`), which displays the same message seen in previous versions:\r\n\r\n\r\n\r\n2) Relative date ranges:\r\n\r\n\r\n\r\n3) Absolute date ranges:\r\n\r\n\r\n\r\n#### Filtering preferences\r\n\r\nAlert filtering preferences are stored in local storage.\r\n\r\nThis PR adds the following new local storage keys:\r\n\r\n```\r\nelasticAssistantDefault.attackDiscovery.default.end\r\nelasticAssistantDefault.attackDiscovery.default.filters\r\nelasticAssistantDefault.attackDiscovery.default.query\r\nelasticAssistantDefault.attackDiscovery.default.start\r\n```\r\n\r\nUsers may use the `Reset` button in the Attack discovery settings flyout to restore the above to their defaults.\r\n\r\n#### Known limitations\r\n\r\nThe following known limitations in this PR may be mitigated in follow-up PRs:\r\n\r\n#### Table cell hover actions are disabled\r\n\r\nTable cell actions, i.e. `Filter for` and `Filter out` are disabled in the `Alert summary` and `Alerts preview` tables.\r\n\r\nThe actions are disabled because custom cell hover actions registered in `x-pack/solutions/security/plugins/security_solution/public/app/actions/register.ts` do NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's _Dashboard_ app.\r\n\r\n##### Default table sort indicators are hidden\r\n\r\nThe `Alert summary` and `Alerts preview` tables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries.\r\n\r\nThe tables _should_ display default sort indicators, as illustrated by the screenshots below:\r\n\r\n\r\n\r\n\r\n\r\nThe default indicators are hidden in this PR as a workaround for an error that occurs in `EuiDataGrid` when switching tabs when the column sort indicators are enabled:\r\n\r\n```\r\nTypeError: Cannot read properties of undefined (reading 'split')\r\n```\r\n\r\nTo re-enable the sort indicators, `DEFAULT_ALERT_SUMMARY_SORT` and `DEFAULT_ALERTS_PREVIEW_SORT` must respectively be passed as the `sorting` prop to the `PreviewTab` in `x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx`, as illustrated by the following code:\r\n\r\n```typescript\r\n <PreviewTab\r\n dataTestSubj={ALERT_SUMMARY_TEST_SUBJ}\r\n embeddableId={SUMMARY_TAB_EMBEDDABLE_ID}\r\n end={end}\r\n filters={filters}\r\n getLensAttributes={getAlertSummaryLensAttributes}\r\n getPreviewEsqlQuery={getAlertSummaryEsqlQuery}\r\n maxAlerts={maxAlerts}\r\n query={query}\r\n setTableStackBy0={setAlertSummaryStackBy0}\r\n start={start}\r\n sorting={DEFAULT_ALERT_SUMMARY_SORT} // <-- enables the sort indicator\r\n tableStackBy0={alertSummaryStackBy0}\r\n />\r\n```\r\n\r\n##### Selected date range not persisted\r\n\r\nThe `start` and `end` date range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation.","sha":"681d40eee64316bea85d04077f918bd7121988b9"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205070","number":205070,"mergeCommit":{"message":"[Security Solution] [Attack discovery] Alerts filtering (#205070)\n\n## [Security Solution] [Attack discovery] Alerts filtering\r\n\r\n\r\n\r\nThis PR enhances _Attack discovery_ by providing users additional control over which alerts are included as context to the large language model (LLM).\r\n\r\nUsing the new resizeable _Attack discovery settings flyout_, users may:\r\n\r\n- Filter alerts via a search bar and filters\r\n- Control the time window (previously fixed to `Last 24 hrs`)\r\n\r\n### Before (feature flag disabled)\r\n\r\nPreviously, users could only set the number of alerts sent as context to the LLM via a modal:\r\n\r\n\r\n\r\n### After (feature flag enabled)\r\n\r\nThe new Attack discovery settings flyout replaces the modal:\r\n\r\n\r\n\r\nIt has two tabs, _Alert summary_ and _Alerts preview_.\r\n\r\n### Alert summary\r\n\r\nThe _Alert summary_ Lens embeddable counts the selected field name via an ES|QL query:\r\n\r\n\r\n\r\nThe Alert summary query is an aggregation. It does NOT display the details of individual alerts.\r\n\r\n### Alerts preview\r\n\r\nThe _Alerts preview_ Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:\r\n\r\n\r\n\r\nUsers may resize the settings flyout to view all the fields in the Alerts preview.\r\n\r\n### Feature flag\r\n\r\nEnable the `attackDiscoveryAlertFiltering` feature flag via the following setting in `kibana.dev.yml`:\r\n\r\n```yaml\r\nxpack.securitySolution.enableExperimental:\r\n - 'attackDiscoveryAlertFiltering'\r\n```\r\n\r\nEnabling the feature flag:\r\n\r\n- Replaces the `Settings` modal with the `Attack discovery settings` flyout\r\n- Includes additional `start`, `end`, and `filters` parameters in requests to generate Attack discoveries\r\n- Enables new loading messages\r\n\r\n### Details\r\n\r\n#### Loading messages\r\n\r\nThe loading messages displayed when generating Attack discoveries were updated to render three types of date ranges:\r\n\r\n1) The default date range (`Last 24 hours`), which displays the same message seen in previous versions:\r\n\r\n\r\n\r\n2) Relative date ranges:\r\n\r\n\r\n\r\n3) Absolute date ranges:\r\n\r\n\r\n\r\n#### Filtering preferences\r\n\r\nAlert filtering preferences are stored in local storage.\r\n\r\nThis PR adds the following new local storage keys:\r\n\r\n```\r\nelasticAssistantDefault.attackDiscovery.default.end\r\nelasticAssistantDefault.attackDiscovery.default.filters\r\nelasticAssistantDefault.attackDiscovery.default.query\r\nelasticAssistantDefault.attackDiscovery.default.start\r\n```\r\n\r\nUsers may use the `Reset` button in the Attack discovery settings flyout to restore the above to their defaults.\r\n\r\n#### Known limitations\r\n\r\nThe following known limitations in this PR may be mitigated in follow-up PRs:\r\n\r\n#### Table cell hover actions are disabled\r\n\r\nTable cell actions, i.e. `Filter for` and `Filter out` are disabled in the `Alert summary` and `Alerts preview` tables.\r\n\r\nThe actions are disabled because custom cell hover actions registered in `x-pack/solutions/security/plugins/security_solution/public/app/actions/register.ts` do NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's _Dashboard_ app.\r\n\r\n##### Default table sort indicators are hidden\r\n\r\nThe `Alert summary` and `Alerts preview` tables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries.\r\n\r\nThe tables _should_ display default sort indicators, as illustrated by the screenshots below:\r\n\r\n\r\n\r\n\r\n\r\nThe default indicators are hidden in this PR as a workaround for an error that occurs in `EuiDataGrid` when switching tabs when the column sort indicators are enabled:\r\n\r\n```\r\nTypeError: Cannot read properties of undefined (reading 'split')\r\n```\r\n\r\nTo re-enable the sort indicators, `DEFAULT_ALERT_SUMMARY_SORT` and `DEFAULT_ALERTS_PREVIEW_SORT` must respectively be passed as the `sorting` prop to the `PreviewTab` in `x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx`, as illustrated by the following code:\r\n\r\n```typescript\r\n <PreviewTab\r\n dataTestSubj={ALERT_SUMMARY_TEST_SUBJ}\r\n embeddableId={SUMMARY_TAB_EMBEDDABLE_ID}\r\n end={end}\r\n filters={filters}\r\n getLensAttributes={getAlertSummaryLensAttributes}\r\n getPreviewEsqlQuery={getAlertSummaryEsqlQuery}\r\n maxAlerts={maxAlerts}\r\n query={query}\r\n setTableStackBy0={setAlertSummaryStackBy0}\r\n start={start}\r\n sorting={DEFAULT_ALERT_SUMMARY_SORT} // <-- enables the sort indicator\r\n tableStackBy0={alertSummaryStackBy0}\r\n />\r\n```\r\n\r\n##### Selected date range not persisted\r\n\r\nThe `start` and `end` date range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation.","sha":"681d40eee64316bea85d04077f918bd7121988b9"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Andrew Macri <andrew.macri@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[Security Solution] [Attack discovery] Alerts filtering
This PR enhances Attack discovery by providing users additional control over which alerts are included as context to the large language model (LLM).
Using the new resizeable Attack discovery settings flyout, users may:
Last 24 hrs)Before (feature flag disabled)
Previously, users could only set the number of alerts sent as context to the LLM via a modal:
After (feature flag enabled)
The new Attack discovery settings flyout replaces the modal:
It has two tabs, Alert summary and Alerts preview.
Alert summary
The Alert summary Lens embeddable counts the selected field name via an ES|QL query:
The Alert summary query is an aggregation. It does NOT display the details of individual alerts.
Alerts preview
The Alerts preview Lens embeddable shows a preview of the actual alerts that will be sent as context via an ES|QL query:
Users may resize the settings flyout to view all the fields in the Alerts preview.
Feature flag
Enable the
attackDiscoveryAlertFilteringfeature flag via the following setting inkibana.dev.yml:Enabling the feature flag:
Settingsmodal with theAttack discovery settingsflyoutstart,end, andfiltersparameters in requests to generate Attack discoveriesDetails
Loading messages
The loading messages displayed when generating Attack discoveries were updated to render three types of date ranges:
Last 24 hours), which displays the same message seen in previous versions:Filtering preferences
Alert filtering preferences are stored in local storage.
This PR adds the following new local storage keys:
Users may use the
Resetbutton in the Attack discovery settings flyout to restore the above to their defaults.Known limitations
The following known limitations in this PR may be mitigated in follow-up PRs:
Table cell hover actions are disabled
Table cell actions, i.e.
Filter forandFilter outare disabled in theAlert summaryandAlerts previewtables.The actions are disabled because custom cell hover actions registered in
x-pack/solutions/security/plugins/security_solution/public/app/actions/register.tsdo NOT appear to receive field metadata (i.e. the name of the field being hovered over) when the action is triggered. This limitation also appears to apply to ad hoc ES|QL visualizations created via Lens in Kibana's Dashboard app.Default table sort indicators are hidden
The
Alert summaryandAlerts previewtables are sorted descending by Count, and Risk score, respectively, via their ES|QL queries.The tables should display default sort indicators, as illustrated by the screenshots below:
The default indicators are hidden in this PR as a workaround for an error that occurs in
EuiDataGridwhen switching tabs when the column sort indicators are enabled:To re-enable the sort indicators,
DEFAULT_ALERT_SUMMARY_SORTandDEFAULT_ALERTS_PREVIEW_SORTmust respectively be passed as thesortingprop to thePreviewTabinx-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/settings_flyout/alert_selection/helpers/get_tabs/index.tsx, as illustrated by the following code:Selected date range not persisted
The
startandenddate range selected when a user starts generation are not (yet) persisted in Elasticsearch. As a result, the loading message always displays the currently configured range, rather than the range selected at the start of generation.