Skip to content

Latest commit

 

History

History
423 lines (268 loc) · 22.7 KB

CHANGELOG.next.asciidoc

File metadata and controls

423 lines (268 loc) · 22.7 KB
Raw

Beats version HEAD

Breaking changes

Affecting all Beats

Auditbeat

Filebeat

Heartbeat

Metricbeat

  • Setting period for counter cache for Prometheus remote_write at least to 60sec 38553

Osquerybeat

  • Add action responses data stream, allowing osquerybeat to post action results directly to elasticsearch. 39143

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats

  • Support for multiline zookeeper logs 2496

  • Add checks to ensure reloading of units if the configuration actually changed. 34346

  • Fix namespacing on self-monitoring 32336

  • Fix namespacing on self-monitoring 32336

  • Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964

  • Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031

  • 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider

  • 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field

  • Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640

  • Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820

  • Support build of projects outside of beats directory 36126

  • Support Elastic Agent control protocol chunking support 37343

  • Upgrade elastic-agent-libs to v0.7.5. Removes obsolete "Treating the CommonName field on X.509 certificates as a host name…​" deprecation warning for 8.0. 37755

  • aws: Add credential caching for AssumeRole session tokens. 37787

  • Lower logging level to debug when attempting to configure beats with unknown fields from autodiscovered events/environments 37816[37816]

  • Set timeout of 1 minute for FQDN requests 37756

  • Fix the paths in the .cmd script added to the path by the Windows MSI to point to the new C:\Program Files installation location. elastic/elastic-stack-installers#238

  • Change cache processor documentation from write_period to write_interval. 38561

  • Fix cache processor expiries heap cleanup on partial file writes. 38561

  • Fix cache processor expiries infinite growth when large a large TTL is used and recurring keys are cached. 38561

  • Fix parsing of RFC 3164 process IDs in syslog processor. 38947 38982

Auditbeat - Set field types to correctly match ECS in sessionmd processor 38955 38994 - Fix failing to enrich process events in sessionmd processor 38955 39173 39243 - Prevent scenario of losing children-related file events in a directory for recursive fsnotify backend of auditbeat file integrity module 39133 - Allow extra syscalls by auditbeat required in FIM with kprobes back-end 39361 - Fix losing events in FIM for OS X by allowing always to walk an added directory to monitor 39362

Filebeat

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

  • Fix panic when sqs input metrics getter is invoked 36101 36077

  • Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

  • Fix Filebeat Cisco module with missing escape character 36325 36326

  • Added a fix for Crowdstrike pipeline handling process arrays 36496

  • Fix m365_defender cursor value and query building. 37116

  • Fix TCP/UDP metric queue length parsing base. 37714

  • Update github.com/lestrrat-go/jwx dependency. 37799

  • [threatintel] MISP pagination fixes 37898

  • Fix file handle leak when handling errors in filestream 37973

  • Fix a race condition that could crash Filebeat with a "negative WaitGroup counter" error 38094

  • Prevent HTTPJSON holding response bodies between executions. 35219 38116

  • Fix "failed processing S3 event for object key" error on aws-s3 input when key contains the "+" character 38012 38125

  • Fix duplicated addition of regexp extension in CEL input. 38181

  • Fix the incorrect values generated by the uri_parts processor. 38216

  • Fix HTTPJSON handling of empty object bodies in POST requests. 33961 38290

  • Fix PEM key validation for CEL and HTTPJSON inputs. 38405

  • Fix filebeat gcs input panic 38407

  • Rename activity_guid to activity_id in ETW input events to suit other Windows inputs. 38530

  • Add missing provider registration and fix published entity for Active Directory entityanalytics provider. 38645

  • Fix handling of un-parsed JSON in O365 module. 37800 38709

  • Fix filestream’s registry GC: registry entries are now removed from the in-memory and disk store when they’re older than the set TTL 36761 38488

  • Fix indexing failures by re-enabling event normalisation in netflow input. 38703 38780

  • Fix handling of truncated files in Filestream 38070 38416

  • Fix panic when more than 32767 pipeline clients are active. 38197 38556

  • Fix filestream’s registry GC: registry entries are now removed from the in-memory and disk store when they’re older than the set TTL 36761 38488

  • [threatintel] MISP splitting fix for empty responses 38739 38917

  • Fix a bug in cloudwatch task allocation that could skip some logs 38918 38953

  • Prevent GCP Pub/Sub input blockage by increasing default value of max_outstanding_messages 35029 38985

  • entity-analytics input: Improve structured logging. 38990

  • Fix config validation for CEL and HTTPJSON inputs when using password grant authentication and client.id or client.secret are not present. 38962

  • Updated Websocket input title to align with existing inputs 39006

  • Restore netflow input on Windows 39024

  • Upgrade azure-event-hubs-go and azure-storage-blob-go dependencies. 38861

  • Fix concurrency/error handling bugs in the AWS S3 input that could drop data and prevent ingestion of large buckets. 39131

  • Fix EntraID query handling. 39419 39420

  • Fix request trace filename handling in http_endpoint input. 39410

Heartbeat

  • Fix panics when parsing dereferencing invalid parsed url. 34702

  • Fix setuid root when running under cgroups v2. 37794

  • Adjust State loader to only retry when response code status is 5xx 37981

  • Reset prctl dumpable flag after cap drop. 38269

  • Redact synthexec cmd output. 39535

Heartbeat

Metricbeat

  • Fix Azure Monitor 429 error by causing metricbeat to retry the request again. 38294

  • Fix fields not being parsed correctly in postgresql/database 25301 37720

  • rabbitmq/queue - Change the mapping type of rabbitmq.queue.consumers.utilisation.pct to scaled_float from long because the values fall within the range of [0.0, 1.0]. Previously, conversion to integer resulted in reporting either 0 or 1.

  • Fix timeout caused by the retrival of which indices are hidden 39165

  • Fix Azure Monitor support for multiple aggregation types 39192 39204

  • Fix for MySQL/Performance - Query failure for MySQL versions below v8.0.1, for performance metric quantile_95. 38710

Osquerybeat

Packetbeat

Winlogbeat

  • Fix error handling in perfmon metrics. 38140 39404

Elastic Logging Plugin

Added

Affecting all Beats

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • dns processor: Add support for forward lookups (A, AAAA, and TXT). 11416 36394

  • [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor 36506

  • allow queue configuration settings to be set under the output. 35615 36788

  • Beats will now connect to older Elasticsearch instances by default 36884

  • Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments

  • elasticsearch output now supports idle_connection_timeout. 35615 36843

  • Update to Go 1.21.10. 39467

  • Enable early event encoding in the Elasticsearch output, improving cpu and memory use 38572

  • The environment variable BEATS_ADD_CLOUD_METADATA_PROVIDERS overrides configured/default add_cloud_metadata providers 38669

  • Introduce log message for not supported annotations for Hints based autodiscover 38213

  • Add persistent volume claim name to volume if available 38839

Auditbeat

  • Added add_session_metadata processor, which enables session viewer on Auditbeat data. 37640

  • Add linux capabilities to processes in the system/process. 37453

  • Add opt-in eBPF backend for file_integrity module. 37223

  • Add linux capabilities to processes in the system/process. 37453

  • Add opt-in eBPF backend for file_integrity module. 37223

  • Add process data to file events (Linux only, eBPF backend). 38199

  • Add container id to file events (Linux only, eBPF backend). 38328

  • Add procfs backend to the add_session_metadata processor. 38799

  • Add process.entity_id, process.group.name and process.group.id in add_process_metadata processor. Make fim module with kprobes backend to always add an appropriately configured add_process_metadata processor to enrich file events 38776

  • Reduce data size for add_session_metadata processor by removing unneeded fields 39500

  • Enrich process events with user and group names, with add_session_metadata processor 39537

Auditbeat

Filebeat

  • Adding Saved Object name field to Kibana audit logs 38307

  • Update SQL input documentation regarding Oracle DSNs 37590

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Add oracle authentication messages parsing 35127

  • Add clean_session configuration setting for MQTT input. 16204

  • Add support for a simplified input configuraton when running under Elastic-Agent 36390

  • Added support for Okta OAuth2 provider in the CEL input. 36336 36521

  • Added support for new features & removed partial save mechanism in the Azure Blob Storage input. 35126 36690

  • Added support for new features and removed partial save mechanism in the GCS input. 35847 36713

  • Use filestream input with file_identity.fingerprint as default for hints autodiscover. 35984 36950

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

  • Made Azure Blob Storage input GA and updated docs accordingly. 37128

  • Made GCS input GA and updated docs accordingly. 37127

  • Suppress and log max HTTP request retry errors in CEL input. 37160

  • Prevent CEL input from re-entering the eval loop when an evaluation failed. 37161

  • Update CEL extensions library to v1.7.0. 37172

  • Add support for complete URL replacement in HTTPJSON chain steps. 37486

  • Add support for user-defined query selection in EntraID entity analytics provider. 37653

  • Update CEL extensions library to v1.8.0 to provide runtime error location reporting. 37304 37718

  • Add request trace logging for chained API requests. 36551 37682

  • Relax TCP/UDP metric polling expectations to improve metric collection. 37714

  • Add support for PEM-based Okta auth in HTTPJSON. 37772

  • Prevent complete loss of long request trace data. 37826 37836

  • Added experimental version of the Websocket Input. 37774

  • Add support for PEM-based Okta auth in CEL. 37813

  • Add Salesforce input. 37331

  • Add ETW input. 36915

  • Update CEL mito extensions to v1.9.0 to add keys/values helper. 37971

  • Add logging for cache processor file reads and writes. 38052

  • Add parseDateInTZ value template for the HTTPJSON input 37738

  • Support VPC endpoint for aws-s3 input SQS queue url. 38189

  • Improve rate limit handling by HTTPJSON 36207 38161 38237

  • Add parseDateInTZ value template for the HTTPJSON input. 37738

  • Add support for complex event objects in the HTTP Endpoint input. 37910 38193

  • Parse more fields from Elasticsearch slowlogs 38295

  • Update CEL mito extensions to v1.10.0 to add base64 decode functions. 38504

  • Add support for Active Directory an entity analytics provider. 37919

  • Add AWS AWSHealth metricset. 38370

  • Add debugging breadcrumb to logs when writing request trace log. 38636

  • added benchmark input 37437

  • added benchmark input and discard output 37437

  • Ensure all responses sent by HTTP Endpoint are HTML-escaped. 39329

  • Update CEL mito extensions to v1.11.0 to improve type checking. 39460

  • Improve logging of request and response with request trace logging in error conditions. 39455

  • Add HTTP metrics to CEL input. 39501 39503

  • Add default user-agent to CEL HTTP requests. 39502 39587

  • Improve reindexing support in security module pipelines. 38224 https://github.com/elastic/beats/pull/

  • Improve reindexing support in security module pipelines. 38224 39588

  • Make HTTP Endpoint input GA. 38979 39410

Auditbeat

Libbeat

  • Add support for linux capabilities in add_process_metadata. 38252

Heartbeat

  • Added status to monitor run log report.

Metricbeat

  • Add support for shards_stats.total_count in Elasticsearch Monitoring data. 38891

  • Add new fields to configure the lease duration, retry and renew when using leader elector with kubernetes autodiscover.https://github.com/elastic/beats/pull/38471[38471]

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Add metrics grouping by dimensions and time to Azure app insights 36634

  • Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms 36647

  • Add linux IO metrics to system/process 37213

  • Add new memory/cgroup metrics to Kibana module 37232

  • Add SSL support to mysql module 37997

  • Add SSL support for aerospike module 38126

  • Add last_terminated_timestamp metric in kubernetes module 39200 3802

  • Add pod.status.ready_time and pod.status.reason metrics in kubernetes module 39316

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

  • Use fixed size buffer at first pass for event parsing, improving throughput 39530 39544

Functionbeat

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

  • Deprecate syslog input in favor of syslog processor. 37555 38277

  • Deprecate o365audit input in favor of CEL input. 37719 38922

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues