Changes to this project are documented in this file. More detail and links can be found in the Telemetry Streaming Document Revision History.
Important: Starting from BIG-IP Telemetry Streaming version 1.36.0, BIG-IP Telemetry Streaming no longer supports BIG-IP 13.1 to 15.0.x. However, if you are still using the BIG-IP 13.1 to 15.0.x versions, you can use BIG-IP Telemetry Streaming 1.35.0 or earlier.
- New "httpTimeout" option to configure HTTP timeout value for incoming REST API requests to Pull Consumers
- MBIPMP-37620: EndpointLoader support pagination
- MBIPMP-37621: Support for additional system poller settings: workers and httpAgentOpts
- Event Listener throws an uncaught error when the buffer pointer was set to a wrong position
- Event Listener allocates more memory than required for chunked data
- MBIPMP-41257: Update iHealth to use new authentication API
- MBIPMP-37253: Update memoryMonitor.provisionedMemory limits (should allow to provision up to runtime.maxHeapSize value)
- MBIPMP-37255: Removed node 4 testing due to the end of support for BIG-IP 13.1 : (Supported BIG-IP Versions).
- MBIPMP-37374: Update Kafka consumer to support multi-host, formatting and custom opts
- MBIPMP-37631: Update Splunk consumer to use data queues, HTTP agent options, events batching
- Update Telemetry_System to require
passphrasewhenusernamedefined. - Set default logging level to "info" instead of "debug".
- Allow to set single-digit minute value for iHealth polling interval
- MBIPMP-37374: Kafka consumer support for ZooKeepeer
- NEXTACC-414: Add Resource Monitor
- Runtime configuration options
- Event Listener performance and memory usage improvements
- NEXTACC-387: Update Event Listener to utilize memory and CPU more efficient
- NEXTACC-396: Add more "build" related metadata to /info endpoint
- NEXTACC-402: Add 'verbose' logging to reduce amount of unnecessary log messages
- TS-752: Update npm packages (@f5devcentral/f5-teem from 1.5.0 to 1.6.1, @grpc/grpc-js from 1.7.1 to 1.8.18, @grpc/proto-loader from 0.7.3 to 0.7.8, @opentelemetry/api from 1.2.0 to 1.4.1, @opentelemetry/exporter-metrics-otlp-grpc from 0.33.0 to 0.41.0, @opentelemetry/exporter-metrics-otlp-http from 0.33.0 to 0.41.0, @opentelemetry/exporter-metrics-otlp-proto from 0.33.0 to 0.41.0, @opentelemetry/sdk-metrics from 0.33.0 to 1.15.0)
- TS-697: GitHub #213 Allow user provided endpoints for the Azure consumers
- TS-705: Update npm packages (@grpc/grpc-js from 1.6.9 to 1.7.1, @grpc/proto-loader from 0.7.0 to 0.7.3, @opentelemetry/api from 1.0.4 to 1.2.0, @opentelemetry/exporter-metrics-otlp-grpc from 0.31.0 to 0.33.0, @opentelemetry/exporter-metrics-otlp-http from 0.31.0 to 0.33.0, @opentelemetry/exporter-metrics-otlp-proto from 0.31.0 to 0.33.0, @opentelemetry/sdk-metrics from 0.31.0 to 0.33.0, eventemitter2 from 6.4.7 to 6.4.9)
- TS-691: DataDog: forward metric data additionally to the logs endpoint
- TS-699: Removed Beacon from documentation
- TS-676: Add gRPC exporter to OpenTelemetry_Exporter (experimental feature)
- TS-678: Add new performance stats - 'system.connectionsPerformance'
- TS-681: Generic_HTTP consumer should allow to tune transport options (experimental feature)
- TS-682: Add 'compressionType' option to Generic_HTTP consumer
- TS-684: DataDog consumer should support proxy configuration
- TS-686: DataDog consumer should split huge payload into smaller chunks according to API docs
- TS-688: DataDog consumer should allow to tune transport options (experimental feature)
- TS-690: Add OpenTelemetry_Exporter raw JSON output
- TS-693: Add secure connection settings to OpenTelemetry_Exporter (experimental feature)
- TS-687: DataDog consumer should send correct timestamp value
- TS-670: GitHub #209 OpenTelemetry_Exporter should set correct 'timestamp'
- TS-669: OpenTelemetry updated to version 0.31.0
- TS-672: Update npm packages (@grpc/proto-loader from 0.6.12 to 0.7.0, eventemitter2 from 6.4.5 to 6.4.7)
- TS-680: 'system.throughputPerformance' in System Poller's default output should match output example in "examples" directory
- TS-685: DataDog consumer should use API v2 for to send data
- TS-668: GitHub #207 Add support for numerical enums, enhance SNMP metrics processing
- TS-666: Created separate Azure Logs tables for pool members and removed that data from the pool tables.
- TS-658: Update npm packages (@grpc/proto-loader from 0.6.9 to 0.6.12)
- TS-568: Added support for SNMP endpoints in Telemetry_Endpoints
- TS-233: GitHub #186: Added outputMode parameter to Generic HTTP consumer to support raw data output
- TS-660 - Metric Consumers should not fail when 'null' found in data
- TS-69 - Prometheus consumer should ignore NaN values
- AUTOTOOL-3077: Update npm packages (fixed medium and low priority vulnerabilities in dependencies - kafka-node)
- AUTOTOOL-3101: GitHub #191: Increased number of messages in a single PutLogEvents command for AWS Logs consumer and added retry logic for InvalidSequenceTokenException.
- AUTOTOOL-2975: Update npm packages (@grpc/proto-loader from 0.6.4 to 0.6.9). Removed unneeded "long" dependency
- AUTOTOOL-3122: Qkview should contain Declaration History file
- Promoted to LTS
- AUTOTOOL-2925: Save recent declarations to a file
- AUTOTOOL-2938: GitHub #184: Added additional sanitation to StatsD metric and tag naming
- AUTOTOOL-3023: RegEx catastrophic backtracking on attempt to mask secrets in JSON data
- AUTOTOOL-2969: GitHub #174: Added info about installed ASM attack signatures
- AUTOTOOL-2959: Update npm packages (@opentelemetry/api from 1.0.3 to 1.0.4, @opentelemetry/sdk-metrics-base from 0.26.0 to 0.27.0, jmespath from 0.15.0 to 0.16.0). Replaced deprecated @opentelemetry/exporter-collector-proto with @opentelemetry/exporter-metrics-otlp-proto 0.27.0
- AUTOTOOL-2976: Replaced grpc-js package by F5-specific grpc-js-0.2-modified
- AUTOTOOL-2979: Update DataDog consumer to use the v2 logs API
- AUTOTOOL-2907: Close Tracer by timeout when inactive
- AUTOTOOL-3038: Add support for ElasticSearch 8
- AUTOTOOL-2801: GitHub #171: Added fqdn property to pool members that use FQDNs
- AUTOTOOL-2890: Update npm packages (@grpc/proto-loader from ~0.3.0 to ^0.6.4)
- AUTOTOOL-2798: GitHub #154: Allow keyless (service account based) authorization for GCP Metrics and Logging consumers.
- AUTOTOOL-2794: GitHub #158: Added "metricPrefix" property to DataDog consumer, which allows a custom prefix to be added to each DataDog metric
- AUTOTOOL-2815: Added the "convertBooleansToMetrics" property to the DataDog, Statsd and OpenTelemetry_Exporter consumers, which determines whether boolean values are converted to metrics
- AUTOTOOL-2842: GitHub #173: Added optional service endpoint URL property to AWS consumers. Used to overwrite the implicit endpoints based on the region.
- AUTOTOOL-2795: GitHub #160: Added "customTags" property to DataDog consumer, which allows specifying custom tags to attach to DataDog metrics and logs
- AUTOTOOL-2800: GitHub #170: Added "poolName" tag to pool member metrics.
- AUTOTOOL-2769: Update npm packages (@grpc/grpc-js from v1.3.4 to v1.4.2, @opentelemetry/exporter-collector-proto from v0.24.0 to 0.25.0, @opentelemetry/sdk-metrics-base from 0.24.1-alpha.4 to 0.26.0, aws-sdk from v2.991.0 to v2.1018.0, eventemitter2 from v6.4.4 to v6.4.5)
- AUTOTOOL-2752: GitHub #159 and GitHub #162: Added "region" property to DataDog consumer
- AUTOTOOL-2753: GitHub #161: Added "service" property to DataDog consumer
- AUTOTOOL-2701: GitHub #156: Added "format" property to Azure_Log_Analytics consumer
- AUTOTOOL-2709: Fixed an issue where Telemetry Streaming would not collect System Poller data if bash was disabled on the BIG-IP device. Properties that require the bash endpoint are now skipped if bash is not available on the target BIG-IP.
- AUTOTOOL-2282: Support ElasticSearch 7 by handling ElasticSearch's 'type' deprecation
- AUTOTOOL-2723: Update npm packages (aws-sdk from v2.980.0 to v2.991.0)
- AUTOTOOL-2492: GitHub #129: Added "throughtputPerformance" stats to "system"
- AUTOTOOL-2574: Added functionality to handle responses from iControlREST that contain duplicate JSON keys
- AUTOTOOL-2663: GitHub #152: Added "isAvailable" and "isEnabled" boolean properties to Virtual Server and update Prometheus output
- AUTOTOOL-2673: Add new OpenTelemetry Exporter consumer (experimental feature), with support for SystemPoller metrics
- AUTOTOOL-2703: GitHub #157 Add compressionType for DataDog consumer
- AUTOTOOL-2679: Add support for converting Event Listener log messages into metrics, in the OpenTelemetry Exporter consumer
- AUTOTOOL-2633: GitHub #148: Fixed issue where Prometheus consumer did not return the correct Content-Type HTTP Header
- AUTOTOOL-2641: GitHub #151: Fixed issue where asmState could report incorrect state value. asmState and lastAsmChange properties are now retrieved from iControlREST
- AUTOTOOL-2626: If event is a well-formed json and "$F5TelemetryEventCategory" has value "raw", then the telemetry does not apply any processing and sends the data as is.
- AUTOTOOL-2670: Update npm packages (aws-sdk from v2.943.0 to v2.980.0)
- AUTOTOOL-622: GitHub #94: Add Data Dog consumer (experimental feature)
- AUTOTOOL-2600: GitHub #125: Added "slowKilled" and "evictedConns" properties to "clientside" of virtual servers
- AUTOTOOL-2605: GitHub #141: Add "eventSchemaVersion" property to F5_Cloud consumer
- AUTOTOOL-2453: GitHub #143: Add new Google Cloud Logging consumer
- AUTOTOOL-2623: Added new property "reportInstanceMetadata" to "Google_Cloud_Monitoring" consumer to control metadata reporting
- AUTOTOOL-2543: GitHub #134: Fixed issue where Prometheus consumer encounters an error when a metric name is registered more than once
- AUTOTOOL-2591: Fixed issue where the Kafka Consumer would create a new Kafka connection on each Telemetry Streaming payload
- AUTOTOOL-2411: The default namespace (f5telemetry_default) is now stated explicitly in traceName and paths of the to the trace files.
- AUTOTOOL-2573: Google Cloud Monitoring Consumer now uses instance_id and zone as time-series resource labels for BIG-IPs that run in Google Cloud
- AUTOTOOL-2613: Update npm packages (@grpc/grpc-js from v1.3.0 to v1.3.4, aws-sdk from v2.914.0 to v2.943.0, statsd-client from v0.4.6 to v0.4.7)
- AUTOTOOL-2555: Add new property "addTags" to StatsD consumer that makes Consumer to send tags along with metrics (experimental feature)
- AUTOTOOL-2555: Fix scrapper code to make StatsD consumer to send more metrics
- AUTOTOOL-2510: Update npm packages (@f5devcentral/f5-teem from v1.4.6 to v1.5.0, aws-sdk from v2.885.0 to v2.914.0, mustache from v4.1.0 to v4.2.0, statsd-client from v0.4.5 to v0.4.6)
- Replace grpc-js-0.2-modified with @grpc/grpc-js
- AUTOTOOL-2535: Refactor Google Cloud (used by the Google Cloud Monitoring Consumer) authentication, and add support for caching short-lived authentication tokens
- Promoted to LTS
- AUTOTOOL-2311: GitHub #119: Added Capacity_Float to system.diskStorage
- AUTOTOOL-2330: GitHub #74: Added system.configSyncSucceeded to default output and f5_system_configSyncSucceeded to Prometheus output
- AUTOTOOL-2439: Added data tracing and 'success' message to Google Cloud Monitoring consumer
- AUTOTOOL-2312: GitHub #114: Added support to the Generic HTTP Consumer for additional modification of data payloads, including the addition of the 'JMESPathQuery' action
- AUTOTOOL-2456: Added new Event Listener feature to trace incoming data and save it to a file for further debugging
- AUTOTOOL-2451: GitHub #127: Fixed issue where 'long' fields in ASM events were causing ASM events to become malformed. Individual event fields are now permitted to be <= MAX_BUFFER_SIZE (16k)
- AUTOTOOL-2464: AWS S3 date path was incorrect
- AUTOTOOL-2488: Should not create multiple Tracer instances that point to the same file
- AUTOTOOL-2297: Update npm packages (aws-sdk from v2.854.0 to v2.885.0)
- AUTOTOOL-2280: GitHub #79: Expose totNativeConns and totCompatConns on SSL profile stats
- AUTOTOOL-2160: Added monitor checks for memory thresholds, which prevents restnoded from crashing on reaching memory limits (runs by default)
- AUTOTOOL-1915: Added new Event Listener endpoint (/eventListener/$eventListener and /namespace/$namespace/eventListener/$eventListener) to send a debug message to an Event Listener
- AUTOTOOL-2266: Added ability to configure "compressionType" for Splunk consumer
- AUTOTOOL-2314: GitHub #120 Added 'swap' metric to 'system' group (System Poller output)
- AUTOTOOL-2115: Namespaces support for iHealth Poller
- AUTOTOOL-494: iHealth Poller creates corrupted Qkview files
- AUTOTOOL-2164: GitHub #108 Fixed Route Domain ID handling for Virtual Server and Pool stats
- GitHub #109: Splunk debug output does not log HTTP Response payload
- AUTOTOOL-2304: Update example output for maxConns, pktsIn, pktsOut, and totConns
- AUTOTOOL-494: iHealth Poller creates corrupted Qkview files
- AUTOTOOL-2310: Tracer unable to access destination directory
- AUTOTOOL-2368: Tracer(s) from other namespaces should not stop when namespace-only declaration posted
- AUTOTOOL-2376: iHealth Poller downloadFolder should not be required on local device
- AUTOTOOL-2091: Trace full payload sent to StatsD Consumer
- AUTOTOOL-2250: System Poller recurring scheduling changed so that System Poller executions do not overlap
- AUTOTOOL-2213: Update npm packages (applicationinsights from v1.8.9 to v.1.8.10, aws-sdk from v2.830.0 to v2.854.0, eventemitter2 from v6.4.3 to v6.4.4, google-auth-library from v6.1.4 to v.6.1.6, lodash from v.4.17.20 to v4.17.21)
- AUTOTOOL-1509: Remove excessive data formatting from tracer
- AUTOTOOL-1987: Added new Namespace declare endpoint (/namespace/$namespace/declare) which supports POST and GET
- AUTOTOOL-2148 GitHub #104: Add support for TLS Client Authentication to Generic HTTP Consumer
- AUTOTOOL-1710: Fix Event Listener startup errors that might cause restnoded to crash
- AUTOTOOL-2227: Splunk multiEvent format should ignore 'References'
- AUTOTOOL-2111: Update npm packages (applicationinsights from 1.8.7 to 1.8.9, aws-sdk from 2.775.0 to 2.830.0, google-auth-library from 6.1.1 to 6.1.4, mustache from 4.0.0 to 4.1.0)
- AUTOTOOL-2212: Add AWS specific certificates to AWS Consumers
- AUTOTOOL-2027 GitHub #91: Add custom timestamp for APM Events
- AUTOTOOL-2043: GitHub #92 Add support for Generic_HTTP proxy options
- AUTOTOOL-1834: Updated config diff handling for namespace endpoint support
- AUTOTOOL-1847: Added 'multiMetric' format for Splunk consumer (v8+ only) (experimental feature)
- AUTOTOOL-1988: Added new Namespace SystemPoller endpoint (/namespace/$namespace/systempoller/$systemOrPoller/$poller?)
- AUTOTOOL-2056 GitHub #93: Add additional traffic stats for virtual servers and pools
- AUTOTOOL-1984 GitHub #85: Add support for Splunk proxy options
- AUTOTOOL-2134: New consumer: F5_Cloud
- AUTOTOOL-2028 GitHub #90: Add support for Kafka TLS client authentication
- AUTOTOOL-2089: Fix error where unavailable Custom Endpoint would return HTTP 500
- AUTOTOOL-1832: telemetry/declare returns HTTP 503 on attempt to post declaration while previous one is still in processing
- AUTOTOOL-1983: No major updates for npm packages (package-lock.json updates only).
- AUTOTOOL-1914: Store up to 10 items in trace file
- AUTOTOOL-2086: Add deprecation notice for Splunk Legacy format
- AUTOTOOL-1829: Add support for the Telemetry_Namespace class in declarations posted to the /declare endpoint
- AUTOTOOL-1830: Added new Namespace PullConsumer endpoint (/namespace/$namespace/pullconsumer/$consumer)
- AUTOTOOL-1967 GitHub #86: Fix bug when TS incorrectly processing iRule stats with multiple events
- AUTOTOOL-1874: Update npm packages (ajv from v6.12.4 to v6.12.6, applicationinsights from v1.8.6 to v1.8.7, aws-sdk from v2.749.0 to 2.775.0)
- AUTOTOOL-1961 GitHub #84: Include _ResourceId if available for Azure Log Analytics consumer. Also, metadata calls are now only happening upon consumer(s) load.
- AUTOTOOL-1833 and AUTOTOOL-1908: Update declaration config parsing and use new normalized configs for components
- AUTOTOOL-1831: Update forwarder to use destinationIds for consumer lookup
- AUTOTOOL-1790: Fixed syslog event hostname parsing for VCMP hosts
- AUTOTOOL-1768: Resolve memory leak in ElasticSearch consumer, by replacing 'elasticsearch' library with 'request' library
- AUTOTOOL-1747: Update npm packages (ajv from v6.12.3 to v6.12.4, applicationinsights from v1.8.5 to v1.8.6, aws-sdk from v2.728.0 to 2.749.0, lodash from v4.7.19 to 4.7.20)
- AUTOTOOL-1894: Update default hostname for StatsD consumer
- AUTOTOOL-1648: Added ability to specify fallback hosts for Generic HTTP consumer (experimental feature)
- AUTOTOOL-1627: Add support for AWS CloudWatch Metrics
- AUTOTOOL-1679: Added 'cipherUses' stats for Client and Server SSL profiles
- AUTOTOOL-810: Fixed Event Listener parsing when received multiple events
- AUTOTOOL-1680: Update npm packages (ajv from v6.12.2 to v6.12.3, applicationinsights from v1.8.0 to v1.8.5, aws-sdk from v.2.704.0 to 2.728.0, statsd-client from v0.4.4 to v0.4.5)
- AUTOTOOL-1640: Add statusReason and monitorStatus fields to System Poller output for BIG-IQ Analytics
- AUTOTOOL-1192: Fix inconsistency in GSLB output: return empty object instead of 'undefined'
- AUTOTOOL-1639: Fix Azure consumer memory leak when calling metadata service on an instance where it is unavailable
- AUTOTOOL-1503: GitHub #39: Update Azure Log Analytics dashboard example
- AUTOTOOL-1695: Fix lodash Prototype Pollution vulnerability
- AUTOTOOL-1694: Fix Splunk legacy format missing poolMemberStat data
- AUTOTOOL-1547: aws-sdk updated to 2.704.0
- AUTOTOOL-1616: Increased code coverage: Event Listener unit tests
- AUTOTOOL-1169: GitHub #9: Add new Prometheus Pull Consumer
- AUTOTOOL-1449: Add information about inlined pollers (System Poller and iHealthPoller) to TEEM report
- AUTOTOOL-1340: GitHub #41: Add support for IAM Roles with AWS_S3 Consumer
- AUTOTOOL-1538: Add TCP support to StatsD consumer
- AUTOTOOL-1506: Added a timeout to Azure metadata service HTTP requests
- AUTOTOOL-557: GitHub #18 Fix renameKeys for networkInterfaces with multiple digits
- AUTOTOOL-1432: Update npm packages (aws-sdk from v2.659 to v2.664 and default mocha from v5.2.0 to v7.1.2)
- Updated @f5devcentral/f5-teem package dependency to 1.4.6
- AUTOTOOL-1448: Introduce new Pull Consumer class, Pull Consumer API, and 'default' Pull Consumer
- AUTOTOOL-1187: Enable use of Managed Identities for Azure Log Analytics
- AUTOTOOL-1186: New consumer Azure Application Insights
- AUTOTOOL-1218: Enable use of Managed Identities for Azure Application Insights
- AUTOTOOL-768: Enable support of AzureGov for Azure consumers and optional region property
- AUTOTOOL-1334: Fix Regular Expression Denial of Service vulnerability and improve start up time on node v4.x and v6.x
- AUTOTOOL-1150: GitHub #30 Fix error when Splunk consumer (configured with 'legacy' format) tries to forward event from Event Listener
- AUTOTOOL-491: GitHub #17 Fix crash in Kafka consumer on attempt to close idle connections to brokers
- AUTOTOOL-1265: Reduce data copying in System Stats
- AUTOTOOL-1266: Reduce data copying in Event Listener
- AUTOTOOL-1376: Rename Google StackDriver consumer to 'Google Cloud Monitoring'
- AUTOTOOL-1015: Update f5-teem to 1.4.2 for new reportRecord API
- AUTOTOOL-1111: Enable configurable polling with Telemetry_Endpoints (BIG-IP paths) and multiple system poller support
- AUTOTOOL-1148: Allow 'OR' logic by adding ifAnyMatch functionality.
- AUTOTOOL-853: Support F5 systems (ex: Viprion) that have multiple hosts
- AUTOTOOL-1051: Event Listener unable to classify AFM DoS event
- AUTOTOOL-1037: Splunk legacy tmstats - include last_cycle_count
- AUTOTOOL-1019: Splunk legacy tmstats - add tenant and application data
- AUTOTOOL-1128: Declarations with large secrets may timeout
- AUTOTOOL-1154: Passphrases should be obfuscated in consumer trace files
- AUTOTOOL-1147: Add 'profiles' data (profiles attached to Virtual Server) to 'virtualServers'
- AUTOTOOL-896: GitHub #26: Use baseMac instead of hostname to fetch CM device
- AUTOTOOL-1160: cipherText validation when protected by SecureVault
- AUTOTOOL-1239: Caching data about the host device to speed up declaration processing
- AUTOTOOL-1062: Update NPM packages
- AUTOTOOL-725 and AUTOTOOL-755: Add support for GSLB WideIP and Pools Config and Stats
- AUTOTOOL-1061: Basic auth does not work with ElasticSearch consumer
- AUTOTOOL-1017: Some Splunk legacy tmstats datamodels have period in property name instead of underscore
- AUTOTOOL-1068: The username and passphrase settings for AWS CloudWatch consumers are now optional
- AUTOTOOL-487: Add support for filtering (System Poller and Event Listener)
- GitHub #22: Reference to pools in virtual server data
- AUTOTOOl-905: Add machineId to System Poller output
- AUTOTOOL-817: Add Google StackDriver as a consumer
- AUTOTOOL-784: Improved error handling to preserve stack trace. Problem was discovered on a Viprion. Opened AUTOTOOL-853 to support Viprion in future.
- AUTOTOOL-690: Collect all supported tmstat tables for Splunk Legacy
- AUTOTOOL-748: Add Telemetry Streaming analytics reporting to F5
- AUTOTOOL-763: Added maxSockets for HTTP and HTTPS connections, defaults to 5
- AUTOTOOL-724: Add example using Generic HTTP Consumer to publish data to Fluentd
- Splunk Tmstat table data is being overwritten when forwarded to Splunk
- Fixed broken promise chain when loading config file
- Add support for new tagging API
- Collect tmstat's cpu_info_stat for Splunk Legacy
- Collect mask and ipProtocol for virtual servers
- Collect device groups, ASM state, last ASM change, APM state, AFM state, LTM config time, and GTM config time
- Collect stats for iRules
- Support for CGNAT events logs
- Added "allowSelfSignedCert" to the documentation examples (closing #14), and added an associated troubleshooting entry in the documentation on clouddocs.f5.com
- Elastic Search Unable to parse and index some messages with previously used keys
- Elastic Search event data objects containing consecutive periods will be replaced with a single period.
- Splunk Host property is null for TS events