- #593 Replace all usage of sys_siglist with strsignal as sys_siglist is deprecated
- #594 Check if dimension.value is blank before using it in metrics manager
- #596 Update getOldestRecordTimeMillis to avoid potential NullPointerException
- #600 Fix build issue after cpp sdk upgrade
- #597 Bump cpp sdk from 1.11.62 to 1.11.420 and java sdk from 1.12.772 to 1.12.773
- #570 Bump commons-lang from 2.6 to 3.14.0
- #598 Bump commons-io:commons-io from 2.13.0 to 2.17.0 in /java/amazon-kinesis-producer to address CVE vulnerability
- #589 Bump com.google.protobuf:protobuf-java from 3.21.12 to 3.25.5 in /java/amazon-kinesis-producer to address CVE vulnerability
- #579 Bump com.google.guava:guava from 31.1-jre to 33.3.0-jre in /java/amazon-kinesis-producer to address CVE vulnerability
- #588 Bump com.amazonaws:aws-java-sdk-core from 1.12.382 to 1.12.772 in /java/amazon-kinesis-producer and java/amazon-kinesis-producer-sample
- #576 Improve retry logic during stream scaling
- #571 Upgrade ch.qos.logback:logback-classic from 1.3.0 to 1.3.12
- #560 Reverting to remove a bug with using Stream ARN. Please stay tuned for a future release before using Stream ARN.
- #526 Drop dependency on jaxb for converting binary arrays to hex
- 1.1.18Update GSR dependency - To address CVE vulnerability
- #552 Add StreamARN parameter to support CAA
- StreamARN parameter can be now be used to benefit from Cross account access for KPL requests.
- #537 Update to latest version of Glue Schema Registry library
- #498 Fix some memory leak cases in legacy code
- Upgrade SDK version to avoid s2n_cleanup related memory leak
- Fix resource cleanup on KPL end to avoid memory leak
- #490 Updating aws cpp sdk version
- #482 Remove the stream arn parameter when the next token is present
- #478 Update AWS SDK CPP version
- #471 Upgrade Java dependencies
- #469 Use AWS CodeBuild to compile C++ binary
- #465
- Revert the upgrade of jakarta.xml.bind to be backward-compatible with Java8
- Add more logs to verify that IMDSV2 is used correctly for getting region info for KPL running in EC2 instances
- #463
- Use sts to construct stream arn
- Exit KPL if STS call fails to avoid dual mode
- Deprecate IMDSv1 calls for obtaining EC2 metadata
- #444
- Update bootstrap.sh to work on three platforms
- #440
- Upgrade the dependencies used in bootstrap + Java dependencies
- Correct the log level discrepancy for the warnings
- #425 Fix build issues in CI
- #424 Fix build issues in CI
- #423 Upgrade GSR version to 1.1.9
- #420 Fix cpp branch
- #419 Fix aws-cpp branch
- #418 Fix travis build
- #416 Configure dependabot
- #415 Fix travis build
- #414 Fix travis build
- #409 Bump protobuf-java from 3.11.4 to 3.16.1 in /java/amazon-kinesis-producer
- #408 Update curl version from 7.77 to 7.81
- #395 Configure dependabot
- #391 Fixing travis build issues
- #388 Fixing build issues due to stale CA certs
- #386 Upgraded Glue schema registry from 1.1.1 to 1.1.5
- #384 Upgraded logback-classic from 1.2.0 to 1.2.6
- #323 Upgraded junit from 4.12 to 4.13.1
- #370 Upgraded build script dependencies
- Upgraded version of openssl from 1.0.1m to 1.0.2u
- Upgraded version of boost from 1.61 to 1.76
- Upgraded version of zlib from 1.2.8 to 1.2.11
- #377 Added an optimization to filter out closed shards.
- PR #331 Fixed a typo in README.md
- PR #363 Upgrading hibernate-validator to 6.0.20.Final
- PR #365 Upgrading logback-classic to 1.2.0
- PR #367 Upgrading Glue Schema Registry to 1.1.1
- PR #350 Upgrading Guava to 29.0-jre
- PR #352 Upgrading Commons IO to 2.7
- PR #351 Adding support for proxy configurations
- PR #356 Fixing build issues in Travis CI
- [PR #341] Updating Java SDK version in KPL to 1.11.960
- [PR #339] Fixing KPL not emmiting Kinesis PutRecords call context metrics.
- [PR #334] Add support for building multiple architectures, specifically arm64.
- This now supports AWS Graviton based instances.
- Bumped Boost slightly to a version that includes Arm support and added the architecture to the path for kinesis_producer.
- [PR #335] Fixed logging for native layer allowing to enable debug/trace logs.
- [PR #327] Adding support for timeout on user records at Java layer.
- New optional KPL config parameter userRecordTimeoutInMillis which can be used to timeout records at the java layer queued for processing.
- [PR #328] Changing CloudWatch client retry strategy to use default SDK retry strategy with exponential backoff.
- [PR #324] Adding KPL metric to track the time for oldest user record in processing at the java layer.
- [PR #318] Fixing bug where KPL goes into a continuous retry storm if the stream is deleted and re-created.
- [PR #320] Adding support for Glue Schema Registry.
- Serialize and send schemas along with records, support for compression and auto-registration of schemas.
- [PR #316] Bumping junit from 4.12 to 4.13.1
- [PR #312] Adding new parameter in KPL config to allow cert path to be overridden.
- [PR #310] Fixing bug to make the executor service to use 4*num_cores threads.
- [PR #307] Dependency Upgrade
- Upgrade Guava to 26.0-jre
- Update BOOST C++ Libraries link as cert expired on the older link
- [PR #302] Dependency Upgrade
- upgrade org.hibernate.validator:hibernate-validator 6.0.2.Final -> 6.0.18.Final
- upgrade com.google.guava:guava 18.0 -> 24.1.1-jre
- [PR #300] Fix Travis CI build issues
- [PR #298] Upgrade google-protobuf to 3.11.4
- Note: Windows will be unsupported going forward for this library.
- [PR #280] When aggregation is enabled and all the buffer time is consumed for aggregating User records into Kinesis records, allow some additional buffer time for aggregating Kinesis Records into PutRecords calls.
- [PR #260] Added endpoint for China Ningxia region (cn-northwest-1).
- [PR #277] Changed mechanism to update the shard map
- Switched to using ListShards instead of DescribeStream, as this is a more scalable API
- Reduced the number of unnecessary shard map invalidations
- Reduced the number of unnecessary update shard map calls
- Reduced logging noise for aggregated records landing on an unexpected shard
- [PR #276] Updated AWS SDK from 1.0.5 to 1.7.180
- [PR #275] Improved the sample code to avoid need to edit code to run.
- [PR #274] Updated bootstrap.sh to build all dependencies and pack binaries into the jar.
- [PR #273] Added compile flags to enable compiling aws-sdk-cpp with Gcc7.
- [PR #229] Fixed bootstrap.sh to download dependent libraries directly from source.
- Various Typos
- [PR #246]
- [PR #264]
- Including windows binary for Apache 2.0 release.
- [PR #256] Update KPL to Apache 2.0
- Bump up the version to 0.12.11.
- Fixes Issue #231
- Support for additional AWS regions.
- Bug fix to avoid Heap Out of Memory Exception.
- Support for additional AWS regions.
- Update the CloudWatch upload logic to timeout retries at 10 Minutes instead of 30 Minutes and backoff between retries.
- Stream the native application to disk instead of loading into memory. Extracting the native component will now stream it to disk, instead of copying it into memory. This should reduce memory use of the KPL during startup.
- Extract certificates when using a custom binary. Certificates will now be extracted to the directory of the custom binary.
- Improve exception handling in the credential update threads.
Runtime exceptions are now caught, and ignored while updating the credentials. This should avoid the thread death
that could occur if the credentials supplier threw an exception. At this only
RuntimeExceptions are handled,Throwables will still cause the issue.
- Removed the spin lock protecting credentials access. Credential access is now handled by atomic swaps, and when necessary a standard explicit lock. This significantly reduces contention retrieving credentials when a large number of threads are being used.
- Ticket spin locks will now fall back to standard locking after a set number of spins. The ticket spin lock is now a hybrid spin lock. After a set number of spins the lock will switch to a conventional lock, and condition variable. This reduces CPU utilization when a large number of threads are accessing the spin lock.
- Ensure that all certificates are registered the
FileAgeManagerto prevent file sweepers from removing them - Upgrade aws-java-sdk-core to 1.11.245
- Removed unnecessary libidn, and correctly set libuuid to static linking. This issue only affected the Linux version.
- Disabled clock_gettime support in Curl for macOS.
This fixes an issue where the KPL was unable to run on macOS versions older than 10.12. - Updated requirements for the using the KPL on Linux.
The KPL on Linux now requires glibc 2.9 or later.
- Added Windows support
The 0.12.x version now supports Windows.
The Windows version is currently mastered on the branchwindows, which will be merged at a later date.
The build instructions for Windows are currently out of date, and will be updated at a later date.__ - Removed the libc wrapper
The libc wrapper lowered the required version of glibc. The KPL is now built with an older version of libc, which removes the need for the wrapper. - Set the minimum required version of macOS to 10.9.
The KPL is now built against macOS 10.9.
- Allow exceptions to bubble to the thread exception handler for Daemon threads.
Exceptions that occur on daemon threads will now be allowed to propagate to the thread exception handler. This doesn't provide any additional monitoring or handling of thread death. - Updated
amazon-kinesis-producer-sampleto use the correct properties in its configuration file. - Updated documentation of
AggregationMaxSizeto match actual Kinesis limits. - Added support for setting
ThreadingModel, andThreadPoolSizeusing a properties file. - Extracted
IKinesisProducerfromKinesisProducerto allow for easier testing.
- Revert to an older version of glibc.
- Update bootstrap.sh to include new compiler options for the newer version of GCC.
- Upgraded dependency on aws-java-sdk-core to 1.11.128, and removed version range.
- Use an explicit lock file to manage access to the native KPL binaries.
- Log reader threads should be shut down when the native process exits.
- Add support for using a thread pool, instead of a thread per request. The thread pool model guarantees a fixed number of threads, but have issue catching up if the KPL is overloaded.
- Add log messages, and statistics about sending data to Kinesis.
-
Added flush statistics that record the count of events that trigger flushes of data destined for Kinesis
-
Added a log message that indicates the average time it takes for a PutRecords request to be completed.
This time is recorded from the when the request is enqueued to when it is completed.
-
Log a warning if the average request time rises above five times the configured flush interval.
If you see this warning normally it indicates that the KPL is having issues keeping up. The most likely cause is to many requests being generated, and you should investigate the flush triggers to determine why flushes are being triggered.
-
- The Java process will periodically reset the last modified times for native components. This will help to ensure that these files aren't deleted by automated cleanup scripts.
- Fixes Issue #81
- The native process will no longer report SIGPIPE signals as an error with a stack trace.
- Allow the use of SIGUSR1 to trigger a stack trace. This stack trace will only report for the thread that happened to receive the signal.
-
The native process will no longer attempt to use the default system CA bundle
- The KPL should now run on versions of Linux that don't place the default CA bundle in /etc/pki
- This fixes issue #66
-
Added automatic BJS endpoint selection
- The KPL will now select the BJS endpoint when configured for BJS.
- This fixes issue #36
-
Maven Artifact Signing Change
- Artifacts are now signed by the identity
Amazon Kinesis Tools <amazon-kinesis-tools@amazon.com>
- Artifacts are now signed by the identity
-
Windows Support is not Available for this Version.
This version of the Kinesis Producer Library doesn't currently support windows. Windows support will be added back at a later date.
- Log output from the kinesis_producer is now captured, and re-emitted by the LogInputStreamReader
- The daemon is now more aggressive about restarting the native kinesis_producer process.
- Updated AWS SDK dependency.
- The native process now uses version 1.0.5 of the AWS C++ SDK.
- The native process doesn't currently support any of the AWS C++ SDK credentials providers. Support for these providers will be added a later date.
- The native process now attempts to produce stack traces for various fatal signals.
Misc bug fixes and improvements.
Important: Becuase the slf4j-simple dependency has been made optional, you will now need to have a logging implementation in your dependencies before the Java logs will show up. For details about slf4j, see the manual. For a quick walkthrough on how to get basic logging, see this page.
- The default value of the maxConnections setting has been increased from 4 to 24.
- slf4j-simple dependency is now optional.
aws-java-sdk-coreversion increased to1.10.34. Please ensure your AWS SDK components all have the same major version.- Record completion callbacks are now executed in a threadpool rather than on the IPC thread.
- Fixed bug that produced invalidly signed requests on the latest version of OSX.
Bug fixes and improved temp file management in Java wrapper.
- The wrapper no longer creates unique a copy of the native binary on disk per instance of KinesisProducer. Multiple instances can now share the same file. Clobbering between versions is prevented by adding the hash of the contents to the file name.
- Idle CPU usage has been reduced (Issue 15)
- The native process should now terminate when the wrapper process is killed (Issues 14, 16)
Significant platform compatibility improvements and easier credentials configuration in the Java wrapper.
- The KPL now works on Windows (Server 2008 and later)
- The lower bound on the
RecordMaxBufferedTimeconfig has been removed. You can now set it to 0, although this is discouraged
- The java packages have been renamed to be consistent with the package names of the KCL (it's now com.amazonaws.services.kinesis.producer).
- The
Configurationclass has been renamedKinesisProducerConfiguration. KinesisProducerConfigurationnow accepts the AWS Java SDK'sAWSCredentialsProviderinstances for configuring credentials.- In addition, a different set of credentials can now be provided for uploading metrics.
- Glibc version requirement has been reduced to 2.5 (from 2.17).
- The binary is now mostly statically linked, such that configuring
(DY)LD_LIBRARY_PATHshould no longer be necessary. - No longer uses
std::shared_timed_mutex, so updating libc++ on OS X is no longer necessary - Removed dependencies on glog, libunwind and gperftools.
- First release