Skip to content

0.9.1
Compare
Choose a tag to compare
@edenhill edenhill released this 26 May 09:21
· 3061 commits to master since this release
0.9.1
2213fb2

New major features

  • Broker-based balanced High-Level KafkaConsumer (broker version >= 0.9)
  • SASL support (client authentication) - requires Cyrus libsasl
  • SSL support (client authentication, transport encryption) - requires OpenSSL
  • Native Win32 support (Visual Studio)
  • Broker-based quota support (KIP-13)
  • Relative offsets support (KIP-31)
  • Message timestamp support (KIP-32)

New enhancements

  • Broker API version query support for brokers >=0.10.0 with fallbacks for older brokers. (KIP-35)
  • Added broker connection max in-flight enforcement: max.in.flight.requests.per.connection
  • Added config aliases "retries", "acks" (#157)
  • request.required.acks/acks: added support for "all" (-1)
  • Flag config properties now support explicit +addx,-dely and "all"
  • Added rd_kafka_last_error() alternative to errno for legacy APIs (#571)
  • Added rd_kafka_get_err_descs() to retrieve full list of error codes.
  • Added rd_kafka_partitions_pause/_resume()
    • Consumer: flush local fetchq for partition(s) and stop fetching
    • Producer: dont send any ProduceRequests to broker for partition(s)
      This is also used when a rebalance_cb is registered between the time of
      rebalancing and until the application has called assign() to synchronized state.
      Prevents app from receiving messages after a rebalance but before assign()
  • Keep track of app_offset: last message+1 provided to application (issue #106). This is the "next offset" for the application.
  • Added log.connection.close to turn off logging of broker-triggered disconnects (issue #437)
  • Added rd_kafka_mem_free()
  • Formal C and C++ API docs (doxygen)
  • Added RdKafka::Message::topic_name().
    Allows retreiving message topic (name) for auto consumed topics where we dont have a local Topic object
  • Add C++ Metadata API (@eugpermar and @incessantier)
  • Changed topic.metadata.refresh.interval.ms to 5m, must be lower than server idle timeout
  • Alias some new java client properties (bootstrap.servers, enable.auto.commit)
  • Added topic-level "compression.codec" configuration
  • Added "builtin.features" property to allow applications to query for built-in features (e.g., gzip, snappy, zlib)
  • Add Consumer::OffsetTail(). (@bwelling)
  • Removed enforce.isr.cnt - should be handled by broker now and request.required.acks=-1
  • Added rd_kafka_topic_opaque()
  • Added rd_kafka_*_conf_get() for querying conf property values on a conf object
  • New high-level balanced KafkaConsumer (requires broker version >= 0.9)
  • Add consumer queue support for C++. (@rthalley and @macisamuele)
  • Fixed alignment errors on Sparc, et.al.
  • Increased topic.metadata.refresh.interval.ms to the official default 10 minutes (#363)
  • Added config property 'socket.blocking.max.ms' to control max socket poll time
  • Added lazy producer message offset reporting
    When 'produce.offset.report' is off, produced offsets will still be reported or the last message in each produced batch.
    This means an application can still keep track of offsets (though a bit more roughly) without the cost of per-message offset reports.
  • Added rd_kafka_yield() to break out of callback loops
  • Propagate UNKNOWN_TOPIC and UNKNOWN_PARTITION to consumers (issue #348)
  • Handle broker name and nodeid updates (issue #343)
    This also migrates bootstrap brokers to proper brokers if they can be exactly matched by name.
  • Added consume.callback.max.messages (issue #248)
  • Added C++ Consumer::consume_callback as front-end of rd_kafka_consume_callback (kodekarl)
  • Add a built in consistent hashing partitioner (@NZKoz)
  • Add ability to specify a CRL for checking remote certificates (@vincentbernat)

Fixes

  • Preserve configured partition.assignment.strategy order
  • Final offset commits on consumer close() are now limited to session.timeout.ms
  • rd_kafka_log_print should print to stderr, not stdout
  • C++ unassign() assigned an empty list, which is not an unassign..
  • Dont fail/retry unsent requests on broker failure, leave them in queue.
  • Portability fixes (FreeBSD, Win32, MUSL libc, SunOS, AIX)
  • C++: added virtual destructors to all classes that have a virtual function (@danny794)
  • Consumer: fix race condition with start(logical_offset) quickly followed by seek(abs_offset)
  • Maintain per-partition op-version in Fetch requests (issue #586)
  • Richer CONFIGURATION.md format (supported values, value type)
  • Fixed a number of possible hang-on-termination scenarios.
  • Improved handling of failed broker requests (exposed by #529)
  • Fixed cases where a timeout_ms of 0 would serve precisely timed events forever
  • Fixed buffer retry counting
  • Dont rely on partition leader broker to be set when querying metadata (issue #518)
  • Ignore intermittent Topic Metadata failures while brokers are rebalancing. (issue #513)
  • rd_kafka_topic_partition_available() returned 1 for leader-less partitions (issue #513). Regression since 0.8.6
  • Backoff sleep on broker UP -> DOWN state transition was never called. This could trigger busy-looping when connecting to a broken broker.
  • Added configurable reconnect delay with jitter: reconnect.backoff.jitter.ms (issue #403)
  • Snappy endian swapping for older glibc 2.9 (issue #505, #407)
  • Fix possible crash on buf retries when broker is unavailable
  • Consumer errors are now rkmessage-based rather than error_cb
  • Added RD_KAFKA_OFFSET_INVALID to indicate invalid offsets in various APIs
  • Minimize default address caching time to 1s (issue #494)
  • Added rd_kafka_errno() (issue #489)
  • Proper generation and installing of pkg-config .pc files for both C and C++ libraries
  • Report unsupported compression codecs (on consume) as CONSUMER_ERR ops
  • RdKafka::Conf->set "socket_cb" and "open_cb" checked for wrong Conf type
  • Deprecate rd_kafka_set_logger() in favour of rd_kafka_conf_set_log_cb()
  • Dont reset partition count to zero on metadata topic error
    A metadata partition count of 0 is returned by the broker with errors such as "No leader broker" for a topic, since these errors are
    typically intermittent we hold on to the existing partition count.
  • Emit ALL_BROKERS_DOWN if none of the initial bootstrap brokers could be added.
  • Consumer offset fixes (#352, #276, #227)
  • Dont loose messages when partition goes away (and there is no UA partition)
    Previously any messages in a partition's queue would be silently lost if a partition went away (and there was no UA partition).
    These messages are now properly delivery reported with UNKNOWN_PARTITION.
  • Provide rkmessage->rkt topic in dr_msg_cb (#333)
  • Fixed wait in rd_kafka_consume_callback_queue().
    It returned immediately if no messages were available (timeout_ms was not used).
  • Fix occasional invalid json on statistics buffers (@alfred-landrum)
  • dr_msg_cb: rkmessage->partition is now properly set (issue #233)
  • Renamed position() -> committed() and added position() that returns next offset. According to the Java client.
  • Workaround Java client and broker Offset metadata parsing bugs (issue #577)
  • Added rd_kafka_query_watermark_offsets() and rd_kafka_get_watermark_offsets() (issue #503, #530)
  • C++: Added commitSync and commitAsync APIs taking a list of offsets (issue #528)
  • Added enable.auto.offset.store (issue #528)
  • sasl.kerberos.kinit.cmd now takes a command+args string template (issue #635)
  • Send consumer group user_data as empty instead of NULL to workarund broker 0.9.0.0 bug (issue #660)
Assets 2
Loading