Improve send performance (#943) #946
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #946 +/- ##
=======================================
Coverage 84.27% 84.27%
=======================================
Files 87 87
Lines 10039 10040 +1
=======================================
+ Hits 8460 8461 +1
Misses 1579 1579
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thank you! @ods
This LGTM and I measured the performance on the same environment using this modification.
Here is the result.
Running Tests for Kafka-Python
Starting benchmark for Kafka-Python producer.
Kafka-Python producer Results:
Number of Runs: 5, Number of messages: 100000, Message Size: 100 bytes.
Average Time for 100000 messages: 6.893932819366455 seconds.
Messages / sec: 14505.508339025255
MB / sec : 1.3833530749345069
===================
Running Tests for AIOKafka
kafka-client-benchmarks/aiokafka-benchmark.py:40: DeprecationWarning: The loop argument is deprecated since 0.7.1, and scheduled for removal in 0.9.0
producer = AIOKafkaProducer(loop=loop, **producer_config)
Starting benchmark for AIOKafka Producer.
AIOKafka Producer Results:
Number of Runs: 5, Number of messages: 100000, Message Size: 100 bytes.
Average Time for 100000 messages: 1.3094082355499268 seconds.
Messages / sec: 76370.37654494504
MB / sec : 7.283246664518837
===================
Improved more than #942, so I close #942.
BTW, is it possible to make this fix available some way soon?
I'm going to include it into 0.9.0rc1, probably on Wednesday. Then we need some feedback before doing final release, especially from people using snappy and zsdt compression. |
|
Pre-release with fix is ready, please check |
|
Thank you for your work. I confirmed everything worked well! |
Changes
Fixes #943
The results of
benchmark/simple_produce_bench.py: